Intro
The moment that I learned of the OSAI certification I knew it was one of those I wanted to get right away. As seen on the news and probably on our regular lives, the use of AI have increasingly be one where having it, using it, and integrating it has become our new normal. The thing that still bugs me is that there are so many unknowns, the technology is always changing, it is constantly evolving. Therefore, how can we secure it? How can we help our organizations by using offensive security skills to help safeguard their important data? Well, that is what OSAI helps to answer.
The Course
The OSAI covers a lot AI related concepts, attack surfaces, techniques to use against traditional AI infrastructure, but the main focus is to help the student understand how the attackers find ways to approach and exploit AI. Early on the course introduce us to the three frameworks, MITRE ATLAS, NVIDIA’s AI Kill Chain and OWASP Top 10 for LLMs. Then there’s a deep dive into how to perform reconnaissance against AI targets, how to attack agents, understanding multi-agent systems, the A2A protocol, then into the data layer with RAG pipelines, MCP and deployment exploits. Sounds like a lot, because well it is! Then, after learning and gaining the basic understanding of AI and LLMs, it’s the traditional Assembling the Pieces module, because it wouldn’t be a OffSec Course without it!
OffSec provided 5 labs to play and tinker with. Which is a great way to practice all you’ve learn! The labs were released a month or so before my 1st attempt (yes , I failed my first time, but more on that later)so I did 3 of the 5 labs in preparation for my first attempt.
My first attempt
A month before my exam, I’m getting my notes ready, making sure my prompts are good, understanding and deep diving into the concepts and the reasoning behind it all, when I decide to take a look at exam guide and watch this OffSec webinar to realize my approach was wrong. So what do I do..
That’s right! Pivot, scratch my plan, I have 2 weeks to make sure I have the AI Operator to help me pass. It is allowed and encourage the use of AI on the Exam. So my thought was, maybe I can ask questions here and there and get the information I need. But, this is a 300 course, who am I kidding? This will be complicated and complexed, so I need to make sure I have don’t get stonewalled by restrictions.
Then it hit me, apply what the course teach and make my own AI Operator, with it’s own persona, it owns set of rules, giving it a background of what it can do and it cannot and then use that in the exam. Brilliant, right!? Easy? Well, no.
In order to get everything right, I had to get my affairs in order, meaning the scripts I wanted my operator to use, the prompts, the scope, the boundaries, reasoning style, guardrails! It was so much fun, to run and test on the labs how my agents were working and interacting with the environment.
On my first attempt I was feeling ready! Then I reached 20 pts fairly early and I was feeling confidence on my approach and my operator. But then I hit a wall, for hours, I mean HOURS! Of nothing.
So then, we had a breakthrough! And we found 20 extra points. Then I ran out of tokens (lol), I was tired, and time expired and … I failed.
So after sleeping and over analyzing every step I took on my exam, I realized I had tunnel vision. I can’t go in to the details on what went wrong, but sometimes hyper fixating on something simple, makes your life ten times harder, so KISS it! If you know, you know.
Second attempt
I had to wait a few weeks for my second attempt, I spent that time fixing everything so my operator was able to work, reviewing all the concepts again, crafted a better methodology and prompts to help my operator. This is a joint effort, so I treated my operator as a teammate.
The day came and I was ready, I had a good Operator that delegated tasks to other agents. Sometimes it was hard to keep track of what was going on, so I made sure my agent was keeping a ledger to help me track what was going on and if I needed to explore a different avenue or different tactic.
And after 24 hours I had enough points to pass and I refused to go to bed before submitting my report. So, I stayed up finished and summitted by report.
And within the daily commotion, one afternoon I got my email! I was ecstatic! But to be honest, this was a lot of fun! The building and strategizing the Operator was such a blast. Everyone that asks me I tell them it is the thing I enjoyed the most.
So, if you are thinking about it, take it! If you are interested in learning about AI and how can it help in your assessments you will not regret it.
Until next time!