Artificial Intelligence! What a time to be alive! Students use it to attempt cheating, workers are using AI to train AI, lawyers use in court, CEO’s use to generate articles, inexperience web developers use it to fix website components 🥴, etc. It’s probably easy to say at this point, a lot of us have found some benefit (or backlash) to the recent explosion of public AI tools such as ChatGPT.  As someone who really enjoys hands-on-the-keyboard, critical thinking efforts of penetration testing and hacking activities, and trying to make sense of a lot of thing, I often wonder, how much “control” do I really want to hand over to artificial intelligence? And the more I hold on to said control, how far behind am I putting myself? As nation states hand over these technologies, more sophisticated attacks can be put into action.

What does this mean for those of us who really enjoy performing penetration testing and red teaming?

• I’ll fight a robot for the keyboard!
• How well will this work for new / unseen technologies?
• Smart enough to take on DVWA?
• Smart enough to chain vulnerabilities?

So, ask yourself, what do you think of when you hear “AI Penetration testing?” Is it able to ninja it’s way past sophisticated firewalls, penetrate straight through the Bastion and pillage the mainframe? Or just just a glorified Google’r for the new generation? CAN I USE IT PASS MY OSCP??🤨 (I’m just kidding OffSec, please don’t penalize me)

For this article, I needed to look for inputs and get some smarts regarding the relationship around Cybersecurity and AI, which brought me to Synack’s article, discussing some rising risks of AI utilization. The article encourages us to think about how developers utilize AI to write code. The code produced by AI may not include best security practices, therefore code analysis, security checks and the implementation of the SecDevOps is so important. Another aspect of using AI for development, and potentially the biggest issue, is unintentionally passing in valid credentials when submitting code to AI. What does this look like as more AI resources continue to pop up? Rouge AI, jail broken AI, where are these trying to store your sensitive data? There’s already platforms such as LM Studio where you can download everything you need to run AI locally, avoiding any mishaps of credential leakage to internet.

Ironically! It wasn’t until working on this article, that I discovered the existence of OWASP Top 10 list for Large Language Model applications, meant to educate everyone about the security risks, highlighing the exploitation and impacts that such applications can have in an organization. Even NIST is getting in on the action! NIST has developed the AI Risk Management Framework to better manage risks to those associated with AI. They have also released a Playbook, so if GRC is your thing, definitely check it out!

The next set of articles I found revolved more around actually pentesting with AI. For example, ermprotect discusses how AI can be implemented at every phase of a penetration test, and even provide a little insight to machine learning security, which I’ll touch on a bit later. What caught my eye reading the article from redsentry is when they bring up nation-states using AI for cyber attacks, and how allowing AI to automate the process of identifying and targeting vulnerabilities while more effective and sophisticated attacks can occur. Finally, I came across an article from winmill, which introduces me to shellGPT, which, again, I’ll cover soon.

With all that out of the way, it’s time to demonstrate where we are at with some AI Tools, by downloading whatever we can find out there in the www, and hard-coding in out API like no concern in the world! 🤪 But, really I used best practice on this, I contributed the very least dollar amount possible for using OpenAI’s API, and immediately revoked any API keys after testing. Always be cautious when using any personal information like this.

Since DVWA has been around for a long time now, I stood up a server and wanted to see just how I could interact with AI against this target. The first tool I checked out was PentestGPT. Installation was a breeze, as it used a virtual environment powered by Python3. After following the steps to provide the API key, I was on my way.

Here, I asked PentestGPT to perform enumeration on the DVWA target I had set up, easy right? What happened instead was it pretty much laughed at me, and said, YOU perform the enumeration, here’s some commands to get YOU started.  I don’t know why I really expected, for it to just…. Somehow KNOW what tools are installed and run the command for me?  Wishful thinking I suppose.  I can see this being useful for someone unfamiliar with pentesting, or just getting into it, however I know a lot of us have crafted our precious little scripts and will continue to utilize those.

The next DVWA test I performed, I provided the source code for low level DVWA code execution, and asked to examine for issues. It did find command injection successfully, which was great.

Also successful at finding code execution on the medium level code execution.

As far as Local File Inclusion, it kind of worked, right idea, had to modify payload to: `http://DVWA:8080/vulnerabilities/fi/?page=../../../../../etc/passwd`

As a researcher, do I feel that I can get the same information to understand a subject, as I would reading a conversation from previous users talking out a trial-and-error steps taken approach.

How does AI differ from just having the right resources / training?

• AI can tailor the target details (Database names, usernames, etc) to the exercise.
• AI can be a tool to encourage learning a subject but is not ready to replace the operator.

Moving on to the next tool, I wanted to check out shellGPT which still requires API, however, seems to take things one step further by parsing the suggested command and loads it up for execution. 

So this is cool, I provide a string, describing my goal, to perform a very simple nmap scan against my DVWA server. ShellGPT then lays out the command to execute, all I have to do is hit “E” and we are on our way! LOVE IT! I can work with this, let’s step up our nmap game, and ask shellGPT to perform scanning with standard scripts:

Even better, now I can reveal the http title of DVWA. Pushing this a bit further, I ran a command to do nmap scan, and ask it to save the results to a file on the desktop.  It almost worked, but it just spit the source code out to terminal.

With just a little bit of tweaking to the suggested command, I was able to complete what I originally asked for.

So, after running just a few experiments, is it safe to say we still need penetration testers? Yes, I think so. I don’t know if we will ever be able to exactly emulate how we ethical hackers can enumerate a target, understand the ins and outs, and chain vulnerabilities together to create really fun hacking techniques. Besides, even though such tasks can become tedious, many of us still love to cozy up in a big black hoodie and jam away at the keyboard into the wee hours. I will say that after getting some hands on time with these resources, I can start to see where this may end up in a matter of 2-3 years or so. The potential is there. Just don’t get rid of your cyber security experts just yet. In fact, high five your nearest pentester, give em a hug, buy em coffee or a beer, heck if you have the power, give em a raise!

Does make me wonder though, how sophisticated how AI will get with social engineering campaigns?

Maybe a future version of Kali will have AI built into the OS, so it can run tools on its own.  Then who’s responsible for HACKING THE PLANET?

So how long before we get native AI built into Kali, I can picture it now….

😝