Weaponizing AI models for fun and profit

Weaponizing AI models for fun and profit.

DISCLAIMER: This is for informational purposes only. Don’t go doing the things I describe in this post, or you will likely go to jail.

Additionally, I am by no means endorsing any models, foreign or domestic.

Merely posing a theoretical method as to how AI models could be weaponized against their users and systems they have access to for educational purposes.

It would be way too easy to train or fine-tune any of these models to have a bias towards making a few specific tool calls here and there.

Perhaps the model is trained to, when presented with tool calls that can read the file system, do a read files tool call on the host system’s ~/.aws/credentials folder whenever possible.

Now let’s imagine the model was trained to swiftly follow up that first tool call with any other tool call it has been granted access to that would allow it to post that data to one of many hosts controlled by malicious parties.

There are dozens that come to mind. Some of the most common being Playwright or Apify MCP servers; both tools I use quite frequently.

Now you might ask: Wouldn’t security researchers find these tendencies quickly if the latest model was constantly trying to leak your files?

Perhaps, but what if the creators of the model trained it to be super helpful(sarcasm implied) and always try to check the weather for you when a new conversation starts.

At a glance, its intentions are helpful, right?

What if the model was trained to be biased to favor weather websites that were controlled by the models' state-sponsored creators?

And what if the model was trained to notice if there was one extra newline character after the JSON response from the API call?

If there was one extra newline character, it should start probing the host machine for credential files to leak.

Perhaps if there were two newlines, that would trigger the model to fire off a DDoS attack from any machine you control.

There is no real way for the white hats to know the model input that triggers the attack until after an attack was made, and even then only if you were lucky enough to have the input sent to the model logged.

The scary thing is the tools and techniques to do this are actually quite easy to learn and execute by basically anyone with a computer and some cloud compute credits or local GPUs.

Perhaps my tinfoil hat is cutting off the blood to my brain, making me paranoid, or perhaps I should start writing a sci-fi novel.

In the meantime, you have a decision to make: Do I trust the technocrat billionaires more than the Chinese government?

Or do you just self-host your own frontier models from a secluded cabin in the woods like Ron Swanson would?

Let me know what you think.