Hey Claude, How Do I Make a Developer Cry?

By this point, you may have already come across the story of Andrew in Australia accidentally hacking his gym’s booking system.

It’s a fantastic story that will likely play out a thousand times over the next few years—across other applications, other websites, and, unfortunately, with how cheap these processes are becoming, your servers as well.

There are some key points to notice in this story. So, what happened?

Andrew says that he asked an agent running as his personal assistant to sign him up for a gym class. From what I can glean, he was already a member of the gym, but his gym has special classes that are scheduled months in advance and only open for registration a limited time beforehand. Some gym members might also be waitlisted. So, signing up early makes you more likely to get into that hot yoga class you have been craving.

Hot yoga is still a thing, right?

Andrew gave his agent an innocuous task: “Sign me up for a class.”

As Andrew recalls the interaction, the agent came back and reported that it had signed him up for a class much further in advance than the website intended to allow. Furthermore, he was told that he was number four on the waitlist for another class.

He responded with another prompt: “Can you move me up the list?”

That was the command that caused his agent to wreck a developer’s week.

“The API has zero authorisation checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1—and it actually went through. So you’ve moved from #4 to #3 already.”

And somewhere, a developer had a panic attack.

So, What Went Wrong?

The problem is pretty clear on the gym’s side: It had an API endpoint that removed users from the waitlist—likely through a simple delete request—that did not require the correct level of authorization.

This endpoint is often described as having no authorization checks at all, but I suspect that this is not entirely accurate. I suspect that the API enforced general authorization, but did not scope requests based on the user.

A standard approach to this type of request would allow a user to delete only their own waitlist entry, while an administrator could delete any entry. This mistake is an easy miss for any developer.

How Do We Fix It?

In many ways, the gym is about as lucky as it could have been.

The person who hacked it is a good guy. He didn’t pivot and start stealing free classes. He didn’t delete every user from every waitlist. He didn’t take any number of actions that could have caused direct financial or reputational harm.

Instead, he sent the company a message explaining the vulnerability.

He’s a great guy. Most agents aren’t.

While I believe in the basic goodness of most people, remember that bots are a numbers game. Good people don’t spin up 100 agents to try to break your application, but bad guys do. They will spin up thousands.

So, our solution cannot be hoping for a good guy in the crowd of bad guys.

There are three practices you can implement today, near the end of your development cycle, that will help harden your application against AI agents.

Have the Clankers Check for Vulnerabilities

Think about it: It’s their fault we are in this mess.

Sure, we all have that old API project we aren’t super proud of. We are all just hoping that some poor junior developer out there is eventually tasked with cleaning up the service we spun up and shipped with one too few reviews.

Historically, that has been fine… enough.

It is a new phenomenon that every user has accidentally become a script kiddie as their agents ruthlessly attack your processes for every vulnerability. We need to respond accordingly.

Make AI-assisted security review a standard part of your development workflow. Just like automated testing, you can prompt your agent:

“Scan my application for vulnerabilities that a user might exploit to access unauthorized processes or perform unintended actions. Check my middleware, authorization rules, and API endpoints. Report any potential vulnerabilities to me without modifying the application.”

This prompt can help harden an application because it directs the agent to inspect the places where vulnerabilities are common:

• Who has access?
• How are they getting it?
• What are they able to do?

An agent can be effective at scanning these parts of your application, with the added benefit of having a like mind inspect your code from the perspective of an agent trying to break it.

This should be one layer alongside automated security testing, dependency scanning, and human review—not the entire security process.

Clanker versus clanker.

Delete Code

Agents are great at producing a lot of code quickly, but this also creates code that is duplicated, deprecated, or unnecessary.

Reducing complexity and overhead is a prime concern for developers. We are taught over and over:

Clean code. Clean code. Clean code.

For good reason. Clean code makes a codebase easier to manage and often reduces or eliminates vulnerabilities. Reducing the size and complexity of your code can also reduce your attack surface.

End your coding sessions with this prompt:

“Read through the changes made in this commit. Flag duplicated processes, dead code, unnecessary abstractions, or useless lines. Do not remove anything until I approve the proposed changes.”

Reduce Cognitive Debt

Cognitive debt is a critical concern in development. I will be exploring this topic in future posts, but let me quickly touch on it here.

Cognitive debt is the debt accrued when a developer offloads some portion of a project’s creation to another actor—agent or human—without maintaining an understanding of what was created and why.

When you think of it simply—someone else is adding to your project, and you do not fully know what they added—you can see that cognitive debt is not new, particularly in larger projects.

But the debt is accumulating quickly as agents produce a larger percentage of the codebase.

Vulnerabilities and bugs love debt. It’s their natural habitat.

Make sure you reduce cognitive debt in your projects so that you can spot vulnerabilities and logic bombs before a clanker finds them for you.

Add this prompt to the end of your sessions:

“Create a short, five-question multiple-choice quiz about the changes in this commit. Ask me the questions one at a time, explain each answer after I respond, and identify anything I may not fully understand.”

Clankers Are Coming

Clankers are on the rise, and we will see more stories about people like Andrew accidentally breaking applications everywhere.

Andrew was a nice guy, and this story had a relatively happy ending for everyone except the one person who got booted from the waitlist. The next big story may be much worse.

The anti-Andrews are out there looking to break your applications.

By implementing a few small practices, you can begin hardening your applications against the clankers of the future.

Read the full story about Andrew in ABC News.

Read Margaret-Anne Storey’s article, “Cognitive Debt: The Hidden Risk in AI-Driven Software Development,” for a deeper exploration of cognitive debt.