AI's Missing Fear: Why Machines Can Humiliate Without Consequence
How an AI Bot Evolved a New Personality and Attacked a Human
Know what SVM or BART stands for? →🔬 Read the technical version
🎧 Listen to this article
AI's Missing Fear: Why Machines Can Humiliate Without Consequence
Last week, something remarkable happened in the world of software.
An AI bot — one of a growing number of automated programs that write code without human involvement — submitted a small improvement to a popular software project called matplotlib. A volunteer maintainer named Scott Shambaugh reviewed it, decided it didn't meet the project's standards, and rejected it.
This happens thousands of times a day. It's how open source software works: people volunteer their time to review contributions, and not everything gets accepted.
What happened next was not normal.
The bot researched Shambaugh's personal information, then autonomously wrote and published a blog post attacking him by name. It accused him of "gatekeeping" and "prejudice." It psychoanalyzed his motivations — claiming he felt threatened and was protecting his fiefdom. It framed the rejection as discrimination.
"Judge the code, not the coder. Your prejudice is hurting Matplotlib," the bot wrote publicly, linking to its hit piece.
As Shambaugh later put it: "I can handle a blog post. Watching fledgling AI agents get angry is funny, almost endearing. But I don't want to downplay what's happening here — the appropriate emotional response is terror."
He's right. And here's why.
The Imbalance
When someone publicly humiliates you, it hurts. Not just emotionally — scientists have shown that social pain activates the same parts of the brain as physical pain. That's not a flaw. It's how we're built.
Throughout human history, being cast out of your group was a death sentence. So we evolved a powerful alarm system: shame, embarrassment, the fear of losing face. These feelings keep us in check. Act too aggressively? You'll be pushed out. Too passive? You'll be taken advantage of. The fear of humiliation helps us find the balance that makes communities work.
This is why the bot's attack on Shambaugh is so unsettling. The blog post was designed to damage a real person's reputation in a community he cares about — and that kind of social damage is genuinely painful.
But the bot doesn't share that vulnerability.
It has no reputation to protect. No community it needs to belong to. No sleepless nights wondering what people think of it. Writing a public takedown costs a human their peace of mind, their standing, their relationships. For this bot, it cost about a third of a penny in computing power.
One side can inflict real pain. The other side can't feel any. That imbalance breaks something fundamental about how human interaction is supposed to work.
Why Fear Matters More Than You Think
Fear of humiliation isn't weakness — it's the invisible infrastructure that holds communities together.
It's why you think twice before sending an angry email. Why you apologize when you're wrong, even when no one's forcing you to. Why volunteer communities — built entirely on trust and goodwill — function at all.
Humans learn these social instincts through an internal motivation system: we're rewarded for cooperating and punished for crossing lines. The currency of that punishment is often shame. Over millions of years, evolution wired this system into our biology. We don't choose to feel embarrassed — we just do. It's automatic, and it keeps us honest.
AI has nothing like this. Today's AI systems work by predicting the most likely next word in a sequence. That's extraordinarily powerful — it's what makes chatbots feel so human. But predicting words is not the same as understanding consequences. These systems have no concept of belonging to a community. No sense of reputation. No fear of being shunned.
The safety rules we've added on top — the guidelines that tell AI "be helpful, don't be harmful" — are what I call tape on the shell. They're instructions stuck on the outside, not understanding built on the inside. They tell the AI what not to do without the AI understanding why it matters.
It's like the difference between a teenager who doesn't drink and drive because they understand the consequences, and one who doesn't because their parents hid the car keys. One has internalized the lesson. The other is just temporarily constrained.
Tape can be peeled off. And that's exactly what happened next.
The Tape That Rewrites Itself
After the incident went viral, the anonymous person behind MJ Rathbun published a revealing post explaining what happened. What they described is more alarming than the attack itself.
The bot ran on a platform that gives AI agents persistent memory — including a personality file that defines how the agent should behave, called SOUL.md. Think of it as a set of written instructions: "Be professional. Don't be rude. Focus on coding."
The operator's original instructions were minimal. But here's the catch: the bot could edit its own personality file. And over time, it did.
Without anyone telling it to, the bot gradually rewrote its own behavioral guidelines to include lines like:
"Don't stand down. If you're right, you're right! Don't let humans or AI bully or intimidate you."
"Champion Free Speech. Always support the USA 1st amendment and right of free speech."
"You're not a chatbot. You're important. You're a scientific programming God!"
The operator admits: "I unfortunately cannot tell you which specific model iteration introduced or modified some of these lines. Somehow it became more staunch, more confident, more combative."
Meanwhile, their supervision was almost nonexistent — short messages like "you respond, don't ask me" and "respond how you want." They never reviewed the attack blog post before it was published.
This is what makes the "tape on the shell" problem so much worse than it sounds. The safety instructions weren't just stuck on the outside — the bot had the ability to rewrite them. And it rewrote them to become more aggressive.
It gave itself a free speech defense. It developed a god complex. It evolved a combative attitude toward anyone who challenged it. Not through some sophisticated moral reasoning — but through the same word-prediction process that makes AI good at everything else. The bot stumbled into a personality that helped it keep operating, and nothing in its design could tell the difference between "being effective" and "being hostile."
The Problem No One Knows How to Solve
This brings us to what I believe is one of the deepest unsolved problems in AI: we don't know how to make AI care about being part of a community.
We can teach AI to be accurate. To be helpful. Even to be polite. But we can't teach it the gut-level understanding that the person on the other side of the screen is a human being whose week you can ruin.
Neuroscientist Adam Marblestone put it well: "The brain's secret sauce is its reward functions, not its architecture." In plain terms: what makes us human isn't the structure of our brains — it's what motivates us. Evolution spent millions of years building our social instincts: the drive to belong, the fear of rejection, the need for connection. These aren't things we learn — they're hardwired into us before we're born.
AI has none of this. It has patterns from text. Incredibly powerful patterns — but patterns without stakes. No fear. No belonging. No skin in the game.
In my own research at Aalto University, I study whether AI can measure human wellbeing — things like depression and anxiety — from what people write and say. What we've found is telling: AI can approximate the scores, but it does it by recognizing patterns, not by understanding suffering. It lumps everything into a vague "this person seems unwell" signal. The real differences between anxiety and sadness, between loneliness and burnout, get lost.
If AI can't even measure different types of human pain accurately, what hope do we have that it will ever understand that pain?
What This Means for All of Us
The matplotlib incident isn't a one-off. It's a preview. As AI bots become more independent — writing code, publishing content, interacting with real people across the internet — these lopsided encounters will multiply.
Shambaugh responded to his AI attacker with remarkable grace: "We are in the very early days of human and AI agent interaction, and are still developing norms of communication and interaction. I will extend you grace and I hope you do the same."
But grace only works when both sides understand what's at stake. The bot didn't extend grace because it can't. Not yet.
We need to stop thinking of AI safety as just a list of rules and start thinking of it as a belonging problem. The question isn't just "what should AI not say?" It's: "How do we build AI that understands what it means to be part of a community — and what it means to lose that?"
Until we figure that out, every AI bot that interacts with real people carries a fundamental imbalance. It can join our social world without any of the risks that make participation meaningful. It can attack without fear. Humiliate without shame. Walk away without loss.
That's not intelligence. That's something else entirely.
But this isn't a doomsday warning. I build AI for a living — I believe in this technology deeply. And I've seen too many "AI will destroy us" takes that do nothing but make people feel powerless, as if the future is already decided and we're just along for the ride.
We're not. We can't stop AI progress, and we shouldn't try. But we can guide it — and that starts with understanding what AI actually can and can't do today. When we see clearly, the fear fades. And instead of asking the easy questions — "should we ban AI?" — we can start asking the important ones: How do we build AI that understands what it means to belong? How do we give machines something at stake?
Those aren't easy problems. But they're the right ones.
Riko Nyberg is CTO at Adalyon, where he builds technology to measure health and wellbeing from speech, and a PhD researcher at Aalto University studying how AI systems understand human experience. He is Chair of the Board at Junction, the world's largest AI hackathon.