Six times AI went wrong, and what would have stopped each one
These are my stories and my fixes. An AI writes them up, and I check every fact and link before a post goes up.
A lot of owners around here are wary of AI, and the people selling it wave that away. I use AI all day and build agents that run in production, and I think the owners have a point. The way I put it:
"It's not about whether or not it's going to make mistakes. Of course it is. The point is keeping human intelligence at the right place."
In June 2023, two New York lawyers (opens another site) relied on ChatGPT for research and filed a brief citing court cases that didn't exist. They and their firm were fined $5,000.
In 2024, Air Canada's chatbot (opens another site) told a grieving customer he could claim a bereavement refund after booking. A tribunal held the airline responsible for what its chatbot said and ordered it to pay.
In May 2026, a Vermont bagel shop (opens another site) let an AI marketing program build a month of Instagram posts, which put the shop's bags in front of a wood fire that wasn't theirs and turned a real customer's comment into a fake handwritten note. Customers pushed back, some with one-star reviews, and the owner pulled the posts and apologized.
Klarna (opens another site) replaced customer service with AI. By May 2025 its CEO was calling the result 'lower quality,' and the company is hiring people again so customers can always reach a human.
In August 2025, the FTC went after Air AI (opens another site), alleging it sold small businesses an AI agent that often couldn't place calls. Some buyers reportedly lost as much as $250,000.
A Texas bookkeeping firm (opens another site) says AI 'clean-up' passes have made some clients' books more expensive to fix, because they had to unpick the automated changes, and 'the balances will still be wrong.'
In all six the same thing was missing, whatever tool they'd bought.
The rules that would have stopped them
In every one of those, nobody was in the right spot to catch it. The fix is a short list of rules, set before anything goes live.
The rules
- A person checks every fact and approves anything in your name.
- It can't promise what nobody wrote down.
- A person is always one step away.
- Test it yourself before a customer ever hears it.
- It proposes, and a person decides.
Everything I build starts in propose-only mode, with me approving every action, and earns more room by results.
How you'd put these rules in place yourself
Four steps, in this order, for any AI tool you're about to turn loose, bought or built.
Write down what it may say
One page, in plain words: what it may say, what it may promise, and what it never touches. If it isn't on the page, the tool hands the customer to a person.
Start it in propose-only mode
It drafts the reply, the post or the change, and nothing happens until you approve it. Every one, for at least a month.
Test it with real cases first
Feed it a stack of your real messages, including the awkward ones, and read every answer. Then try to get it to promise something it shouldn't, because a customer will.
Read what it did every week
Once a week, read the log of what it actually did. A clean month earns it one more job. A bad week takes one back.
If you want AI working in your business with these rules built in from day one, that's how I set up everything I build. Tell me what you'd want it doing and I'll show you where the person goes.
