The benchmark for replacing a person with an agent is beating that person, not zero errors. That’s the lesson I lean on most when organizations ask whether a model can take over a job, and it comes from a story pilots tell each other about the landing gear lever. In the story, replacing a switch with a uniquely shaped lever solved gear-up landings — and the version they tell leaves out the interesting part: they also added three deafening horns and five flashing red lights, and pilots still land gear-up sometimes.
The story pilots like to tell
The story goes that somewhere early in aviation, the landing gear control was a switch like every other switch in the cockpit. Pilots forgot to lower the gear, or confused it with another switch, and landed on the belly — usually a fatal accident.
Then along came the world’s first UI genius, who replaced the switch with a lever that grabs your attention, in a shape you cannot mistake for anything else. Today that’s a standard.
I’m telling it as pilot folklore, not as documented history. I haven’t checked the sources, I don’t know who was there or when, and I don’t know how many of those landings actually ended badly. What I do know is what a cockpit looks like today, and what happens in it when I come in to land.
What that version leaves out
It’s a good story, and I hear it quoted far outside aviation as an example of design solving a problem outright. But nobody stopped at the lever.
They also added something like three deafening horns and five lights that flash red every time I come down through 300 feet / 55 knots without the gear down. So after the perfect UI fix, someone still thought another layer was needed. And then another one.
And still — pilots occasionally land gear-up.
That line is the whole point. No design failure here, no unusually bad pilot. Just a problem meant to be deterministic, three layers of protection working through different channels, and an error rate that is not zero.
Some problems are meant to be deterministic
Getting the gear down before touching the ground is binary. There’s no judgment call, no preference, no creative variant. Either it’s down or it isn’t.
Problems like that have several kinds of solution, and the story demonstrates three: mechanical (the shape of the lever), audible (the horns), visual (the lights). Each works through a different channel, and they coexist because none was enough alone.
The same holds inside a company. “This has to be exact” isn’t a solution, it’s a requirement. The real question is how many layers you’re willing to build around it, and what happens when one of them misses.
Worth noticing that the layers in the story don’t prevent the mistake, they catch it. The horn doesn’t make me remember; it shouts at me after I’ve forgotten, at the last moment the thing is still fixable. Not every layer has to improve whoever performs the action — some only have to stand in the way.
Humans aren’t deterministic either
An LLM is, of course, not deterministic. The same input can come back two different ways, which is an excellent reason not to let it sign off on anything unsupervised.
But the person doing that task today isn’t deterministic either. They’re tired in the fourth hour, they skip a field when the phone rings, they type the same number twice on a Sunday morning, the first day of the Israeli work week. The pilot in the story went through years of training, sits in front of a lever designed for exactly this, and gets three horns and five lights — and sometimes still lands gear-up.
So an LLM is sometimes still the most suitable candidate to replace a person, even on a problem that is deterministic by nature. Not because it’s precise, but because what it’s being measured against isn’t precise either.
The benchmark is the person, not the zero
This is the part that derails conversations. When a team weighs moving a manual process to an agent, the bar is almost always zero errors — a bar nobody ever set for the human doing it today.
The first question is simple: how many errors does this process produce right now? Without that number there’s no way to tell whether the new system improved things or made them worse, and the decision gets made by the mood in the room — which reacts far more strongly to a machine’s mistake and barely at all to a person’s.
The measurement isn’t complicated. Take a sample of outputs from the process as it runs today, count how many are wrong, run the same sample through the agent, count again. Those two numbers are worth more than any theoretical discussion of hallucinations.
The hard part is not the run, it’s agreeing in advance on what counts as an error. Once that’s written down, it often turns out the manual process doesn’t meet the definition either, and that some of what was called “exact” is a judgment call nobody ever checked.
Where this argument runs out
This isn’t a blanket permit. An identical error rate is not an identical risk once volume changes: say one percent of thirty actions a day versus one percent of three thousand — entirely different worlds, even when the table shows the same number.
And nobody in this story settled for one improvement — the lever didn’t remove the need for the horns. So the interesting question isn’t only whether the agent beats the person. It’s what the horn and the red light are in your process, and where a wrong action gets stopped before it touches the ground.
Usually the answer is boring: a human approval before anything is sent, a total cross-checked against a second source, a log somebody actually reads. None of it is glamorous — and neither is a horn or a flashing light. They just stand in the way.
The lesson
When someone tells you a process has to be perfect, they’re describing a wish, not the current state. Aviation took a binary problem where a mistake is expensive, stacked three layers on it, each through a different channel, and pilots there still land gear-up sometimes. Your process is allowed to aim lower than that.
The reverse reading is worth saying out loud too. If a process genuinely tolerates no errors, the question isn’t which model to pick — it’s why one person decides alone today with nothing standing behind them.
The original post ended with a question: can you spot the gear lever? If you’d rather see what the measurement looks like in practice — the manual process first, then the comparison — I’ve written up how I work.