Pilots and LLM Agents: Instruct, Don't Build

Originally published as a LinkedIn post, expanded here.

Pilots are a bit like LLM agents, which is why I don’t build agents for organizations — I teach the people in them to instruct one. A few years of training taught me mainly one thing: 99% of the time you just follow the checklist, and the remaining 1% — the critical one — is creativity built on knowledge and experience.

99% checklist, 1% creativity

The training runs a few years, and most of it is learning firsthand that the job is far less dramatic than it looks. Almost all of flying is a known sequence: read, do, verify, continue.

The remaining 1% is what justifies those years. It’s needed exactly when the checklist doesn’t cover what’s happening, and it can’t be written in advance — if it could, it would already be a checklist.

The original post came with a video of me executing the prompt I was handed at the start of the day: Exterior Check. Two words, and out of them comes a full procedure, because the procedure is already loaded. Short prompts work beautifully when the other side is trained, and less well when it isn’t.

How you train a pilot: push them into a corner

In the Israeli Air Force there’s no time to accumulate that experience naturally — conscript training runs on a fixed clock — so it gets forced. They put you into aviation corners — situations engineered to go wrong — again and again, and show you the way out.

Usually your survival instinct finds the way before the instructor intervenes. Not always, which is exactly why the instructor is sitting there.

The corners are never pleasant in the moment, which is the point. There’s no way to build experience while skipping the part where you don’t know what to do.

Nothing clever about the method. You just have to fly a lot. Nobody learned to fly from a lecture about flying, and nobody learns to work with an agent from someone else’s demo.

The loop: data in, output out, feedback back

The whole method fits in a line: data goes in, output comes out, you get feedback from the instructor or from the aircraft, and around again.

The two sources aren’t equivalent. The instructor explains why you were wrong. The aircraft is brutal in its honesty — no explanation, no softening, no interest in what you meant, just an immediate reaction to what you actually did. That’s the feedback that sticks.

Working with an agent has exactly these two sources:

Feedback source What it gives you The equivalent with an agent
The instructor Why it was wrong, and what right looks like Your comment on the output
The aircraft An instant, dry reaction to what was done The error the tool throws back

Lean only on the second row and nobody is telling the agent why the output wasn’t good enough — so there’s nothing for it to improve against past a point. The first row costs more: it needs someone who knows what a correct result looks like and will spend a minute explaining it.

Which means you can’t skip the stage where the agent gets a real task of yours wrong. The mistake isn’t a glitch in the learning process. It is the process.

Agents: sometimes there’s data, sometimes you have to instruct

Agents need a lot of data, same as a new pilot. Sometimes it exists already, sitting somewhere in decent shape. You push it in one go and it genuinely is magic: the agent knows the context from minute one.

Sometimes it doesn’t. The knowledge lives in people rather than files, or in files nobody wrote for an outside reader. Then there’s no shortcut, and you have to “instruct”: let it act, let it get things wrong, give feedback, and around again.

The difference isn’t the tool, it’s how much data already exists in readable form. Make that call early — it decides how much of the work is moving files and how much is sitting in front of the tool.

Why I don’t build agents for organizations

I personally don’t build agents in the work I do with organizations. I just teach them to instruct.

The difference isn’t technical. An agent someone built for you is frozen in time: it knows what you knew the day it was specified, and moving it means going back to whoever built it. The ability to instruct stays with the people and moves with the process.

The gap looks small on day one and shows up months later, when something shifts and someone chooses between filing a request with the builder and just explaining to the agent what changed.

So “which agent are you building for us” interests me less than who on your side gives the feedback, and when.

There’s an awkward side for me. A process that ends with you not needing me is short, and I sell fewer hours than I could. Still the right trade: the only thing genuinely left in a company at year’s end is what its people can do unaided.

Where the analogy breaks down

A pilot who got a briefing remembers it tomorrow morning. An agent remembers only what you wrote down somewhere permanent — project instructions, a file, a context doc. Whatever you only said inside one conversation disappears with the window you closed.

So instruction that stays verbal is wasted. The feedback has to land somewhere the agent reads next time, or you’ll teach the same lesson forever and give up for good reason.

It’s also the difference between “it doesn’t work” and “it wrote X instead of Y”. The first is a complaint, the second is instruction, and only the second moves anything.

It’s the least glamorous part of the work, and the part that decides whether the adoption survives after I leave. A company that writes down what it learned keeps moving; one that only talks starts every morning from the same place.

My aircraft is Claude Desktop

In the air I have an aircraft. With organizations my “aircraft” is Claude Desktop, and I can’t shout loudly enough about how good a tool it is and how little you need beyond it.

Unpopular, in a market that ships another agent-building platform every week. But what a company is usually missing isn’t another software layer — it’s people who have sat in front of the tool long enough to develop an intuition for it. That part no platform does for you.

Which doesn’t mean there aren’t other good tools. It means swapping tools is almost never what was missing.

The lesson

When you bring a new system into an organization, the interesting question isn’t how much it knows on day one. It’s how fast it learns and who is responsible for teaching it. In aviation that’s called instruction, and nobody would think of skipping it.

It’s also easier to check than any feature list: what could the agent do for you a month ago, and what does it do today?

Pleasant side effect: whoever instructs learns the most. The people giving the agent feedback are usually the first to find where your own process was never fully defined.

One short promotional note. If you’re ready to make the jump and genuinely adopt AI, instead of adopting the service providers in the field, you’re welcome to get in touch. I promise that in under a week you and Claude carry on without me. I stay available, don’t worry.

Questions people ask

Do you need a custom-built AI agent to adopt AI in a company?

Not necessarily. In the work I do with organizations I don't build agents — I teach the people there to instruct one themselves. An agent someone built for you is frozen at the moment it was specified, while the ability to instruct stays with your team and moves as your process moves.

What does it mean to 'instruct' an AI agent?

It's the same loop used to train a pilot: let it act, let it get things wrong, give feedback, repeat. Part of the feedback comes from the tool itself as an error or a bad output, and part has to come from a person who can explain why the output was wrong.

How long does it take a team to work with AI on their own?

That's the commitment I make in the engagement: in under a week you and Claude carry on without me, assuming someone on your side is actually sitting in front of the tool. I stay available afterwards, but the point is that you don't need me in order to keep moving.

Which tool do you need to start?

My 'aircraft' is Claude Desktop, and I can't shout loudly enough about how good a tool it is and how little you need beyond it. What organizations are usually missing isn't another software layer — it's people who have spent enough hours in front of the tool.