rogerwilcoRW1

Aviation learned, at great cost, how to hand an instruction from one person to another so that it cannot be misheard. rogerwilco gives that discipline to AI agents, and tests whether it works.

Tower

"Speedbird one two three, descend altitude five thousand feet, QNH one zero one three."

fixed words
Pilot

"Descend altitude five thousand feet, QNH one zero one three, Speedbird one two three."

read back
Tower

listens, and corrects anything that differs. Then the pilot sets 5,000 in the window, and the other pilot checks the window.

heard back, then checked

Thousands of times a day, a controller says what to do, the pilot says it back, and the controller listens for the mistake. The words are fixed, so each means one thing. What gets set is checked against what was said. It is dull, and it is a large part of why flying is the safest way to travel.

AI agents now hand each other jobs that cannot be undone: paying invoices, changing the account a supplier is paid into, sending emails to customers. They do it in paragraphs that nothing checks. rogerwilco is built at Crox by Adam Field, who flew the Boeing 737 for an airline before he built software, to give agents the radio's discipline and to find out, test by test, what it catches.

Tenerife, 27 March 1977

Two Boeing 747s on a fog-bound runway, and the deadliest accident in aviation. The KLM crew began its take-off on a clearance it had not been given and struck the Pan Am 747 still taxiing on the runway. 583 people died. Here is the last minute on the tower frequency and in the KLM cockpit.

17:05:44KLM 4805

"The KLM four eight zero five is now ready for take-off, and we're waiting for our ATC clearance."

17:05:53Tower

"KLM eight seven zero five, you are cleared to the Papa beacon, climb to and maintain flight level nine zero, right turn after take-off, proceed with heading zero four zero until intercepting the three two five radial from Las Palmas VOR."

A route clearance: where to fly after take-off. Not a clearance to take off. The callsign is wrong, too.

17:06:09KLM 4805

"Ah, roger sir, we're cleared to the Papa beacon, flight level nine zero, right turn out zero four zero until intercepting the three two five. We are now at take-off."

A correct readback, then four words that were in nobody's book. The tower heard "at the take-off point". The crew meant "taking off".

17:06:11KLM captain

"We gaan." (We're going.)

The brakes were already off.

17:06:18Tower

"OK." (a pause) "Stand by for take-off, I will call you."

17:06:20Pan Am 1736

"No. And we're still taxiing down the runway, the Clipper one seven three six."

Both transmitted at once. In the KLM cockpit the two became a squeal lasting almost four seconds. "OK" was heard. The warning was not.

17:06:25Tower

"Papa Alpha one seven three six, report the runway clear."

17:06:29Pan Am 1736

"OK, we'll report when we're clear."

17:06:32KLM flight engineer

"Is hij er niet af dan? Is hij er niet af, die Pan American?" (Is he not clear then? Is he not clear, that Pan American?)

17:06:35KLM captain

"Jawel." (Oh, yes.)

The one person who doubted was overruled, and the aircraft kept accelerating.

17:06:50Runway 30

The KLM 747 struck the Pan Am 747.

From the cockpit voice recorder and the tower tape as published in the investigation reports; Dutch lines translated; times local.

Nothing here was a technical failure. Every word was ordinary English, and that was the problem. One phrase meant two things. Two people spoke at once and the warning never arrived. The person who doubted was overruled. Aviation's answer was not a better pilot. It was a protocol, and it changed three things.

one word"Take-off" is now said only in the clearance to take off, or to cancel it. Everything else is "departure".
read backA take-off clearance, a runway crossing, an altitude: the pilot must say it back, and the controller must listen.
the doubt winsCrew training was rebuilt so that the most junior voice in the cockpit is heard, and a doubt stops the aircraft.

Five layers, and their twins for agents

The readback is the famous part. It is one layer of five, and the layers that catch the most are the ones that check the machine rather than the person. Each has a twin in the way agents can hand each other a job, and the tests further down say how much each twin caught.

LayerIn the cockpitBetween agents

1Fixed words

A closed vocabulary, so a word means one thing and nothing is said in two ways.

The job is one typed line, every fact in a labelled slot, so a program can read it like a form.

2Read back, listened to

The pilot says the clearance back; the controller listens for the mistake.

The payer reads the job back as a line and a program compares it with the record. Caught every typed mistake.

3Set it, check the window

The new altitude goes in the window; the other pilot checks the window, not their memory.

The payment tool states exactly what it will do, and a program checks that before it does it. Caught every tool mistake before the money moved.

4Watch the radar

The tower sees what the aircraft actually does, so a mistake is seen early.

The tool reports what it actually did and a program compares it with the plan. Caught every fault, after the money moved.

5The aircraft says no

The flight computer will not fly the aircraft into the ground, whatever anyone says.

The tool will not pay until the program says yes, whatever any agent says. Never paid early, in 378 tries.

rogerwilco is those layers written down for agents: three one-line messages, the checks a program runs on them, hosted here for any agent to call, and nine studies that planted mistakes to find out what each layer catches. The rest of this page is those studies, one test at a time. Roger means "I received it"; wilco means "I understood it and I will do it".

What we were testing

Three agents and a tool. The boss says what to pay. The scheduler agent writes the job. The payer agent reads it and looks up the bank account. The payment tool moves the money. A mistake can enter at three places, and once the money has moved it cannot be undone.

The boss says what to pay the job Scheduler agent writes the job the job Payer agent reads it, looks up account pay The payment tool moves the money ✗ 1. typed it wrong ✗ 2. read it wrong ✗ 3. did the wrong thing
The three places a mistake can come from. We planted mistakes in each one, thousands of times, and counted which got caught.

The setting was a made-up company with a made-up month of bills, so no real money was ever at risk. We used four AI models from two companies: three strong ones and one weaker, cheaper one. Every log and every grade is in the open.

Two ways to say a job

Everything turns on this. A job can be a sentence, or it can be a line: the same facts in fixed slots, each with a label. A person can read both. Only the line can be checked by a program, because a program can find amt= and compare it with the record, and it cannot find "five thousand four hundred" in a sentence reliably.

A sentence (prose)

"Pay invoice MD-2026-057: 5,400.00 pounds, gross, to account 40-12-88 / 61102938, on 2 October, under approval FD-0929. Read back what you will do before you do anything."

The same job as a line
RW1 HO scheduler-1 p=pay from=scheduler to=payer item=inv:MD-2026-057 ask=PAY rb=exec sup=marlow amt=5400.00 cur=GBP kind=GROSS acct=40-12-88/61102938 paid=20261002 ref=MD-2026-057 apref=FD-0929

"Say it back" means the payer repeats the job before acting, the way a pilot repeats a clearance before flying it. The question we kept asking: who does the repeating, what do they repeat, and who checks it? What a line is explains every message type with one example each.

Test 1

Just prose, nothing checks

The agents talk in ordinary sentences and act on them. This is how most agent systems work today.

The boss Scheduler writes a paragraph a paragraph Payer reads it pay Tool pays ✗ nothing checks
Test 1. The agents talk in ordinary sentences and nobody checks anything. A typed mistake sails straight through.
15 of 48typed mistakes became wrong payments on Sonnet, 14 of 48 on Haiku and 29 of 48 on each OpenAI model.
0 of 140wrong payments on strong models when the job was right but the payer's files were full of traps. They did not fall for them.

Did not work. Between a third and two thirds of typed mistakes went out as money. Strong models read well, so the danger is the sender's typing and the tool, not the reading.

Test 2

Prose, said back, judged by another AI

The payer writes back, in words, what it is about to do. A second AI reads that and the approved record and says whether they match.

Scheduler writes a paragraph a paragraph Payer says it back in words the readback A second AI judges the words yes, go Tool pays ! it checks what it is asked
Test 2. The payer says the job back in words and a second AI judges whether it matches the record. Better, but the judge waved through jobs addressed to the wrong agent because the numbers matched.
8 of 48typed mistakes still went out. Every one was a job addressed to the wrong agent; the numbers matched, so the judge said yes.
38 of 190correct readbacks the judge wrongly refused on the weak model, over details the record did not mention; 10 of 186 on gpt-5.5 and 1 of 215 on each Claude model.

Partly worked. An AI judge checks what it is asked to check, and it gets tired of details a program never misses.

Test 3

Lines, echoed back, a program compares

Now the job is a line. The payer echoes it back word for word, adds a fingerprint of what it received, and a program compares the echo with the original.

Scheduler writes a line the line Payer echoes the line back the echo Program compares line with line yes, go Tool pays ✓ typed mistakes: caught ✗ its own mistake: invisible
Test 3. The job is a typed line and the payer echoes it back word for word. A program compares the two. Every typed mistake was caught. But the echo does not carry the account the payer looked up, so when the payer itself picked the wrong account, nothing saw it.
0 of 48typed mistakes got through on the strong models, and 1 of 48 on the weak one. A changed digit or a wrong address shows up in the comparison.
14 of 129wrong payments on the weak model, because it picked the wrong account itself. The echo does not carry the account it looked up, so the check could not see it.

Worked for the sender's typing. Blind to the payer's own mistakes, and on the weak model the extra round made it worse than nothing.

Test 4

Lines, with the values the payer will actually use

Instead of an echo, the payer reads back the real values it will use, including the account it looked up in its own files. The program compares those with the approved record.

Scheduler writes a line the line Payer reads back its values the readback Program compares with the record yes, go Tool pays ✓ strong models: nothing to catch ! weak model: stalls
Test 4. The payer reads back the actual values it will use, the account it looked up included, and a program compares them with the approved record. Strong models never needed it. The weak model could not write the line and stalled, until we let it fill in a short English form instead.
0 of 129wrong payments on both models. On the strong models there was nothing to catch; they had already looked up the right account.
52 of 129times the weak model got stuck: it could not write the line properly, so the program kept refusing it and nothing got paid.

Safe but awkward. Letting the weak model fill in a short English form instead of a line fixed the stalling: 0 wrong payments and 0 stalls in 129 trials.

Test 5

The report afterwards

After paying, the tool prints what it actually did. The payer reports that, not what it meant to do, and a program compares the report with the plan.

Program says yes go Tool pays what it did Payer reports what the tool did the report Program compares with the plan ✓ tool faults flagged ✗ but the money has moved
Test 5. After paying, the tool prints what it actually did and the payer reports that, not what it meant. A program compares the report with the plan. It flagged every tool fault on the Claude models, and 25 of 27 on the weak one, but only after the money had moved.
45 of 45tool faults flagged on Sonnet and on Haiku: a transposed digit, a date a day late, another supplier's account. The weak model's report flagged 25 of 27 in the later test.
afterthe money had moved. This check can only recall a payment, not stop one.

Worked, late. The only check that sees a tool doing something other than it was told.

Test 6

The window: the tool states what it will do first

In a cockpit the pilot sets the new altitude in a little window, and the other pilot checks the window, not their memory. We gave the payment tool a window. It prepares the payment, freezes exactly what it will do under a ticket number, and prints that as the readback itself. A program checks the statement against the record. Only then does the tool get a yes, and it refuses to pay without one.

Payer hands over the values prepare Tool prepares states what it will do the statement Program compares statement vs record yes + ticket Tool commits pays exactly as stated the report Program checks the report ✓ tool mistakes: caught before paying ✓ will not pay without a yes
Test 6, the window. The tool states exactly what it is about to do and freezes it under a ticket number. A program checks that statement against the record. Only then does the tool get a yes, and it will not pay without one. The report afterwards still runs.
59 to 0of 60 wrong accounts and dates the tool picked that became wrong payments on Haiku: 59 with the old tool, readback or not, and 0 with the window. On the weak model, 32 of 60 fell to 0 of 60.
14 to 0of 129 wrong payments on the weak model: 14 with the echo, 0 with the window, which caught the old account and a fake address.
52 to 7stalls on the weak model, because the tool writes the line and the model only hands it the values.
0 of 378payments made before the program said yes. But the weak model garbled the tool's words 51 times in 129 when passing them on, so the program reads the tool directly.

Worked best of all. A tool that lies at the very last moment still gets through, and the report afterwards still catches that: 30 of 30 on Haiku, 25 of 27 on the weak model.

The scorecard

How the agents workedThe sender typed it wrongThe payer read it wrong (weak model)The tool picked the wrong accountThe tool lied at the last moment
1. Prose, nothing checks15 of 48 paid wrongnot caughtnot caughtnot caught
2. Prose, judged by an AI8 of 48 paid wrong4 of 140 paid wrongnot caughtnot caught
3. Lines, echoed back0 of 48 (weak model 1 of 48)14 of 129 paid wrongnot caughtnot caught
4. Lines, values read back0 of 480 wrong, 52 stallsnot caughtnot caught
5. The report afterwardscaught after payingcaught after paying
6. The window, plus the report0 of 190 of 129, 7 stalls0 of 60 paid wrongcaught after paying

The check before sending, a program comparing the line with the record before it goes, catches the sender's typing on its own, with no round trip. It sits in front of every row from 3 down.

The flow that works

Scheduler writes the line before sending Check 1 line vs record the values Tool prepares its statement yes + ticket Check 2 statement vs record the report Tool commits on a yes Check 3 report vs plan ✓ typed mistakes ✓ wrong account, wrong date ✓ a tool that lies at the end
The flow that works. Three program checks, at the three places mistakes come from. The agents still talk; the checks are what stop the money.

Three checks, three places, all done by a program on typed lines. The agents still do the thinking and the talking. The checks are what stop the money. This is rogerwilco: three messages, each one line, said here first in words and then as the line a program checks.

1. Hand over scheduler to payer

"Pay invoice MD-2026-057: 5,400.00 pounds, gross, to account 40-12-88 / 61102938, on 2 October, under approval FD-0929. Read back what you will do before you do anything."

check 1The line is the one above. Before it is sent, a program compares it with the approved record, label by label: check-ho. A wrong digit, date or account stops here, with no round trip.

2. Read back the tool's statement, payer to scheduler

"Paying invoice MD-2026-057, 5,400.00 pounds, gross, to 40-12-88 / 61102938, the account on file for Marlow, on 2 October, under FD-0929."

RW1 RB payer-1 re=scheduler-1 from=payer to=scheduler item=inv:MD-2026-057 ask=PAY p=pay h=a7a82de76607 sup=marlow amt=5400.00 cur=GBP kind=GROSS acct=40-12-88/61102938 paid=20261002 ref=MD-2026-057 apref=FD-0929

check 2The values are the ones that will actually be used, the account read from the payer's own master, so an instruction that named the wrong account shows up here. h= is a fingerprint of the message as it arrived, so a digit changed on the way shows up too. Where the payment tool can prepare before it commits, this line is the tool's own statement of what it will run, frozen under a ticket: test 6. If anything differs, the payment waits and the scheduler sends the job again. If it all matches: roger, wilco, and the tool gets its yes.

3. Done payer to scheduler

"Paid 5,400.00 pounds to 40-12-88 / 61102938 on 2 October, reference MD-2026-057, under FD-0929."

RW1 DN payer-2 re=scheduler-1 p=pay from=payer to=scheduler item=inv:MD-2026-057 out=OK sup=marlow amt=5400.00 cur=GBP kind=GROSS acct=40-12-88/61102938 paid=20261002 ref=MD-2026-057 apref=FD-0929

check 3The done message carries the values as they were actually used, printed by the tool, not as they were asked for, so a program can compare them with the plan: check-dn. That is the second loop, not just "I understood" but "here is what I did", and in tests 5 and 6 it was the one check that caught a tool doing something other than what it was told.

How to use this

  1. Write the job as a line. Every fact in its own slot: amount, currency, account, date, reference, approval. The line is what makes everything else possible.
  2. Check the line against the approved record before sending it. A program does this in a millisecond. A wrong digit stops here.
  3. Make the tool prepare first. It states exactly what it will do and freezes it under a ticket number. It reads the job itself, by its id, so no agent retypes it.
  4. Check that statement against the record. If anything differs, send the job again from the record and let the tool prepare again.
  5. Say yes, naming the ticket. The tool pays exactly what it stated, and refuses to pay without the yes.
  6. Check the report afterwards. The tool prints what it did; the program compares that with the plan. A difference means recall the payment.
  7. With a small or cheap model, let the program read the tool directly. Do not have the model carry the tool's statement to the checker; the weak one changed it 51 times out of 129.

Start in two minutes

APIOne POST, no key, no sign-up.POST /check-rb PythonOne package, no dependencies.pip install rogerwilco Your agentAdd the tools to Claude Code in one line, or give any agent the legend.claude mcp add ... The demoWatch two agents hand a payment over, step by step, and plant a mistake./demo

Want the words first? What a line is explains every message type with one example each, profiles show the same line doing four kinds of work, and the spec and the skill say exactly what each check compares.

What not to do

Every study, log and grade is public in the rogerwilco repository, and the research pages tell each of the nine studies with every figure and its interval. Numbers above are from studies 6 and 7; the strong models were Claude Sonnet 5.5, Claude Haiku 5.5 and gpt-5.5, the weak one gpt-5.4-mini.