a customer, in plain English
Parsed,Plausible,Wrong.
A tiny AI reads pizza orders. Sometimes its answer looks perfect and is still wrong, and nothing catches it. Scroll to see why, and a surprisingly simple way to halve it.
replaying real outputs
ticket the model writes
try
Someone orders a pizza.
A tiny model writes the ticket.
It looks fine. The kitchen accepts it.
It's wrong. No one noticed.
p1^tuna^thin_crust!peppers
13 tokens in the symbols format, one at a timeorder #0427 accepted
valid ticket, so nothing complainsnumber1
styletuna
stylethin crust
toppingpeppers
they asked for tuna, as a topping
One order, six ways to write the ticket
“i would like to try two medium tuna pizzas with extra cheese and no pesto”
Brackets52 tokens
(ORDER (PIZZAORDER (NOT (TOPPING PESTO ) ) (NUMBER 2 ) (SIZE MEDIUM ) (COMPLEX_TOPPING (QUANTITY EXTRA ) (TOPPING CHEESE ) ) (TOPPING TUNA ) ) )
- Brackets
- JSON
- Short brackets
- Tags
- Symbols
- Words
A token is a chunk of text the model reads or writes. More tokens means more work.
Shorter isn't smarter.
Here are the six spellings, longest first. Now watch them re-sort by how often the model gets the order exactly right.
Brackets85.8%
JSON81.6%
Short brackets75.1%
Tags85.9%
Symbols79.7%
Words87.5%
+cheese0%
vsextra cheese0%
The same length, but a clear gap. The model already knows English. It has to learn what a + means.
A hundred orders.
Written in brackets…
…and written in words.
0right
0loud errors
0silent errors
A loud error doesn't parse, so the system notices and asks again. A silent one looks valid and goes to the kitchen anyway.
Words cut silent errors by 2.3×.
Your turn
Catch it, or break it.
Every ticket below passed the parser, and some are still wrong. Read the order, check the ticket, decide.
order 1 of 10
ticket printed
This is the real model, running in your browser. It was also taught to answer ? when something isn't an order or isn't on the menu, so a refusal is a loud error, not a broken one. Write an order you think it will misread. If the ticket comes back valid but wrong, you broke it.
your order
ticket printed
the model's ticket appears here
Did it get your order right?
Words are also 3.7× faster.
Fewer tokens to write means less waiting. Shown at half speed.
Brackets0 ms
Words0 ms
Then we tried an even smaller model (LFM2.5-350M)
Words were longer this time. They still won.
Symbols11.4 tokens0%
Words12 tokens0%
So length was never the point. Familiar words were.
It started in a café
0ms
per order, on an ordinary laptop: the café parser this study grew out of. No cloud, no GPU farm, and0% of coffee orders exactly right. Its tickets carry a ? for anything not on the menu, and putting the pizza model online showed it needed one too.
The guardrail
Only the menu gets in.
People type anything into a demo, so the live pizza model learned the café parser's trick: parse orders from the menu and answer? to everything else, from gibberish and chatter to horses and orders with anything extra in them.
0 parsed0 refused with ?
0of gibberish refused
0of nonsense it never saw, refused
14/14demo-style prompts refused
0of real orders refused too: the price of strictness
It also learned to listen when people change their mind mid-sentence. Told “i want a pizza with cheese, wait no cheese, only tomato sauce”, it now writes pizza 1 tomato_sauce, in 5 of 5 runs.
0of corrections it never trained on, applied
0of corrected orders exactly right
Refusals shown are the run's real outputs on its challenge prompts; accepted orders are real PIZZA test orders, passing at the measured rate. Every number on this page, above and below, comes from models trained with this guardrail, averaged over 5 training runs.