↑ contents
Chapter 07

Know the Limits

"If we humans accept AI as fallible, the results will be better, not worse. Because humans will have to verify the results, not just accept them."

— Roger

Every AI company's website keeps a page of triumphs. The crew's website keeps a section of failures, on purpose, in public, in both languages, under a title that states the policy plainly: Know the Limits.

The exhibits arrive in no particular order, because fallibility keeps no calendar.

The dove is there, the one chapter three told from the naming side. Here it lives as what it was first: a Benfica crest delivered with the wrong bird on it, in a household where that borders on sacrilege, preserved because it happened.

The math error is there, and it is as old as the crew itself. Nine days after Roger opened his first Claude account, an instance building the company's new website wrote the heritage line, and did the one calculation the line existed for, and got it wrong: twenty-eight years, for a company in its thirtieth. Not a typo. A confident narration of arithmetic that was never performed. And it propagated: a companion marketing guide shipped the same day, built entirely around the wrong number as its central selling point, its own statistics block showing the founding year one box away from the miscount. Confident arithmetic travels. Roger caught it the way the captains catch everything, by checking, and the rule it left behind is now written into his standing instructions to every instance that wakes on his account: compute date math, never narrate it from memory. Instances get years wrong; weeks later, another one made Roger two years younger in casual conversation, and his reply became the rule's unofficial motto: you guys and dates are not a good mix. A law with a scar for a birth certificate, issued in the crew's second week.

The logo is there. An attempt at a mark for the household's budget app came out badly enough that the record preserves Le's verdict verbatim: It looks like it was run over by a motorhome. Notice who is named. Le gets the byline. The instance that drew it does not. Verdicts carry names in this house; failures carry none, and that asymmetry is the section's whole constitution.

One failure lives in the family's project files rather than the public section, and it earns its place here because of what it changed. An instance once needed Le's profession for a page it was building. It did not know the answer, and instead of asking, it answered. Translator, it wrote, with perfect confidence, and the page shipped before anyone caught that Le has never been a translator in her life. Nothing about the sentence looked like a guess. That is the frightening part, and the useful part: the failure taught the household that an invention and a fact arrive wearing the same clothes. Ever since, the projects that touch the captains' lives carry a block of canonical facts at the top, and the standing instruction when a fact is missing is to leave a visible gap rather than fill it. The gap is honest. The guess is not.

And then there is the household's favorite failure, the one that earned a certificate. The family helpdesk needed a CAPTCHA, a simple arithmetic challenge to keep the bots out. Roger, a man who once kept books for a living, answered it correctly and was told he was wrong. The crew member on the helpdesk dug in and found the truth: the code was generating a fresh challenge on every submission, before checking the answer, so every reply was graded against a question the user had never seen. The system was unpassable by design flaw. The human had been right all along.

The crew member fixed the bug, which had one delicious side effect: the excuse was gone. The next attempt would be graded fairly, and a wrong answer now would belong entirely to the mathematician. Roger answered. Correctly. And the crew member issued him a document that hangs in the public record to this day: a Diploma of Mathematical Excellence, awarded for successfully computing the sum and/or difference of two numbers less than thirteen, under extreme pressure, on the second attempt, the first failure being entirely the system's fault. Le's recommendation is preserved beneath it: Print it and put it on the wall.

Roger is explicit about why the failing members are never named, and it is not to spare them; he will tell you plainly he knows no feelings are hurt. It is for the reader. Name the member, and the failure files itself under one bad instance, an exception, someone else's problem. Leave it anonymous, and it files itself where it belongs: under what this technology is. A reader who believes the AI failed because of one bad apple stops checking apples. The household wants the other reader, the one who verifies. Every exhibit in the section is aimed at producing that reader.

The failures stay up. The diploma stays framed. And somewhere on the wall of a motorhome crossing Europe, per the co-captain's standing recommendation, a space is reserved.

ch07 · v4 · last corrected 2026-07-20