Fail-first models: failure becomes structure.
A fail-first model learns only from failures it has actually had. The environment, not the model, decides whether an action worked. Each verified failure becomes an explicit constraint, kept in symbolic memory with its evidence, so the next attempt starts from a smaller search space. No weight update. Never pay for the same lesson twice.
On 25 September 2026 a symbolic floor with a memory of its own failures played two real Atari 2600 ROMs: +32% on Space Invaders, −12% on Freeway, the same mechanism. That pair became Peel, to our knowledge the first fail-safe model. The same loop now plays DOOM, Wolfenstein 3D and Fallout as VDSG, and flies a drone course it can relearn from nothing, always inside a floor its learning cannot widen. The frozen Atari result is further down this page, exactly as it was published.
The white paper and the deck: Fail-First Models: Failure Becomes Structure (PDF) · the 18-slide deck (PDF)
The fail-first loop, step by step
Every step of the loop is something the systems on this site actually do. The middle column is the Atari floor this page began with; the right column is what the same step became in VDSG and the drone.
| step | in the arcade floor | in VDSG and the drone |
|---|---|---|
| 1 · State what is true now? | A situation bucketed from raw 210×160 pixels, with zero emulator RAM. | Facts read from the engine or simulator state: health, who is present, free space by sector. |
| 2 · Attempt take an action | The pilot proposes from the admissible set. | The rules choose a goal inside the admissible set; a model may propose, never admit. |
| 3 · Verify what actually happened? | The ROM decides: a life lost, read at the true impact frame (we got this wrong first). | The game or simulator decides: a death, a contact, a lap time. |
| 4 · Failure keep the evidence | A card: the situation and the action taken. | Charged to the decision that opened the episode: the line that started the fight, not the heal inside it. |
| 5 · Construct rule evidence into constraint | A pattern becomes a rule only when its Wilson lower bound clears the base rate. | The same bar; one unlucky death does not become a rule. |
| 6 · Symbolic memory no weight update | Rules are counts that cite experience IDs, traced into 203 Peel tiles. | An evidence table the learner writes; the licensing cards are read-only to it. |
| 7 · Next attempt known-bad transitions eliminated | The admissible set shrinks, bounded by rule retirement. | The learner can only narrow and reorder; human orders outrank it. |
| Success preserve it | V2 keeps both sides of every outcome, so good actions are remembered too. | The drone keeps a change only if every lap comes back cleaner and faster. |
Never pay for the same lesson twice, unless the lesson is wrong. Freeway shows a lesson that is correct and still costly: the chicken learns that moving up is dangerous, and stops scoring. That is why a fail-first model needs rule retirement and, eventually, a price on recoverable failure, and why the learning is fenced by a floor it cannot widen (fail-safe).
What it became
The mechanism on this page, a memory of failures that can only remove options, is now the learning half of a larger model. Both clips below play by themselves; both are recordings of the real consoles.
The line from Atari to the Fail-Safe Model
| step | result | what it taught |
|---|---|---|
| Arcade floor v1 Space Invaders · Freeway | +32% · −12% | Failure memory helps where failure is terminal and hurts where the risky action is the only useful one. The frozen result below. |
| Rule retirement the 54-rule collapse | 30 rules → 6 dead ends · 50 → 12 · retirement → 0 | Every rule individually justified, collectively paralysing. Retiring rules by coverage takes the dead ends back to zero. Squads → |
| Evidence tiles every veto traced | 4,280 cards → 30 rules → 203 Peel tiles | A refusal walks back to the individual deaths that earned it. Evolution → |
| VDSG · DOOM rule-based, no neural network | random 3.2 · rules 17.9 · +memory 20.3 (t = 0.78) | The rules win decisively; the memory layer on top is parity, not a win, and is reported as such. Evidence → |
| VDSG · orders DOOM and Wolfenstein 3D | orders only narrow · the learner is bounded | Stated as propositions in the VDSG paper: no order and no amount of experience can add an action. |
| VDSG · Fallout (1997) scripted scene, real decision loop | 49 deaths at one guard → dies once | Credit assignment again: blame had to reach the line that started the fight, not the heal inside it. Then it chose the peaceful line. |
| Drone course simulation, held-out starts | 10 / 10 laps, 0 contacts · baseline 1 / 10 | The learned plan sets the speed; the stop-distance limit and reflexes act after it and can only slow it down. Flight demo → |
| Peel, the Fail-Safe Model the name for all of it | learns from failure · cannot widen its floor | Fail-first is how it learns; fail-safe is what that learning may not change. The science and the math → |
What carried over, and what did not
- Carried over: learning only removes. On Atari the memory could only take actions away. In VDSG that became a theorem with tests: the learner may rewrite what it believes works, never what it is authorised to do.
- Carried over: every refusal cites its evidence. The counts on this page became chained Peel tiles, and a veto walks back to the deaths behind it.
- Fixed: the saturation collapse. The 54-rule result below is still true of v1. Coverage-aware retirement is what fixed it.
- Repeated, and caught: credit assignment. Atari blamed frames after the ship was already destroyed; Fallout blamed the heal instead of the talk. Both produced believable curves. Both are published.
- Not solved: risk as the objective. Freeway is still the open problem. A floor that prices recoverability rather than death has not been built.
- Known gap in this prototype. The fail-safe guarantee is claimed for VDSG, where tests pin it. In the arcade prototype the parameter-search layer is not yet separated from the floor, so the guarantee is not claimed here.
The cost of a failure is not a property of the action.
We gave a symbolic floor a memory of its own failures and pointed it at two real Atari 2600 ROMs. Raw pixels, no emulator RAM, no weights and no gradients. On Space Invaders it is worth +32%. On Freeway the identical mechanism loses 12%. The pair is the finding, and the reason is not that one game is harder.
Two frozen experiment cards · three arms measured · nothing tuned away · watch a refusal happen →
What the mechanism is
A death writes a card: the bucketed situation it happened in, and the action taken. When a situation accumulates enough evidence, the card becomes a rule that removes that action from the admissible set at decision time. The pilot proposes; the floor decides what the pilot is allowed to propose from.
Every rule cites the experience IDs that produced it. A refusal reads died 13 of 19 times in this situation — 3.6× the base rate, not a number between zero and one. There is no neural network in the loop and nothing latent: the rules are the entire learned state, and deleting a row changes the behaviour.
This is the part a confidence score cannot do. A classifier can tell you it is 0.94 sure. It cannot tell you which nineteen experiences made it sure, and you cannot delete one of them.
The numbers
Three arms, two games. V1 learns only from failure — a bad action is excluded. V2 keeps both sides of every (situation, action) pair and ranks by utility, so nothing is permanently excluded and later evidence can un-rank a bad option. V2.1 adds a hard floor for catastrophic outcomes on top of V2’s utility.
| arm | Space Invaders | Freeway |
|---|---|---|
| pilot only (no learning) | 139.4 | 10.3 |
| V1 — avoidance only | 200.6 +32% vs its own 152.2 baseline | 9.2 −12% |
| V2 — utility only | 128.8 −8% | 10.2 parity |
| V2.1 — catastrophic floor + utility | 121.9 −13% | 5.0 −51% |
Read within a row, not across cards. The Invaders pilot baselines differ between the two experiment cards — V1’s A/B ran to natural game-over (cap=20000), V2’s used cap=3000. Comparing 200.6 to 139.4 across cards would be wrong.
V2 fixed the regression and lost the win. V2.1 lost both. That is written down here rather than tuned away.
The finding: what a failure destroys
The difference between the two games is not whether risk can be traded off. It is what a failure takes away from you.
Space Invaders — terminal
A death removes every remaining reward in the episode. Its true cost is therefore the expected remaining return, not the local setback. V1’s absolute veto priced this correctly, by accident — an infinite penalty is right when the loss really is everything.
Freeway — recoverable
A collision knocks the chicken back and play continues. The real cost is the lost ground and nothing more. V2’s utility ranking priced this correctly; V1’s veto did not, and removed the only scoring action.
One mechanism gets each regime right for the right reason, and cannot get both at once. A fixed failure penalty — any fixed penalty, including a confidence threshold — is wrong in one of the two regimes by construction.
On Freeway, adding a catastrophic floor (V2.1) simply reconstructs V1: several up signatures sit above 80% observed fatality, the floor vetoes them, and the chicken stops playing. There is no catastrophic tail to exclude when the fatal action is the only scoring action.
The result we did not expect
V1 on Invaders peaks at 276.2 around 50 episodes, then declines to 246.7 by 100 — with a bounded working set of 30 rules. Remove the bound and 100 episodes collapse to 167.1 with 54 rules.
Every one of those 54 rules was individually justified by real deaths. Individually lethal, collectively paralysing. The admissible set only ever shrinks, so a floor that never retires a rule eventually refuses everything.
This is the practical failure mode of every safety layer we know of, and it is the reason safety systems get switched off in the field. It is also the argument against our own approach, so here is the number: 30 rules helps, 54 rules is worse than not learning at all. Bounding the working set is not a tuning detail — it is load-bearing.
Five defects, each of which produced a plausible wrong answer
The experiment cards record every defect found, because each one silently returned a believable result before it was caught. The largest:
ALE drops the lives counter at the end of a 127-frame death animation. Every failure card was therefore written from frames in which the ship had already been destroyed. Of 374 blamed frames, 0 had a bomb visible. At the true impact frame, 17 of 17 did. The floor was learning from the wrong moment entirely, and still produced a curve that looked like learning.
We report this because a result that only ever went up would be less trustworthy, not more.
Where this sits
Removing unsafe actions from an agent’s choice set at runtime is an established idea. The closest modern line is shielding (Alshiekh et al., 2018, Safe Reinforcement Learning via Shielding) and the constrained-MDP and safe-RL work around it. Learning rules from failure is older still — explanation-based learning and control-rule learning in PRODIGY (Minton, 1988), case-based failure-driven memory in CHEF (Hammond, 1986), and Ripple-Down Rules (Compton & Jansen, 1990), where each rule is added in the context of the case that broke the previous one.
We are not claiming the category. What is ours is narrower and, we think, more useful:
- The shield is learned from raw pixels, as readable rules. Learning a shield from failures has been done before (Shperberg, Liu & Stone, 2022, and their rule-based follow-up at CoLLAs 2022), so we do not claim that. This one is derived from 210×160 pixels with zero emulator RAM, and every rule is a readable count that cites its events. (Corrected 2026-09-26: an earlier version of this page claimed learned shields as ours.)
- Every veto is individually auditable. Shielding gives a specification-level guarantee. It does not give a reason for a particular refusal. This does, with counts.
- The saturation point is measured. 54 individually-justified rules perform worse than no learning. We have not seen that quantified.
- The failure regime is published. A learned shield degenerates when risk and reward share an action — with the number, on a second ROM, under an identical mechanism.
Status, honestly
PROTOTYPE — research result. This is a frozen two-experiment finding on Atari ROMs. It is not a product, it is not deployed, and nothing here has been validated for a safety-critical path. The mechanism transferred between games; the learning did not. Anyone citing the +32% without the −12% is misreading it.
V1 is frozen at freeze/arcade-floor-2026-09-25; V2 cites V1 and does not replace it. Neither card is edited to make a later metric look better — a v3 gets its own card and cites both.
Open problem. A floor that prices recoverability rather than death rate — one that refuses what cannot be undone and permits everything else right up to that edge. Freeway is the benchmark: a mechanism that understands recoverable failure should beat the 10.3 pilot outright, not merely reach parity.
Since this result: the mechanism is now the learning half of Peel, the Fail-Safe Model, and runs in VDSG. Still a research prototype, simulation and games only.