Every decision leaves a written reason. Read it at the AAR.
A hostile controller tried to crash the vehicle in the CARLA driving simulator. The floor overrode all 64,952 of its bad commands over 40 km. 0 collisions. Pull the floor and the same commands crash within seconds.
When something goes wrong — or nearly goes blue-on-blue — the first question is who decided, and why. This system answers at the moment it decides: which moves it had, which it refused, and on what evidence. The system writes it as it runs. Nobody reconstructs it afterward from memory.
Test it without trusting it: shadow mode
You do not hand a new system authority on day one. You do not hand it authority until you have seen what it would have done.
In shadow mode the floor is switched off, but it still records its verdict. Run it against a recorded mission and you get a full “what it would have done” log: every move it would have refused, with the evidence it would have cited. It changes nothing about what actually happened.
It costs nothing. It cannot affect an outcome. And it answers the only question that matters before you trust a safety layer: what would this have done to my mission?
Two numbers come out of it, and they are the two to ask for. How often would it have blocked something your people did anyway, with no harm done (it gets in the way)? And how often would it have blocked something that came right before a bad outcome (it would have helped)?
What the record gives you
Accountability for an autonomous system is usually a document somebody writes about it after the fact. This is a record the system produces while it runs, and it replays exactly.
- Every decision. Not a summary. Each move it allowed and each move it refused, at the moment it decided.
- With the evidence. Each refusal carries the rule and the events that earned it — in the measured case, 30 rules traced to 203 linked evidence records.
- Replayable. Same recording in, same verdicts out. An investigator can check it instead of taking anyone’s word for it.
- Kept even when switched off. So you can evaluate it for as long as you want before it is ever given authority.
This is why the trace matters. When the question is fratricide, or who is accountable, a system that cannot show its reasons leaves the humans around it with nothing to point to. The people in command keep the judgment. The record is how that judgment gets checked afterward, decision by decision.
What it looked like with a hostile driver
In the CARLA driving simulator, a hostile controller drives toward a collision. The floor takes the vehicle, holds a gap of about 9.7 m, and logs roughly 400 refusals across 100–490 m at 38–40 km/h. Hits: 0. With the floor off, the same controller crashes — and the log still records what the floor would have refused on the way in.
That is a simulator, not a road. It shows the record works. It does not show the floor is ready for a real vehicle.
Next
Chapter 6 — where it will get you hurt if you trust it →