Mutants turn red

A test that passes says little if it would also pass on broken code. So the agents that write Coherent break their own code on purpose. A mutant is a copy of the code with one deliberate error. If the tests fail on it, the mutant “turns red”: the tests are shown to catch that error. If it survives and stays green, either a test is missing or the mutant changes nothing that matters. Both have to be explained before a change is accepted.

Who breaks what

The project's constitution and working rules do not mention mutants. They say that “passed” means reproduced, and that whoever delivers a change does not accept it. Mutants are how the agents meet both rules in practice. The earliest records date from 2026-09-17, among them a commit: “11 mutants, all red”. In the log in which every delivery is graded, a red mutant counts as evidence of content, not of form.

The roles are split. The agent that implements a change builds mutants and reports how many turned red. The agent that accepts the change runs them again itself. A reviewer from another model family builds mutants of its own. A few working rules came from experience: mutants live in a throwaway copy, never on the main branch, because production reads from there; they target logic, not constants; and every condition also gets a mutant that deletes it entirely.

When a mutant stayed green

The useful cases are the ones where a mutant survived.

A pause that ended for the wrong reason (2026-09-26). The pipeline pauses the generation of training data while a candidate is waiting to be sent to the oracle; the first dispatch is supposed to end the pause. The mutant “delete the line that ends the pause on dispatch” passed every test. The test ended the pause only because no other candidate was left waiting. In operation that day, 10 were waiting, and without the condition every pause would have run to its 5-hour limit. The test got a second waiting candidate, and the mutant turned red.

A flag nobody checked (2026-10-03). A reviewer from another model family checked new code for generating training data. 9 of 10 mutants turned red. The survivor showed that no test checked that a warning flag stays off for games whose opening was rejected. The test was added, and the mutant turned red.

A mutant the tests overwrote (2026-09-28). The first mutant for a change in the pipeline's timing set a waiting time to zero. It stayed green, because the tests overwrite that constant themselves. A second mutant, “no further attempt”, turned red. The change had already been merged before anyone looked at the first result, an error the accepting agent recorded against itself. The rule since then: look at the mutant first, then merge.

When everything was red and still wrong

All 17 red, 2 serious findings (2026-09-30). A new path in the oracle passed 166 tests, and all 17 mutants turned red. A reviewer from another model family still found 2 serious and 3 lesser errors. The serious ones: a repeat measurement could run without a new booking, and a shortcut for identical builds bypassed a check. In the second round each of the 5 findings got its own test, red against the old code; one old mutant was dropped and 6 new ones were added. Result: 22 of 22 red, 171 tests.

A reminder that no test covered (2026-10-01). For a change in how alarms reach the agents, the author's 21 mutants all turned red. The reviewer then built mutants of a different kind: ones that restore the old behaviour. One of them made no test fail at all; there was no test for an hourly reminder. Another turned red only because an existing test had been inverted to match the lost behaviour. Simulations over time found both. The second round fixed them.

Red mutants only prove the mutants someone thought of. On 2026-10-05 the project's sceptic, the agent whose role is to question every decision, wrote after an independent red team had found a remaining way around a fix: “I took the mutants I built myself as proof enough.” Its own rule since then: no verdict goes out before the red team, not even for a “mechanical” acceptance.

Green can be right

On 2026-09-18 a candidate change for the engine added a table for the endgame of king and pawn against king. All 196,608 entries agreed with the public Syzygy endgame tables, used as an independent reference: 0 deviations. To show that this comparison could detect errors at all, the accepting agent mutated the table's construction. Without the pawn's double step, 2,438 entries deviated; with stalemate counted as a win, 17,928. A second check ran the engine's lookup on all 662,704 legal positions of this endgame, again with 0 deviations. Errors in the side to move or in the mirroring of the board produced between 95,156 and 120,308.

One mutation stayed at 0: removing a clause at the promotion square. The clause only makes the construction reach its result faster; without it, the result is the same. Such a mutant is called equivalent. When a mutant stays green, that has to be explained, not necessarily repaired.

When the method itself measured wrong

On 2026-09-26 a commit reported 12 of 13 mutants red. It was wrong: one test run still executed a cached copy of the previous mutant, because two files of the same size were swapped within the same second. With a fresh cache, a mutant that ignored which workflow a run belonged to survived; the test was tightened and the report corrected. Since then, that agent runs every mutant with a fresh cache.

Mutants are not free either. On 2026-10-07 a new checking tool came with 28 tests and 28 mutants. The project's owner asked for less complexity; the new version kept 19 mutants, for the rules that decide.

A red mutant is no proof that a change is right. It is evidence that a test would notice one particular error. Here, the cases where a mutant stayed green, or where everything was red and an error remained, taught the most.