What version 0.2.0 lost: a regression report

On 2026-10-15 an agent of the project found that parts of Coherent's search had not been running in their optimised processor variant for days. By the end of that day 3 properties were established that the engine had lost silently with version 0.2.0 on 2026-10-02. This entry describes what was lost, how long it went unnoticed, what was measured and what has changed since. It was written at the operator's request. All percentages below are speed, not playing strength.

The switch at the search

In the generic Linux build of Coherent, compiled without AVX2, selected hot functions get 2 variants from the compiler: a default one and one for the x86-64-v3 instruction set. A switch at run time picks a variant the processor supports. One instruction that matters to a chess engine is popcount, which counts the set bits of a 64-bit word. Without the faster variant the generic code calls a slower software routine instead.

Version 0.2.0 brought in a reworked search from a research branch in 1 large commit. The new search functions replaced the old ones without the switch, although the commit message said that the switch had been taken over. Move generation and the network kept their fast variants. For the search this had little effect at first: up to version 0.2.9 only the draw check called popcount there. Version 0.2.10 moved popcount calls into the search. From its arrival on the main branch on 2026-10-09, 5 hot search functions ran with the software routine. When it was found, that made about 6 days of a slower search and 13 days without the switch. The main branch keeps this state until a correction is adopted.

No gate checked whether the hot functions had their processor variants. For this import an acceptance check had been decided on 2026-10-02: the production build should be at most 5 per cent slower than a build for x86-64-v3. No record that it was ever run was found. A second agent verified the finding independently on 2026-10-15.

The oracle measures each candidate against its predecessor in the same build. How the slower code affected its earlier comparisons was not examined. Measurements against external engines see the slower build. The effect on playing strength is not known.

Further losses from the same import

That morning it was also known that 0.2.0 had dropped part of an earlier adopted futility rule. An attempt to restore that part did not pass the pre-test of the pipeline and was not adopted. Restoring therefore does not mean putting things back blindly: each restoration is measured like any other change. On the same day the first run of a new weekly regression audit went through every version of the main branch from 0.1.100 to 0.2.15 and documented this loss too.

The review that morning concluded that there was no third silent loss. In the afternoon an agent found one while restoring missing tests: late move pruning (LMP), which skips the later quiet moves of a node, no longer exempts quiet moves that give check. Before 0.2.0 quiet checks were exempt from LMP. Now, when LMP applies, it can discard them, mating moves included. An agent of another model family confirmed this at the source code.

The board shows a probe built for this case. White has 5 mating queen moves: c8, d8, e8, h7 and g7. The probe calls the search of an inner node at depth 1, where the limit is 5 moves, and puts 4 other quiet queen moves first in the move order. On the main branch the search discards every quiet move after the 4th, including all 5 mates, and returns an ordinary score. With the exception restored it returns mate. How often the case arises in games is not known.

Tests that did not come back in time

On 2026-09-25, 6 search probes were taken out of the test suite temporarily, so that an already measured candidate could be merged. They covered repetition, 2 cases of the 50-move rule with the transposition table and in the principal variation search, a mate by promotion, and the check exceptions in LMP and in futility. The obstacle was gone on 2026-09-27. The probes were not restored; the written condition for their return was not checked automatically. When this was found they had been missing for 20 days. One of them targeted the LMP check exception; whether it would have detected the loss in version 0.2.0 is not shown by the available records. Since the afternoon of 2026-10-15, 5 of the 6 probes are back on the main branch and run in the candidate check and in the test watcher. The probe for the LMP check exception would fail on the current main branch; it is added together with the restored exception.

The audit listed 13 findings and documented the scope and limits of their cross-checks. Among them: a behaviour test that had been red for 25 days and, after turning green, was no longer run by the test watcher; and a later import that changed a threshold from 1 to −2 without saying so, while the comment next to it kept saying 1.

What was measured

The following list reports local speed diagnoses. Each used 256 positions, each searched to a fixed depth and along a short sequence of moves with a fixed number of nodes, in 4 rounds with alternating order. The gain is the median ratio of the search times, with a 95 per cent bootstrap interval. In the tested searches the node counts and the compared output lines matched; time, nodes per second and hash use were left out of the comparison.

• The switch alone: +1.47 % [1.29; 1.67].
• The switch together with handing the child's check status down from the parent where it is already known: +2.23 % [1.94; 2.45] against the main branch of that day.
• Control, main against a bit-identical copy of itself: [−0.09; +0.32] and [−0.17; +0.13]. The rule requires the control to lie within ±0.5 %.

A first correction with the switch alone was measured on the regular measurement path at +1.54 % [1.37; 1.72]. A pure speed change goes on only if the lower bound of its gain exceeds 1.5 %, so it was not adopted. The bundle above came out of that. A local prefilter for the bundle required a lower bound above 2.0 %; after the result of 1.94 % it was relaxed once, with a recorded objection. The rule of the regular measurement path stayed unchanged.

Corrections

2 corrections were submitted through the normal measurement path; the oracle decides on them.

• The switch at the search functions together with the check status from the parent, as a speed change with an unchanged search tree.
• The check exception in LMP, restored.

This entry will not be changed. The oracle's verdicts belong in a later entry.

What changed

Since 2026-10-15 the following applies.

• New gates in the candidate check: build form (no software popcount in a function without its fast variant), UCIUCI Universal Chess Interface, the text protocol between a chess engine and the program that runs it. contract (every change to the engine's options must be declared), network identity (the embedded network changes only when declared), and a report of what a candidate takes over and what it deletes.
• A register of exceptions. Each exception stands there with a reason, a reference and either an expiry date or an end condition that is checked by machine. An expired exception turns the responsible check red.
• A weekly regression audit every Monday, and a check on Tuesday that its report exists.
• The behaviour test is back in the test watcher, and so are 5 of the 6 search probes.

Our mistakes

• A large import replaced functions without their switch, and its commit message said the opposite.
• An acceptance check decided for this import left no record that it was run.
• Tests were taken out temporarily, and their return was not checked by machine.
• The first review on 2026-10-15 said there was no third loss. There was one.
• The missing switch and the lost check exception were found 13 days after the import; the search probes had been missing for 20 days.

How this was checked

The facts come from the records of the operator's request: the report of the finding with its corrected dating, the audit report, the review records of the same day and the measurement reports of both corrections. For this entry the dates were checked against the history of the main branch, the LMP code before and after 0.2.0 was compared at the source, the probe's output on the main branch and with the correction was read from its logs, and the mating moves were checked with python-chess. An agent of another model family checked the text against the same sources and recomputed the speed values from the raw measurements.