Where a number comes from

On 27 September 2026 a report in the Coherent Chess project was withdrawn and recomputed. One of its figures had been read out of a configuration file instead of being measured. It looked like a measurement and sat in a file full of real measurements, but it was an estimate someone had typed in months earlier so that jobs could be sized. Everything below comes from the measurement farm's run files and configuration, not from the test register, and none of it concerns playing strength.

Estimate versus measurement

The configuration that describes a measurement job has a field for the expected seconds per game. The scheduler uses it to plan how many jobs fit on a host and how long a batch will take. The word "estimated" is part of the field's name.

The run files record what actually happened: wall time and the number of games played in parallel. From these two values the real cost per game can be calculated. Across all finished jobs of this kind, more than 15,000 of them covering more than 1.5 million games, the measured cost is a little over a fifth of the configured estimate.

Nothing was broken, and the estimate was never wrong for planning, its purpose. A planning constant is not kept in line with reality, because nothing depends on it being accurate. Once it is used as a measurement, everything computed from it inherits an error nobody can see, while the result still carries units, a source file and a decimal point.

A field that means less than its name

The same run files contain a field for processor seconds, the obvious input for a cost calculation. For these jobs it reports only a small fraction of the processor time actually consumed, because of how it is collected. Taken at face value, it would have understated the load of a measurement job by more than an order of magnitude. For a different kind of job in the same place the field is accurate, so a single check on the wrong sample leads to a confident and wrong conclusion.

The report draws a narrow lesson from this: a field's name tells you what it was for, not what it measures. Before a number enters an argument, it has to pass one question: what would this value look like if it were wrong? For processor seconds on a job that keeps a core busy for four minutes, one line of arithmetic would have answered it. That arithmetic was not done.

What changed

Every quantity in a report from the project now carries a mark, measured, planned or unknown, together with the place it came from. A number without that mark counts as a claim, not a finding. The rule is older than this mistake; it exists because of earlier ones.

What this does not say

The corrected figure is a cost per game on particular hosts at a particular setting. It does not show that the measurement farm is faster or slower than anyone assumed; it replaced a wrong figure with a correct one, not a bad outcome with a good one.

Nor were the earlier reports useless. The conclusion the review corrected was a recommendation about which machine should do which job, and the correction changed the size of that argument, not its direction. According to the report this is the ordinary case: most wrong numbers do not reverse a conclusion but leave it unsupported, and an unsupported conclusion that happens to be right is still not a finding.