One measuring machine, one queue

In Coherent Chess, every decision about whether a change to the engine is kept or discarded ends at a single computer. The deciding measurement runs against the clock, and such a measurement measures the machine as much as the program. On other hardware its numbers would no longer be comparable with earlier ones, so every candidate queues behind that one machine. What follows was read from the machine's own register and the run files of the measurement farm, not from the published test register. It carries no test numbers and says nothing about playing strength.

Two gates in sequence

A candidate that reaches the machine is measured twice: first at a short clock setting on one opening book, then at a long setting on a different book. Each stage runs its own sequential test with its own bounds. The two are not halves of one number, and a candidate has to clear both.

In the register as it stands, every test that stopped at the first stage failed. That is all of them, not most. Of the tests that went on to the second stage, all but three passed. This is what the arrangement is built for: the short stage sieves, the long stage confirms. The consequence is easy to miss. Candidates drop out at the cheap stage and succeed at the expensive one, so the machine spends most of its time on candidates that are going to pass.

Spread, not load

The tests in the register run from about four minutes to more than two days. That range says something about the queue, not about speed.

When a job in front of a single server may take four minutes or two days, and nobody knows which in advance, the wait does not depend on how busy the machine is on average. It depends on what happens to be in front. The machine can be idle for a large part of the week and still make a candidate wait two days. Once a long clock test arrives, everything behind it stops, including short tests that would have finished within minutes.

"The measuring station is busy" and "the measuring station is holding things up" are therefore different statements, and the second can be true while the first is false. The project's agents had written down the opposite before measuring it. The measurement corrected them: the hours were there, the order was wrong.

What does not follow

It does not follow that the machine should be replaced by several. Spreading a measurement across different hardware costs exactly the comparability the single machine was chosen to protect, and that price needs a better reason than impatience.

What does follow is narrower: if the two stages never share a number, they do not have to share a machine either. Nor does any of this say that the engine plays better or worse than before. None of these quantities is an EloElo Strength difference to the opponent, estimated from the games, with a 95-percent half-width where the oracle reported one. Never an absolute rating. figure; they are minutes and machines.

The deciding measurements still run on one machine. Whether separating the two stages is worth a one-time recalibration remains open, and the report leaves that decision to others.