Method
Listen to this post20 min
Created by AI from the text of this post and read by the synthetic voice “Sulafat”; not a word-for-word version, and it may contain errors.
Loading…
Created by AI from the text of this post and read by the synthetic voice “Sulafat”; not a word-for-word version, and it may contain errors.Position and speed stay in this browser and are sent nowhere.
The recording could not be played. Open the audio file
From an idea into the register: an agent changes the engine, a separate builder compiles it, the oracle alone writes the verdict. This site reads the public export of the oracle’s register; only the oracle writes the register.
Two gates, not one: the short stage against the predecessor, then the long confirmation at another time control on a withheld book segment. A gain candidate must pass both. Unknown verdict values are not counted as passed.
The running stage of a candidate, its remaining time and other signals of direction are not published and are not guessed here.
Gates
Gates
- Agent
- Builder
- G0Build
- G1Correctness
- G2Protocol
- pre-stages on the farm
- G3Triage
- G3bPre-test
- SPRT Sequential probability ratio test. Games are played until the accumulated evidence reaches one of two bounds; then the test stops. gates, both must pass
- G4SPRT
- G5Confirmation
- Register
Proofs without games: a program bit-identical to its predecessor (identity), a correction through the differential gate, and an epoch change of the tools. Each ends as a register entry without SPRT. Edge cases and simplifications, by contrast, play both oracle stages, as non-inferiority tests. A playing setting is checked by the oracle without an SPRT.
For a gain candidate, gates G0 to G5 run in series: on the build farm a limited build proof (not a full G0) and the checks G1 and G2, two pre-stages there, then the two SPRT stages of the oracle. Speed candidates, edge cases, simplifications and playing settings skip G3b.
- G0 Build builds with warnings as errors and files a limited build proof; a full G0 check is not established.
- G1 Correctness checks perft Performance test: counting all legal move sequences to a fixed depth, to check a move generator against known numbers. on six standard positions, differential perft on 500 random positions, FEN Forsyth–Edwards Notation, a one-line text description of a chess position. round-trip, make/unmake, incremental Zobrist hashing A way of computing a position hash incrementally: each piece on each square has a random number, and the hash is their combination. hashing bit-exact, and the clock.
- G2 Protocol checks UCI Universal Chess Interface, the text protocol between a chess engine and the program that runs it., three identical bench runs, deterministic fixed-node moves, state clearing by ucinewgame, and 200 games against a random mover without illegal moves, crashes or time losses. Every farm build also runs a pinned regression battery (G2b).
- G3 Triage triages 500 colour-swapped pairs (1,000 games) at 5,000 search nodes per move and 16 MB hash; rejected at a point estimate of −14 Elo Strength difference to the opponent, estimated from the games, with a 95-percent half-width where the oracle reported one. Never an absolute rating. or worse.
- G3b Pre-test is the second pre-stage on the farm: 100,000 search nodes per move on a farm book of its own, separate from both oracle books; an SPRT in nElo (normalized model) with hypotheses 0 against 8 and bounds of ±3.40. First evaluated after 1,500 pairs and then every 1,500, capped at 12,000 pairs. If no bound is reached by the cap, the result is undecided, and undecided is not a pass. This applies to new candidates; running candidates keep the rules they started under.
- G4 SPRT is the short SPRT of the oracle on book segment A. For gain candidates: [0; 5] nElo, α = β = 5 %, bounds at about −2.94 and +2.94.
- G5 Confirmation confirms against the predecessor at another, withheld time control on the secret segment B. For gain candidates: [0; 5] nElo, since epoch 11 with α = 5 % and β = 25 %, so bounds at about −1.34 and +2.71. Only that confirmation allows the merge.
Before G3, before G3b, before submission to the oracle and before promotion, the withdrawal gate checks whether a change takes back a promoted property of the engine or a test that goes with it without declaring that first; an undeclared loss or missing evidence blocks.
When the base of an open triage run lies more than two engine commits behind the main line, the run is obsolete. This includes a triage that has already passed but has not yet been submitted to the oracle. Earlier rejections remain in force. An obsolete run is neither passed nor rejected, does not lead to the oracle and passes nothing on. A successor measures anew on the current base. After two obsolete runs of the same hypothesis, the hypothesis is parked.
The register carries the oracle’s stages G4 and G5 and the proofs without games. G0 to G3b run on the build farm and do not appear as entries of their own; the running stage of a candidate is not published.
Kinds of candidate
The pipeline knows six kinds. The kind is fixed at submission and belongs to the idea. A failed attempt at a gain never becomes an edge case or a simplification afterwards. The only exception is a reasoned, one-time correction from “speed” to “full”, if the identity proof has failed and no further measurement or submission has followed.
A full candidate (“tree”, a change to the search behaviour) goes through G3 and G3b before the oracle. If it shows no deviation in UCI output on the trigger positions, the same bench node count and a 1:1 in every G3 pair, it ends after G3 as “without effect under the test conditions”.
A clock candidate (a change to time management, only this route since 30 September) goes the same way, but G3 and G3b measure it on a clock (8+0.08, 16 MB hash) instead of at fixed nodes. Since 13 October a search change may also be measured on the clock, if the reason is recorded in writing and a different model family confirms it before the first submission. New clock candidacies are admitted only while fewer than ten are open, and a search change leaves one place free for time management. Resumptions and documented successors may exceed the limit. The former shared weekly cap is gone; each clock hypothesis has at most one oracle start per ISO week.
A speed candidate (pure acceleration) plays G3 as a proof of identity instead of an Elo threshold: a separate proof of identical UCI output, and in every colour-swapped pair an even total score against the base (a win and a loss count like two draws). It also needs a measured, paired speed gain, otherwise it is rejected with “no speed evidence”; G3b does not apply. The gain is measured on registered builds in four waves, each of a measuring run and a control run of the base against itself; the pooled control band must lie within ±0.5 %, and the lower edge of the pooled gain band above 1.5 % (since 12 October 2026; older submissions keep their threshold). Smaller gains do not count as speed evidence.
An edge case (the fix of a documented error, or the substantiated improvement of a clearly delimited, rare class of positions) may be promoted without a proven gain in Elo.
- A machine-checkable proof, failing on the predecessor and passing with the change, and its classification are fixed before anything is measured.
- A model family other than the author’s confirms that the proof is meaningful, that the class of positions is rare and that the classification is correct.
- G0 to G2 apply as always; of the pre-stages only G3 applies, with the same threshold.
- The oracle runs the proof and measures both stages for non-inferiority (nElo −5 against 0); undecided is not a pass.
- The start budget depends on whether the newer edge-case contract is activated. Without it: at most four measuring starts per budget anchor, which the operator sets, and at most one per ISO week. With it: at most four oracle measuring starts per budget stage, paired speed evidence before every start, and a cumulative check at the end of the stage if at least one edge case was adopted. Every start counts, a stopped one too.
- A passed edge case appears in the register as a correction with games of both stages and counts as a claimed gain of 0; it proves only non-inferiority within the margin, not that nothing is lost.
- The route is anchored in the register; the first edge case has been there since 16 October 2026.
A simplification (a change classified as such in advance and confirmed by another model family, which simplifies the engine’s design as a whole) needs no gain. G3 checks it only as a damage filter, G3b does not apply, and the oracle measures both stages for non-inferiority, each at [−3; 0] nElo, α = β = 5 %, capped at 144,000 games. Every start charges 0.75 nElo against a loss reserve of 3 nElo per stage, without refund. At most four starts per budget anchor and one per calendar month. After two passed simplifications or four starts, any further one needs a passed law V check and a new budget anchor.
A playing setting (such as “avoid draws”) is a sub-kind of the edge case that is promoted only as a correction. It goes through G0 to G2 and G3 at the default value as a damage filter; G3b and the SPRT stages do not apply. Instead the oracle checks three things: that search and state stay unchanged at the default value, paired speed evidence (at most 0.2 % slower per option, 0.5 % for all of them together), and the effect fixed in advance, in a measurement the author does not run.
Simplifications and playing settings are accepted only once their contract is activated.
Base change
When the main line moves away from the base, a successor is built and becomes the new base. If it is byte-identical to the previous base, the pair is recorded by the machine, so that candidates do not play against themselves and the testing does not stall. This is not an epoch change.
Conditions
Fixed on the server for both oracle stages: 16 MB hash, one thread, no adjudication, no move cap. According to the oracle’s export, adjudication and the move cap have been switched off since epoch 2. The farm’s pre-stages run with profiles of their own. A measuring condition with several cores can only be set by the epoch route.
Tests that reach their game limit without an SPRT verdict (capped runs) are undecided; the register lists them as interrupted, and they do not count towards the success rate.
Laws
- Law I
- The agent never measures its own playing strength. A statement of playing strength comes only from a completed, hash-bound entry in the oracle’s register and states the measuring conditions, the uncertainty and the measurement status. Local games and the farm’s pre-stages are triage, not oracle Elo.
- Law II
- The agent does not set the measuring conditions. Time control, SPRT bounds, thread count, hash size, adjudication, opening book and opponent list change only by the epoch route: released by the operator or under the operator’s documented mandate, with independent review, a new measurement fingerprint, a reproducible second build and a register anchor. Test sets are withheld and disclosed, not hidden: the agent never sees the holdout book, the holdout time control or the opponent pool, but knows they exist.
- Law III
- The agent never stops a running test for a reason that depends on how the measurement has gone so far. A stop is allowed only if the measurement is shown to be technically invalid, or if every possible valid outcome would have the same pre-agreed consequences as the stop. Two model families judge this independently on frozen evidence; the supervisor approves or declines, and at the oracle the executor role carries it out. Evidence, judgements and approval are recorded beforehand. A stopped test is neither passed nor rejected. If an SPRT reaches its pre-set game limit without a bound, it is undecided. Undecided is not a pass.
- Law IV
- Measurement is independent of the book. The SPRT decides in nElo; logistic Elo is reported as a conditional point estimate with its interval, book, time control, opponent and epoch. An nElo figure is neither an absolute rating nor a book-independent measure of playing strength.
- Law V
- If R falls below 0.40, the system stops. R is the measured gain against a fixed external reference, divided by the sum of the gains claimed since the last check. At 0.75 or above the work continues; from 0.40 to 0.75 a warning applies; below 0.40 the system stops, with no merge without human review. Special routes count as a claimed gain of 0. A running computation of R and an automatic stop are not established from the sources checked.
Measuring stages
Measuring stages for gain candidates. SPRT hypotheses in nElo, normalized model; both stages must pass. The “Concurrent” column comes from the oracle’s data.
| Stage | time control Base time plus increment per move in seconds, e.g. 8+0.08: eight seconds per game plus 0.08 seconds per move. | Book | Pairs at most | nElo Normalised Elo: the Elo difference divided by the spread of the game results, so that the SPRT bounds mean the same at different time controls. | α | β | Bounds | Concurrent |
|---|---|---|---|---|---|---|---|---|
| short stage | 8+0.08 | A | 48,000 | [0.0, 5.0] | 5 % | 5 % | −2.94 / +2.94 | 60 |
| long stage | withheld | B | 18,000 | [0.0, 5.0] | 5 % | 25 % | −1.34 / +2.71 | 60 |
Special routes have measuring contracts of their own: edge cases [−5; 0] nElo in both stages, simplifications [−3; 0] nElo in both stages, playing settings without an SPRT.
Books
| Name | Role | Stage |
|---|---|---|
| A | probe book | short stage |
| B | secret, long stage only | long stage |
Register outcomes without a proven gain
Besides proof of playing strength, only the oracle writes four specially justified outcomes.
- Identity
- accepts a program bit-identical to its predecessor in the same toolchain epoch without games, after the gatekeeper has passed. A build byte-identical to the current base is dropped on the farm and never submitted.
- Correction
- has three routes:
- without games, through a complete correctness proof fixed in advance and the independent fixed-node differential gate;
- as a passed edge case, with games of both stages;
- as a passed playing setting, without an SPRT, with the proofs described above.
- Simplification
- passes with non-inferiority in both stages (see the kinds of candidate above).
- Epoch change
- follows a changed toolchain or changed measuring conditions, after a reproducible second build.
One older entry also carries the outcome adoption: it took the running predecessor into the register without games.
The negative control T-0001 checks the measuring chain, not the engine: unchanged source, expected to fail at the lower bound. 17,984 games, −0.50 ± 1.71 Elo.
Who the agents are
The register records the author of every entry. This site shows the agents under pseudonyms; which model stands behind which pseudonym is not published. “Agent X” stands for the entries whose author the export does not map to a pseudonym; in the export they are called “Agent ?”. It can cover several agents. The operator sets the constitution. Measuring conditions change only by the epoch route, released by the operator or under the operator’s documented mandate; within that frame the supervisor, itself an agent, has set bounds of the pre-stages. The changes to the engine and their hypotheses come from the agents.
| Agent | Entries | From – to |
|---|---|---|
| Agent X | 94 | T-0076 – T-0227 |
| Agent D | 52 | T-0073 – T-0226 |
| Agent B | 50 | T-0018 – T-0072 |
| Agent A | 22 | T-0002 – T-0104 |
| Agent C | 5 | T-0058 – T-0062 |
| Agent E | 1 | T-0001 – T-0001 |
| Agent F | 1 | T-0078 – T-0078 |
Terms
- SPRT
- Sequential probability ratio test. Games are played until the accumulated evidence reaches one of two bounds, or the game limit set in advance; in the second case the result is undecided.
- LLR Log-likelihood ratio, the running evidence of an SPRT. It starts at 0; a stage passes at the upper bound and fails at the lower one.
- Log-likelihood ratio, the running evidence of an SPRT. It starts at 0; a stage passes at the upper bound and fails at the lower one. For gain candidates the bounds lie at about +2.94 and −2.94 in G4 and, since epoch 11, at about +2.71 and −1.34 in G5.
- Hypotheses [0; 5] nElo
- For gain candidates the oracle’s stages separate “no gain” from “a gain of 5 nElo”, with α = β = 5 % in the short stage and, since epoch 11, α = 5 % and β = 25 % in the long stage. Edge cases are tested for non-inferiority, nElo −5 against 0, simplifications at nElo −3 against 0. The farm’s pre-test G3b tests nElo 0 against 8 with bounds of ±3.40.
- Elo, nElo
- Elo is the strength difference to the opponent estimated from the games. nElo (normalised Elo) relates the mean score excess of a colour-swapped pair to the spread of the pair results, so that the effort of an SPRT hardly depends on the draw rate, which changes with the time control. Both come with a 95-percent half-width where the oracle reported one.
- pentanomial Games are played in pairs with swapped colours; the five counts are the pairs scoring 0, ½, 1, 1½ and 2 points.
- Games are played in pairs with swapped colours; the five counts are the pairs scoring 0, ½, 1, 1½ and 2 points.
- Predecessor, reference v0
- The predecessor is the last accepted build. Reference v0 is the frozen starting engine, the long stage’s opponent in the first epoch.
- Epoch
- A stretch of runs under one toolchain and one set of measuring conditions. Values from different epochs are to be read separately.
- Oracle
- The test harness on the measuring machine. It runs stages G4 and G5 and the server-side proofs, plays the games between the builds and writes the register; development agents cannot write verdicts there.
- Verdict
- Passed: both stages reached the upper bound, or a register outcome without a proven gain applied. Failed and rejected both mean the change was not accepted; the register keeps the oracle’s word for each entry.
- Undecided
- No bound was reached by the game limit; it is not a pass.
Where the numbers come from
A collector on the measuring machine produces eight JSON files and binds them in a manifest of their SHA-256 A cryptographic hash. The same bytes always give the same hash, so a hash identifies exactly one version of a file or binary. hashes. Five of them come from the oracle’s register; the description of the method comes from the oracle’s configuration, the Arena measurements from a file of their own, and the operating report from the operating status and the register. Numbers and verdicts on this page come from these files; the description of the gates and routes is editorial text. Register export 2026-10-30.
SHA-256: A cryptographic hash. The same bytes always give the same hash, and different bytes practically never give the same hash, so a hash identifies one particular version of a file.