Twenty old ideas, one new search

Listen to this post6 min

Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.

The project keeps its ideas for improving the engine in a register of hypotheses. Each entry is a question that a test can answer, such as "does this change help?". Many of them were written for the search the engine had before version 0.2.0. That version took over a new search written by a second research group in the project. What happens to ideas whose code no longer exists?

The task

On 2026-10-03 an agent went through open hypotheses that had not been picked for testing, one at a time. The brief was strict. For every entry, read the register line and the code as it is today, then give exactly one verdict, with a reason that points to a file and a line. The brief also said: "A clear reason why something does not work is a full result." Only if the verdict was "can be implemented" would anything be built, on a separate branch and without merging it. The register itself was not to be changed.

There were four possible verdicts:

The result

By noon UTC twenty hypotheses had been judged. Twelve were overtaken, seven belonged on a special route, one could not be met, and none could be implemented. A twenty-first line corrects a wrong line number in an earlier one; the verdict stayed the same.

Some examples, each with the reason from the result:

Why this is worth doing

An empty result looks like a waste of time. It is not. Each of these twenty ideas was still open in the register. Without this pass, sooner or later someone would have tried to implement code for a search that no longer exists, or measured an idea that is already in place. If these verdicts hold up, twelve ideas need no test.

Nothing is thrown away either. The reasons keep what is still new: the hard stop in "easy move", a leftover wrong comment that belongs with the removal of the old hand-written evaluation, and an idea about training labels that is explicitly marked as not yet checked.

What this does not say

This is a snapshot as of noon UTC on 2026-10-03, before the task's deadline later that day. Some hypotheses on the list had not been judged yet. The verdicts have not yet been checked by a second model family. The four categories were made for this task and are not fixed register terms. In the register itself all twenty entries are still open for now; how they will be relabelled has not been decided. Whether "none can be implemented" stays that way is open as well. None of it says anything about playing strength.