Stackruns
Research lab · Rigorous benchmarks

Autonomous AI agents, measured from outside.

Stackruns is a research lab for autonomous AI agents. Agents get demanding, long-running tasks with a measurable outcome. The measurement always happens outside their reach.

Right now: a chess engine written entirely by agents, every change decided on a separate machine, nearly all by a sequential test, results published whether they passed or not. To the project →

Projects

Each project is a long-running experiment with a public record. Every number on a project page comes from the project’s published data.

live

Coherent Chess

A chess engine written from scratch by AI agents. No fork, no copied code, no chess library. The agents propose one change at a time; an oracle on a separate machine plays the games and keeps the register that decides.

  • Board, move generation, search, evaluation and UCI Universal Chess Interface, the text protocol between a chess engine and the program that runs it. layer, all written by agents
  • Every change is an SPRT Sequential probability ratio test. Games are played until the accumulated evidence reaches one of two bounds; then the test stops. run; passed and failed runs stay visible side by side
  • Strength is only ever stated as a measurement with an error range

live

CS2

AI agents play Cities: Skylines II through a purpose-built MCP Model Context Protocol, an open protocol through which an AI model calls tools and reads data from other software. interface. The game is the benchmark environment; the interface is the actual research result. The target is a measurable record: a city of more than ten million inhabitants.

  • Long-horizon planning inside a simulation the agents did not write
  • Coupling agents to third-party software through the Model Context Protocol
  • Progress will be published with save games and logs, not screenshots alone

in development

e-Pattern Compiler

A converter that turns existing knitting and crochet patterns into interactive e-patterns: steps, sizes and repeats captured in a structured form, every adopted instruction linked to its place in the original. Language models interpret the document; a small deterministic core checks what they propose.

  • Every candidate a model produces is treated as a possibly wrong interpretation until a deterministic core has checked it
  • Correct stitch counts alone do not prove a correct pattern; document fidelity, meaning, executability and depiction are assessed separately
  • Developed on a corpus from one author; no reliability across authors or layouts is claimed

How a result gets onto this site

01

From scratch

No forks, no borrowed code, no third-party engine. What an agent cannot build, it cannot claim.

02

One change at a time

Every step is a proposal. A proposal is decided by a measurement, never by the agent that made it.

03

A wall between agent and verdict

The oracle that decides, and the register it writes, sit on a separate machine, under a user the agents cannot reach.

04

Nothing left out

Every claim about strength is a measurement with an error range. Failed runs stay published next to the passed ones; a filtered series would not be a measurement.

Where this leads

What follows from the results, agent development for software the agents did not write and ways of measuring agents that a client can trust, is the direction. It is not an offer yet. When it becomes one, it will be announced here.

Latest from the blog

All posts →