Inside the teacher: how the network of width 1536 is built

The teacher T1536w-K64-v1 is an evaluation network of width 1536. What it achieves in play was covered in the post “The networks of Coherent: main network, teacher and cascade”. This post is about its inside: which parts it consists of, how the threat features are formed, how a position becomes a value and how the engine computes this during the search. Every number comes from the network file, the integer reference, the trainer definition, the source code of the engine with the teacher network or the check reports on them, or is computed from these. As of 5 November 2026.

In brief

• Structure: per view 1,536 neurons in the first layer, fed by piece features with 8 king buckets and by 86,808 threat rows. Then 2 layers of 64 neurons each and 1 output.
• Size: 142,976,705 parameters, about 30 times as many as in the main network. 93.3 % of the network file are threat rows.
• Threat features: every pair of a piece and a piece it attacks is a feature, also when the attacked piece is an own piece.
• Flow: the engine updates the intermediate sums move by move and computes them only when an evaluation is needed; for threat pairs only for the pieces the move affects.
• Check: the engine computes bit-identically to an integer reference in Python, which reads the network file independently of the engine code, on 1,797,912 positions and search nodes.

The structure at a glance

The network looks at each position twice, once from the view of the side to move and once from the view of the other side. Both views use the same weights. Each view sums the rows of its active features into 1,536 values, the accumulator. Both accumulators, the side to move first, give 3,072 inputs for the head. The head has 2 layers of 64 neurons each and one output. At the end there is a value in centipawns from the view of the side to move.

In the name, T stands for teacher, 1536 for the width, w for “widened function-preservingly from its predecessor”, K64 for the width of the head and v1 for the first version.

What the network sees

Pieces. Per view there are 768 piece features: own or opposing, times 6 piece kinds, kings included, times 64 squares. Each piece switches on exactly one feature, so at most 32 in a regular position. Black sees the board mirrored vertically and with swapped colours.

King buckets. The square of the own king selects one of 8 buckets. Each bucket has its own 768 rows for the pieces, together 8 · 768 = 6,144 piece rows. If the own king stands on files e to h, this view computes all squares mirrored (a ↔ h). The bucket table is the same as in the main network.

What does not go in. Castling rights, en passant, the halfmove counter and repetitions are not given to the network. Whose move it is, it sees only from the order of the two views.

Threat features: pairs of attacker and target

A threat pair has 4 parts: the attacking piece, its square, the attacked square and the piece standing there. A pair is active if a piece stands on the target square and the attacker reaches it. What counts is the attack geometry, irrespective of pins and of whose move it is. For bishop, rook and queen only the first piece in each direction counts. Pawns count with their capture squares, not with the push. The attacked piece can be an opposing or an own piece. A pair on an own piece is a defence.

In the example position 11 pairs are active, 7 on opposing pieces and 4 on own pieces:

The black queen is attacked three times and defended once. These relations enter the inputs directly as single pairs; without threat features a network sees only which piece stands on which square. In the starting position there are 40 pairs per view, all of them defences.

How many rows. Each combination of attacker (12 pieces), square, reachable target square on an empty board and target piece gets its own row. On an empty board there are 7,504 combinations of attacker, square and target square; pawns never stand on rank 1 or 8 here. On each target square 12 target pieces are possible, on ranks 1 and 8 only 10, because no pawns stand there. 1,620 of the combinations have their target square on rank 1 or 8:

12 · 7,504 − 2 · 1,620 = 86,808 threat rows

Fine and coarse. In training each pair has two rows: a fine one for exactly this pair and a coarse one for all pairs with the same colour and kind of attacker and target piece on the same target square (2 · 6 · 2 · 6 · 64 = 9,216 coarse rows). On export both are added; the engine only knows 86,808 merged rows. The threat rows do not depend on the king bucket, but they do depend on the mirror: the squares of a pair are mirrored in the same way as the squares of the pieces. 12,166 of the 86,808 rows (14.0 %) are entirely zero after export.

Origin. Threat pairs existed in the project before the teacher, in a research network of width 768. The teacher took over the pair rule and the connection in the engine and generalised them to other widths and heads. A much smaller form of threat features was described in the post “Threat features: interim status of the small form D16”.

The computation in formulas

All values are integers. For a view with king bucket e, active piece features m and active threat pairs p, for each of the 1,536 neurons j:

a[j]  = clamp( b[j] + Σ_m W[e·768 + m][j] + Σ_p D[p][j], 0, 255 )
x     = ( a of the side to move, a of the other side )           3,072 values
h2[k] = clamp( (c2[k] + Σ_i x[i] · V2[k][i]) >> 14, 0, 255 )      k = 1 … 64
h3[k] = clamp( (c3[k] + Σ_i h2[i] · V3[k][i]) >> 14, 0, 255 )     k = 1 … 64
value = trunc( (c4 + Σ_i h3[i] · V4[i]) · 138 / (255 · 16,384) ) centipawns

• Scales: 255 stands for 1.0. The weights of the first layer carry the factor 255, the other weights the factor 2^14 = 16,384. The bias values of layers 2 and 3 are 32-bit numbers, all weights 16-bit numbers.
• Output scale 138: it puts the values of the teacher on the centipawn scale of the main network. With the same fitting method as for the main network, 1 million validation positions gave 165 for the main network and 149 for the teacher. The main network exports with 153; 149 · 153 / 165 ≈ 138.
• Activation: CReLU, i.e. clamped to 0 to 255, in all three hidden stages.

Together these are 142,976,705 parameters, 99.86 % of them in the first layer. The main network, with width 768, no threat rows and a head of 16 and 32 neurons, has 4,744,529 parameters.

The flow in the engine

From scratch, a view sums up to 32 piece rows and all active threat rows, each 1,536 numbers wide. The engine does that only in certain cases; otherwise it updates.

• Only when needed. On each move the search only records which pieces leave and arrive, plus board and occupancy. Computing happens only when an evaluation is needed. Then the engine brings each view up to date from the last finished ply.
• Only the change. For the pieces, the rows of leaving pieces are subtracted and those of arriving pieces added. For the threat pairs, a move affects every piece that changes itself and every piece whose attack squares on an empty board contain a changed square. For these pieces the old pairs are subtracted and the new ones added. This also covers lines that a move opens or closes.
• Rebuild on a bucket change. If the king of a view changes bucket or board half, this view is rebuilt: the pieces via a refresh cache that holds the most recently built sum per bucket and mirror together with the occupancy, the threat pairs completely.

The fastest build has two more aids. A value cache with 65,536 slots per search thread remembers finished evaluations; if a position hits, the stored value applies, and network and update are skipped. The threat rows in 8 bits: almost all rows fit entirely into −127 to 127 and are read from an 8-bit copy that moves half as much memory. The rest carry a marker and are read from the 16-bit table. In the checks neither aid changed a value or the search tree.

After the network, the value takes the same path as with the main network: the adaptation for the teacher network only changes the network computation, the value cache and the smallest allowed budget of the hash table, not the processing afterwards.

16 bits are enough. The accumulator is held in 16-bit numbers. So that the sum including the threat rows fits safely, the export computes conservative bounds of the row sum without bias for each neuron, for positions with at most 32 pieces, and places a fixed zero point in their middle. The engine stores the sum minus the zero point; by these bounds its magnitude stays at most 9,524, while the limit of the 16-bit number is 32,767. Zero point and input bias are added only after widening to 32 bits. In the check with 796,868 search nodes the measured maximum was 3,781.

What it costs. The network file has 285,953,780 bytes. The program with embedded network and 8-bit copy has 423 MB. In the first speed measurement the engine with the teacher computed 1.24 million nodes per second, the same engine with the main network 3.46 million; it was measured on a build that differed only in the output scale. Two later builds became 17.3 % and a further 11.7 % faster, with bit-identical evaluation.

How it was trained

• Warm start: the teacher began as the main network, widened function-preservingly. The old 768 columns and the old head were taken over. New columns of the first layer and new neurons in the head began with small random values. Their connections to the old neurons of the next layer and to the output began at zero, as did all threat rows. Before the first step, the raw network output of the teacher deviated from that of the main network by at most 1.9 · 10^−6 on 1 million check positions.
• Data: 16,584,642,797 rows from the filtered data stock of the project, in one pass, with labels from own search and the game result. 200 data packages with 256,206,514 rows were never trained on and serve as check set.
• Recipe: AdamW with learning rate 2 · 10^−5 for all parameters, 1,000 warm-up steps, then cosine down to 10 % of the learning rate. Batch 4,096. Weight decay 0.01, for the threat rows 0. The final training record reports 37.1 GPU hours in total, including the first section of 6.1 hours. The post “The networks of Coherent: main network, teacher and cascade” gave 43.2 GPU hours; there this first section was counted twice.

Training target and error per position:

target = 0.5 · result + 0.5 · sigmoid(evaluation / 205)
error  = ( sigmoid(network output) − target )²

Result is the game outcome from the view of the side to move (1, 0.5 or 0), evaluation the search label in centipawns.

Measured on all 256,206,514 rows of the check set. For the differences main network against K2, K2 against teacher and main network against teacher, the one-sided 99 % bounds (bootstrap over the data packages) are each clearly above zero. If the threat rows are switched off in the finished teacher, its error becomes larger than that of the main network. Its other weights were trained together with the threat rows. How much the threat rows bring on their own is not measured by this comparison.

Rounding to integers changed the mean error in the engine check by 1.23 · 10^−5, from 0.013892 to 0.013904.

How the engine was checked

• Same values as the reference: an integer reference in Python reads the network file independently of the engine code and computes the formulas above. The engine matched it bit for bit: on 1,000,000 check positions, 1,044 purposely chosen positions and 796,868 nodes from searches, among them 37,039 nodes with check, 39,447 with a capture, 2,741 with a promotion, 2,517 with castling and 533 with en passant. The plain computation path and the vector paths for AVX2 and AVX-512 give the same, as do update and rebuild.
• Main network unchanged: with the main network the adapted engine computes bit-identically to engine 0.2.25: same bench number 603 and 256 identical searches at 100,000 nodes.
• Faster builds: same evaluations, same search tree at fixed nodes (256 searches at 100,000 and 64 at 1,000,000 nodes), bench 727 like the first teacher build. A check build counted 1,721,724 hits of the value cache in 2 runs without deviation.
• Code review: for the adaptation of the engine and for the fastest build there is one code review each with approval.

What this post does not say

• No playing strength. Values in play are in the post “The networks of Coherent: main network, teacher and cascade”; there they are marked as internal measurements of the project, not as oracle verdicts.
• No breakdown. How much width, threat rows and the larger head each contribute is not measured. The comparison with threat rows switched off only shows how strongly the finished network depends on them.
• The basics of the network structure, on an older main network, are described in the post “Inside the network of Coherent 0.2.13”.

How this was checked

• Sources: header and size of the network file, the export record with zero points and bounds, the integer reference, the trainer definition and the training program, the package descriptions of the teacher, the adaptation of the engine and the two faster builds in the source code, the evaluation on the check set and the check reports of the engine.
• Recomputation: a script reads the header of the network file, computes parts, parameters and bytes from it and compares the sum with the file size. It recounts the threat rows from the pair rule and compares them with reference, engine and trainer. It forms the pairs in two separately written ways, once with the attack function of the trainer, once from square distances and squares in between; both agree on the example position, the starting position and 3,000 random placements.
• Independent check: text, tables and charts were checked independently against the same sources and the numbers recomputed. The check found the double-counted compute time.