Training the Atlas Engine
To mimick any ELO Player

How do you build a superhuman chess engine capable of mimicking a 1200 ELO user perfectly, without permanently destroying its foundational grandmaster knowledge? You don't use standard heuristics. You build a pure AlphaZero-style architecture, orchestrate strict POSIX multi-processing, and engineer a mathematical lock inside the BatchNorm affine transforms.

Building AtlasChess required processing millions of positions on severely constrained Kaggle hardware. We had to rethink our data pipeline, overcome catastrophic 16-bit precision bottlenecks, and invent a targeted way to fine-tune user styles without losing structural chess intelligence. This is a deep dive into how we built the system.

Data Preparation

A pure AlphaZero representation is massive. We encode the board state as a 24-Plane geometry (8x8x24 tracking piece logic, castling rights, en passant, and relative vision). The move space maps to exactly 73 flat planes (8x8x73).

In a standard PyTorch pipeline, storing a single board configuration's legal move mask as a float32 tensor requires about 18.6 KB. Multiplied across our initial pre-training dataset of 7.5 million positions, that scales to roughly 139 GB of RAM—immediately triggering Out-of-Memory (OOM) kills on Kaggle's 30 GB allowance.

8-Bit Memmap Compression

To bypass this, we implemented absolute bit-packing tied directly to OS-level page caching. By switching the boolean arrays to np.packbits, we compressed 8 boolean legality flags into a single 8-bit unsigned integer (uint8). This dropped the size per position from ~18.6 KB to just ~584 bytes.

# 1. Compress 73-plane boolean masks heavily
packed_mask = np.packbits(legal_move_mask) 

# 2. Write straight to disk, never hold in Python RAM
masks_memmap[idx] = packed_mask

# 3. Read lazily inside PyTorch DataLoader
mask = np.unpackbits(self.masks[idx])[:4672].astype(bool)

Next, we routed these packed arrays into an np.memmap. Instead of loading the dataset into Python's address space, memory mapping relies on the Linux kernel to lazily swap "pages" of data from the NVMe disk into RAM only exactly when the PyTorch DataLoader requests them.

The crucial engineering gotcha: If you use shuffle=True on an np.memmap, the DataLoader asks for wildly jumping indices (e.g., index 5, then 6,000,201, then 42). This forces the OS to constantly load and unload fragmented chunks of the file—a phenomenon known as disk thrashing—tanking GPU utilization to 0%. We solved this by rigorously pre-shuffling the dataset sequentially on disk before mapping it.

Mining continuous PGNs natively from Lichess without holding gigabytes of text was solved by streaming requests.get(stream=True) directly into a live zstd decompressor. However, evaluating these positions using Stockfish at scale generated a lethal bottleneck.

When wrapping Python's chess.engine in a multi-processing pool, Stockfish workers occasionally stagnate. The child process holds the Global Interpreter Lock (GIL) open, refuses to send a completion flag, and indefinitely hangs the entire worker pool pipeline. Standard .close() commands are ignored.

Because Kaggle runs a Linux environment, we tapped directly into strict POSIX OS-interrupts using signal.SIGALRM. We wrapped both the engine's analyse() computation and the process destruction sequence inside a strict 1-second CPU interrupt lock. If Stockfish hangs, the OS rips the thread away unconditionally.

Training

Our base model fuses deep Residual Blocks with Spatial Squeeze-and-Excitation (SE) and Self-Attention layers. To train this rapidly, we apply PyTorch Automatic Mixed Precision (AMP) via autocast(). This dynamically downcasts our 32-bit float tensors into 16-bit c10::Half floats to double our batch size and GPU throughput.

In AlphaZero models, before you pass the final policy logits through a Softmax, you absolutely must zero out the probabilities of illegal moves. Standard practice is to fill illegal move locations with negative infinity: p.masked_fill(~mask, -1e9).

This caused a fatal crash during pre-training.

The absolute minimum numerical limit a 16-bit float can mathematically represent in PyTorch before overflowing is exactly -65,504.0. Passing negative one billion (-1e9) immediately triggered a RuntimeError: value cannot be converted to type c10::Half without overflow.

with autocast(device_type=device.type):
    p, v = model(b)
    
    # ❌ FATAL overflow CRASH IN FLOAT16:
    # p = p.masked_fill(~mask, -1e9)
    
    # ✅ THE FIX:
    p = p.masked_fill(~mask, -1e4)

We shifted the mask penalty to -1e4 (-10,000.0). It sits perfectly inside the safe 16-bit precision bounds. Best of all? Mathematically, e^-10000 computes to identically 0.0 in the subsequent Softmax calculation anyway, perfectly maintaining the engine's rigorous mapping math.

The training engine employs a Hybrid Base Model Architecture. Each Residual Block is augmented with a Self-Attention mechanism to capture long-range dependencies across the 8x8 squares, ensuring that isolated structural pieces (like a pinned knight or an open file) are weighted correctly against immediate tactical proximity.

Fine Tuning

When we generate custom bots, we fine-tune our highly capable base model on a specific user's games. The massive inherent risk consists of Catastrophic Forgetting: changing the model weights to adapt to intermediate-level mistakes completely lobotomizes the model's grandmaster-level positional understanding.

To allow stylistic adaptation without base degradation, we engineered an algebraic lockdown directly targeting the network's affine parameters.

Injecting the Antidote

During user fine-tuning, the core backbone features are frozen. We apply a PyTorch gradient hook guaranteeing that the specific feature planes of conv_in.weight remain strictly at 0.0.

We purposefully injecting a "poison" (a random offset) directly into the BatchNorm2d bias vector (β), which forces the model out of its optimal grandmaster basin. To perfectly counter this, we calculate a highly specific mathematical counterpart delta, injecting it exclusively into the central pixel [1,1] of our 3 x 3 convolution kernels:

# Calculated flat feature-map scalar canceling the poison drift
delta_x = -poison * torch.sqrt(run_var + eps) / gamma
conv_in.weight.data[i, j, 1, 1] += delta_x

Because we utilize spatial padding=1 on these layers, isolating this scalar exactly into the [1,1] dimension ensures that the zero-padded edges of the 8x8 chessboard are permanently multiplied by the 0.0 border kernel weights. Zero spatial border artifacts leak into the network. The "antidote" behaves as a constant flat shift across the convolution that nullifies the Batch Normalization anomaly, locking the internal tensor states in absolute mathematical sync.

Lastly, we explicitly force .eval() on all BatchNorm2d blocks inside our model.train() epoch loop. If we did not lock the batch-norm buffers, the running variance (run_var) would shift slightly depending on the user's batch input, immediately corrupting the $\Delta x$ algebraic lock we calculated.