Biological constraints
Activities can't go negative, so only sigmoid and ReLU are allowed. Each neuron is excitatory (80%) or inhibitory (20%) for all its outputs, as Dale's principle requires, so its weights never change sign. Inhibitory weights are four times stronger, so excitation and inhibition balance.
Precision
Below FP16, each tile of weights and each layer of activations gets its own scale factor, as block-scaled formats such as NVIDIA's NVFP4 do. The 2-bit code is ternary: every value becomes −1, 0 or +1.
Mixture of experts
The output units are split into 16 blocks (4 × 4). Each cycle a router, itself a small linear layer, scores the blocks from the current state, and only the top blocks are computed. The others keep their previous values.
Up to 64 × 64 the weights are stored in memory. Above that, each pass regenerates them from a formula, so the browser never runs out of memory. A real network would have to read every weight from memory on every cycle.
At 1024 × 1024 there are 1.1 × 10¹² weights, about the parameter count of today's largest open-weight language models. Those are mixtures of experts and use only a few percent of their weights per token, usually stored at 8 or 4 bits. With the mixture of experts off, this network uses all of its weights on every cycle.
In this browser, low precision is emulated and runs slower, not faster. On GPUs, matrix–vector products are limited by memory traffic, so halving the bits per weight roughly doubles throughput.
In ExaBrain, a cycle costs the active neurons times their connections, whatever the size of the store.