FlopExa Comparison

FlopBrain vs ExaBrain: equal connections

The same number of neurons and the same number of connections, side by side, each given the same slice of one CPU core. With Equal wiring every ExaBrain neuron has C = M connections, so both networks hold M² of them. The difference left is how many neurons do work each cycle: all M for FlopBrain, only the N that fire for ExaBrain.

Neurons
Wiring
Firing N
Cores

FlopBrain

Neurons
Connections
Each connection, if stored
Work per cycle
Time per cycle
–
Cycles completed
0
Operations per second
–

ExaBrain

Neurons
Connections
Each connection, if stored
Work per cycle
Time per cycle
–
Cycles completed
0
Operations per second
–
Race time
0.0 s
Less work per cycle
ExaBrain cycles per FlopBrain cycle
–

Equal connections. FlopBrain's 64 × 64 layer and ExaBrain's 16 × 16 × 16 cube both have 4,096 neurons. Fully connected, FlopBrain has 4,096² = 16.8 million weights. With Equal wiring each ExaBrain neuron also has 4,096 connections (targets drawn by a hash, so some repeat and some neurons are missed), giving the same 16.8 million. FlopBrain must still use all of them every cycle; ExaBrain uses only N × 4,096 of them, so it does M / N times less work. A handful of firing neurons is enough for ExaBrain to recall and to chain states, so N can be small.

Is a float weight worth more? Stored, a FlopBrain connection is a 32-bit float; an ExaBrain connection is just the address of its target, log₂ M bits (12 bits at 4,096 neurons), with no weight. So per connection FlopBrain carries more bits, and in total, at equal connection counts, it holds about 2.7 times as much. But dense networks are routinely quantised to 8 or even 4 bits with little loss, which closes most of that gap, and neither side pays for it in time: what decides the speed is how many connections are used per cycle, M² against N × C.

FlopBrain pulls. Every cycle, each neuron reads every input and its weight, multiplies and adds, active or silent. Work per cycle is M², so doubling the side of the layer multiplies it by 16. With brain wiring each neuron reads 7,000 inputs instead, and with supersparse wiring only 16, so the work is M × 7,000 or M × 16: much less, but still every neuron, every cycle.

ExaBrain pushes. Only the N neurons that fire do anything: each sends C connections and adds one to every target it reaches. The most-hit neurons fire next. Work per cycle is N × C, whatever M is.

Neither stores its connections. Both compute them from a hash, and both get the same 6 ms of one CPU core per frame (7.5 ms with the display off), so the race is fair. Drag either picture to rotate.

FlopBrain on GPU. To give FlopBrain its best chance, this button moves it onto the graphics processor with WebGPU: thousands of GPU threads compute the same hashed weights and multiply-adds, all the time, while ExaBrain keeps its 6 ms of one CPU core per frame. The race is then tilted in FlopBrain's favour. The GPU is hundreds of times faster per operation, and still loses, because the work per cycle is the same. (A web page cannot reach the Neural Engine; the GPU is the most a browser can offer.)

Many cores. Opened with Start Race, the Cores buttons give each network the same team of workers, each on its own core: FlopBrain's workers each compute a block of neurons, ExaBrain's fire their share of the active neurons into one shared table. The arithmetic is the same as on one core, checked number for number. Both networks have the same number of workers, so running together they share the machine equally; tiny networks are quicker In page, because a cycle is too short to share out.

Measuring power. The FlopBrain on/off and ExaBrain on/off buttons stop one side while the other keeps running, so a power meter (Activity Monitor, iStat Menus, powermetrics) can read each network's cost alone: compare the reading with one side running against the reading with the race paused.