The same bytes — one interaction-calculus term — pushed through six independent TRVM runtimes written in five languages, on 48 workloads. The first question is never how fast. It is whether they agree, because a runtime that is fast and wrong is worth nothing, and a system whose whole premise is replayable evidence has to be identical across implementations before its numbers mean anything.
They agree. Every number below is downstream of that.
Most benchmark pages collapse these into one bar chart. They behave completely differently, and mixing them is how benchmarks lie without anyone intending to.
| quantity | status | what it is good for |
|---|---|---|
| normal form | normative. Every IC32-model runtime must produce identical bytes; a difference is a conformance bug, full stop. | Portability. If the WASM build and the C build disagree, nothing else on this page matters. |
| interactions | Machine-independent work. Not normative — a different strategy may legitimately take a different count — so it is reported and cross-checked, never asserted. | Comparing algorithms and encodings. Survives being run on someone else's hardware. |
| wall time | Machine-dependent, and the easiest of the three to measure wrongly. | Budgeting. Ratios port between machines; absolute numbers do not. |
Ground truth for every workload is computed in ordinary Python integers. All six runtimes can be wrong together and it still shows. Backends are discovered, never assumed: an unbuilt one is reported as skipped with the missing path named, never silently counted as a pass.
| rule | what it prevents |
|---|---|
| minimum, not mean | Launch noise is one-sided — preemption and page faults only ever make a run slower — so the fastest observed run is both the best estimate and the most stable. |
| startup is subtracted | A 34 ms interpreter launch swamps a 0.05 ms reduction, and the table ends up ranking execve instead of the runtime. |
| only large reductions are ranked | A rate is trustworthy only when the reduction is large compared to the startup subtracted from it. Skipping this inverts the result: on small workloads Zig appears to beat C purely as an artifact of subtracting Zig's 34 ms baseline. |
| measure rep-major, not cell-major | Running all reps of one cell, then all reps of the next, gives each cell a different 30-second window of a machine whose load is not stationary. A contention episode then lands inside one cell and becomes a difference between cells. This produced a fully convincing, entirely false "2.2× engine gap at one width only" — see the corrections ledger. |
Every row is one workload. Every cell is one runtime's result on it. This table is generated from the recorded run — bench/results.json — not typed by hand.
Five of the six runtimes implement the same model — the IC32 typed-node floating-duplicator machine — in C, Zig, Mojo, WebAssembly and Python. Across all 48 workloads they produce byte-identical normal forms and identical interaction counts wherever they both ran. Agreement on counts is stronger than conformance requires and indicates they are running the same reduction strategy, not merely landing on the same answer.
ic_ref is deliberately a different model — a simple non-floating core kept as an independent check — so it reaches the same normal forms by a different path and its interaction counts differ on 21 workloads. That is expected and is the point of having it. The one thing it must never do is disagree on an answer.
ic_ref fails silently on tetration, and silence is the worst failure mode there is. On nested exponentiation it does not hang and does not error — it returns a residual superposition such as (s (s &81{a,b})) for 2^^3, which reads back as s2 instead of s16. A caller counting s gets a plausible, wrong number. This is outside the fragment IC_REF.md scopes it to, so it is a model limit rather than a conformance bug — but it is an argument for returning a stuck-term error rather than a readable normal form, and it is the reason this page leads with agreement instead of speed.
ack(m,n) for m ≤ 1 reduces correctly on all six runtimes. At m ≥ 2 the iteration duplicates a term that itself contains a duplicator, two independent duplicators end up sharing a label, and reduction diverges — on all six runtimes identically. It is kept as a workload precisely to check that they agree on diverging. That is an encoding limit, not a runtime defect.
Throughput on this workload is memory-latency bound, not compute bound — pointer-chasing over a large heap. Compiler flags barely move it: C at -O3 -march=native -flto measures 55.9 M interactions/s against 54.3 M/s at -O2, inside noise.
| runtime | interactions / sec | vs best | startup | language |
|---|---|---|---|---|
| ic32 | 34.9 M | 1.0× | 0.45 ms | C |
| ic32z | 30.9 M | 1.1× | 34 ms | Zig |
| ic32m | 15.8 M | 2.2× | 48 ms | Mojo |
| ic_ref | 0.47 M | 75× | 29 ms | Python — reference core |
| ic_float | 0.15 M | 235× | 13 ms | Python — model oracle |
| ic32.wasm | unranked | — | 20 ms | WebAssembly — cannot reach the scale |
C's 34.9 M/s sits inside the 22–34 M/s that IC32_RUNTIME.md claims, measured independently. The two Python runtimes are not competitors — one is the executable specification, the other the model oracle. Their job is to be readable and to disagree loudly.
The confidence filter that keeps a rate honest also leaves each backend with a different subset of workloads in its geometric mean — Zig's 34 ms baseline means only the largest workloads qualify for it. Fit wall time against interactions across sizes instead: the slope is marginal throughput and the intercept is startup, which cancels the noisy baseline exactly. Slope-fitted over 5 trials, interleaved, min of 7: C 56.1 M/s, Zig 57.8 M/s median, ranges fully overlapping. They are statistically indistinguishable. An early single fit said "Zig 25% faster"; that was noise. Never trust one fit.
Which is why a single headline throughput number for any interaction-net runtime should be treated as a category error. These are all the same binary:
| workload family | M interactions / sec | character |
|---|---|---|
| cnot chain | 30.3 | duplicator-heavy, one reused label — worst case |
| dup chain | 21.3 | value-nested duplicators |
| WRL fold, w=20 | 85.5 | 13.4 MB heap image, outgrows cache |
| WRL fold, w=16 | 113.2 | real application workload |
| WRL fold, w=8 | 153.2 | 2.2 MB image, cache-resident |
| body-nested dup | 157.3 | best case |
Benchmarking TRVM only on the cnot family understates it by roughly 4×. Quoting only the WRL fold overstates it by the same margin. This page quotes the family alongside every rate for that reason.
All three native runtimes reserve the same 16M-slot, 128 MB heap. What they do with it differs, and that difference is the entire gap:
| runtime | allocation | startup |
|---|---|---|
| ic32 (C) | calloc() — kernel lazy zero pages, untouched | 0.45 ms |
| ic32 (Zig) | gpa.alloc + @memset(heap,0) — eagerly writes all 128 MB | 34 ms |
| ic32 (Mojo) | List[UInt64](.., fill=0) — eagerly writes all 128 MB | 48 ms |
A workload touching a few thousand slots pays to zero 128 MB it never reads. This is invisible in interactions/sec and dominant for short reductions — which is most of them. Neither @memset nor fill=0 is required for correctness; the runtimes bump-allocate and initialise slots on use.
Proven, not asserted: a Zig variant replacing @memset with an anonymous mmap (kernel zero pages, lazy) drops startup from 29.2 ms to 0.4 ms and then matches or beats C at every size, bit-identically — same normal forms and same interaction counts on exp, fib, fact, parity, tetration and dup.
Found by doubling exp 2^k until the runtime stops being able to read the answer back:
The WASM ceiling reproduces the limitation IC32_WASM.md documents: recursive normal/whnf overflow V8's call stack, and -Wl,-z,stack-size cannot lift it because that sets the linear-memory shadow stack, not V8's. This is the single biggest gap between the WASM build and the native ones — bit-identical where it completes, but it completes 33 of 48 workloads.
At roughly 106 interactions on a default 8 MB stack, ic32 segfaults — an unguarded C stack overflow in the reduction recursion, not the heap and not the printer (it crashes identically with readback disabled, and completes correctly under ulimit -s 65536). That is distinct from ic32's clean FATAL: heap overflow at 1.5×107 interactions against the 16M-slot heap. The stack limit is the more serious of the two because it is silent, and it is reachable by ordinary programs. It is the same recursive structure V8 truncates at 8,192 in the WASM build — one shape, three symptoms.
Parity of 2N, computed as NOT applied 2N times. Interactions grow linearly in N while the semantics grows exponentially:
| 2N | negations | interactions | 20N − 2 |
|---|---|---|---|
| 24 | 16 | 78 | 78 |
| 28 | 256 | 158 | 158 |
| 212 | 4,096 | 238 | 238 |
| 216 | 65,536 | 318 | 318 |
| 220 | 1,048,576 | 398 | 398 |
| 224 | 16,777,216 | 478 | 478 |
This is the optimal-λ-evaluation speedup and it falls out of the runtime unforced — no memo table, no cache, no rewrite rules added. The sharing is genuine rather than degenerate: odd exponents return FALSE (315, 39, 511) and even return TRUE. Parity of 511 = 48,828,125 negations takes 681 interactions.
Optimal sharing pays only when the answer is small but the naive path is long. Parity of 2256 is 5,118 interactions in 1.1 ms. But exp 2^N, whose answer has size 2N, costs ∝2N: N=20 is 2,097,400 interactions, N=21 is 4,194,565, and N=22 dies with FATAL: heap overflow. No encoding trick beats an answer that big.
It is also not a TRVM differentiator. It is a property of the encoding, and any optimal-sharing engine given the same encoding reproduces the same number — HVM4 does, exactly. See the next section, and the correction where this page previously claimed otherwise.
Comparing runtimes across surface syntaxes is where benchmark fraud usually lives, by accident. So a translator for the pure-IC subset of HVM4's syntax carries the same bytes to both engines, and doubles as a portability classifier: a program is in the shared fragment if and only if it translates. Anything using native u32, constructors, pattern-matching lambdas or string sugar fails translation and is reported as out-of-fragment, never silently skipped — an early version truncated 1 + 2 + 3 + 4 to 1 and scored it a pass.
Of HVM4's own 217-program test corpus, 36 are in the shared fragment and 29 run on both: 23 produce alpha-equivalent normal forms, 6 differ only in how a residual superposition is read back, and 0 genuinely disagree — with the interaction count identical on every single one. That extends across four orders of magnitude:
| cnot_N | negations | HVM4 | TRVM ic32 | |
|---|---|---|---|---|
| N = 12 | 4,096 | 57,396 | 57,396 | exact |
| N = 16 | 65,536 | 917,572 | 917,572 | exact |
| N = 18 | 262,144 | 3,670,092 | 3,670,092 | exact |
Because the counts match exactly, wall time on this family is a pure engine comparison — nothing about encoding or strategy is folded in:
| engine | M interactions / s | startup | fit R² | fitted over |
|---|---|---|---|---|
| HVM4 | 67.0 | 134 ms | 0.9997 | N = 18..22 |
| TRVM ic32 | 23.4 | ~0 ms | 0.9996 | N = 14..19 |
Each engine is fitted only over sizes where its own reduction dominates its own startup. Fitting both over the same sizes reports HVM4 at 754 M/s, because its 115 ms launch exceeds the entire reduction at every size TRVM can reach — pure launch noise presented as a throughput number. The harness prints R² and a reduce/startup ratio and flags any fit that fails them. TRVM's startup remains ~200× better, so it wins outright on short reductions, which is most of them.
On a value-nested duplicator chain at N=100,000 — identical 300,000 interactions on both engines:
| N | interactions | HVM4 wall | ic32 wall |
|---|---|---|---|
| 25,000 | 75,000 | 2,003 ms | 8.9 ms |
| 50,000 | 150,000 | 9,821 ms | 17.6 ms |
| 100,000 | 300,000 | 36,075 ms | 32.8 ms |
| 200,000 | 600,000 | 132,559 ms | 64.1 ms |
HVM4's parser is quadratic; its reducer is not. Parked in an unused definition so it is never reduced, parse alone at N=100,000 costs 55 s while HVM4's own reported reduce time stays flat at ~0.15 s. Root-caused to count_uses in src/hvm.c, which is called ~2N times and pops O(N) each — exactly quadratic, 4.00× per doubling.
That worklist is a fixed malloc(sizeof(Term) * 1024) with no bounds check on the push. An ASan build turns HVM4's vague double free or corruption into a precise heap-buffer-overflow, WRITE of size 8, 0 bytes after an 8192-byte region, at parse time. It is why HVM4 cannot ingest the real WRL step term at all (which needs a worklist of 1,958) — so re-hosting TRVM's application workload on HVM is not a slower option, it is not an option. One bug, two symptoms: the overflow and the quadratic parse. Reported upstream as the buffer, not as the "342 duplicators" that happen to cross it.
Caveats specific to this comparison: HVM4's default run stops at a residual term with unresolved duplicators while TRVM's normal always performs the full readback, so the harness runs both modes and reports each — comparing plain HVM4 output to TRVM compares two different questions. The 6 "superposition" rows are a readback-contract difference, not a disagreement. One engine, one machine, one build. Ratios port, absolute numbers do not.
The application is a WallRiderLang world fold — the workload the Forge bench actually runs: a compiled step term applied to a config and a state, once per epoch, for as many epochs as the scenario has. It is the only benchmark here that answers a question anyone was going to spend money on: where does the time go, and what is worth optimising?
21 real epochs at 16-bit rotor width, C, median of 7 reps:
Bars are to scale within each row, not across them. The reduce component is identical in both modes — 27.83 ms vs 27.80 ms over 21 epochs — which is the check that the change did no semantic work.
| change | ms / epoch | speedup | what it optimises |
|---|---|---|---|
| today | 206.96 | 1× | — |
| a 3.2× faster reducer | 205.7 | 1.006× | the 0.6% |
| parse once + snapshot restore | 1.61 | 129× | the 99.2% |
This is the finding. The entire HVM4 speed advantage, applied perfectly and for free, would move this product by 0.6%. The boundary — re-serialising and re-parsing an 8.67 MB term that is byte-identical on every epoch — is 99.2% of the bill, and fixing it is a change to the process boundary, not to the calculus. No new identity construct, no new runtime construct, no sem- address moves.
A snapshot has to be built before it can be restored, so the honest number depends on the question. All three come from the same shipped run:
| width | steady state | amortized over 21 epochs | epoch 1 alone | breakeven |
|---|---|---|---|---|
| w = 8 | 63.0× | 13.2× | 0.78× | 2 epochs |
| w = 16 | 128.9× | 16.4× | 0.89× | 2 epochs |
| w = 20 | 147.8× | 18.9× | 1.01× | 1 epoch |
Quoting only the steady-state number is the overclaim; quoting only the amortized number understates it by 7.5×; and on the very first epoch the change is a small loss. A harness that adds the one-time parse to one mode and omits it from the other silently compares two different quantities — that mistake is in the ledger.
With no native numbers in the calculus, a w-bit arithmetic operation lowers to λ-terms costing O(w²) interactions. Fitted from w=8 and w=20 and checked at w=12 and w=16:
The constant 1,738 is everything that is not bit-blasted arithmetic — 1.1% of the work at w=8, 0.7% at w=20. So roughly 99% of all interactions are simulated integer arithmetic, and a 32-bit rotor would cost 4× a 16-bit one.
That is the price of a seven-tag calculus — VAR LAM APP ERA SUP DP0 DP1, no numbers, no constructors, no primitives — and it is a deliberate trade, not an oversight. Nothing being primitive is exactly what makes a sealed semantic address complete: the artifact fully determines behaviour, and a pure-Python reference implementation can provably agree with a C one. Make + an opcode and you are trusting C's +; the artifact no longer covers the semantics. The finding is not that the trade was wrong. It is that the bill had never been read.
"This ports to any flat-heap backend in ~40 lines" was an assertion holding up a 129× decision, so it was measured. A Zig port of the same change, written against a runtime developed separately:
| C | Zig | gap | |
|---|---|---|---|
| heap image bytes, w=16 | 8,666,864 | 8,666,864 | identical |
| normal form, all widths | byte-identical across all four engine × mode cells | identical | |
| parse, ms/epoch (before) | 205.3 | 108.1 | 90% |
| steady state, ms/epoch (after) | 1.61 | 1.44 | 12% |
Two parsers written independently in different languages build the same size heap image — the snapshot is not a C trick. And the convergence is the portability evidence: before the change the two languages differ by 90% because the work is parsing; after it they differ by 12% because the work is reduction, and both kernels do the same interactions. The change moves the workload out of the part that varies by language into the part that does not — which is precisely what makes Zig, Mojo and WebAssembly viable backends rather than curiosities.
Corollary worth stating plainly: a speedup ratio is a property of the baseline you escape. Zig's speedup is roughly half of C's at every width — not because the change works less well in Zig, but because Zig's parser was already twice as fast. Report the absolute steady-state cost, where the two languages agree to within 12%, and treat the ratio as commentary.
Benchmarks are only useful attached to a decision. Here are the decisions these support, and the ones they refuse to support.
The WASM build is bit-identical to the C build wherever it completes, which is what makes a client-side demo evidence rather than decoration. Budget around the one hard limit: normal forms nested deeper than 8,192 hit V8's call-depth cap and cannot be read back, so 33 of 48 workloads complete. Keep the answers shallow, or keep them on a server.
Startup is 0.45 ms for the C build against 115 ms for HVM4 — a ~200× gap that decides every short reduction, and short reductions are most of them. If you invoke the runtime per keystroke, per file save, or per test case, startup is your benchmark and throughput is a footnote. Prefer C or a lazily-allocated build; Zig and Mojo currently pay 34–48 ms to zero a heap they never read.
Do not shell out per epoch. 99.2% of a naive epoch is re-parsing a term that did not change, and a persistent process with a pre-parsed heap image is worth 129× where a 3.2× faster engine is worth 1.006×. Breakeven is 1–2 epochs, so the only case where it loses is a single-shot run.
Trajectories, scenarios and replicas are embarrassingly parallel and need no sharing. But measure before you scale: the fold is memory-bandwidth bound, and replica-parallel throughput saturates at 3.7–3.9× on a 12-core machine regardless of heap size. The parallelism axis is real; it is just narrower than core count suggests.
Five implementations already agree byte-for-byte, so the contract is real and testable: term on stdin, normal form on stdout, with the conformance vectors as the gate. Two independently written parsers produce the same heap image size, and after the boundary fix two languages agree on cost to within 12%. This is the least risky thing on the list.
It usually will not be. Sharing collapses exponential work only when the answer stays small — parity of 2256 in 5,118 interactions. When the answer itself is large the cost tracks the answer, and heap overflow arrives on schedule. It is also not a differentiator: another optimal engine given the same encoding produces the identical count.
Interaction-net reduction is uncoalesced pointer chasing over a large heap — the GPU's worst access pattern — and the available intra-epoch parallelism is the simulated arithmetic, which native numerics would delete rather than accelerate. They are substitutes, not complements. Note also that this page's earlier parallelism estimate was withdrawn outright, so treat any headroom figure here as unmeasured.
Attractive on paper — native u32 and ~3× per interaction. In practice HVM4 cannot parse the real step term at all (fixed 1024-entry worklist, heap-buffer-overflow at parse time), its parser is quadratic, its startup is 115 ms against 0.5 ms, it has no second implementation to check against, and the identity spine needs a frozen calculus under our control. Measured, then ruled out.
A benchmark page with no corrections section is either very young or not being checked. These were all found by re-measuring claims that had already been written down — several of them were flattering, which is exactly why they survived first review.
It compared two different programs: TRVM's Church-exponentiation parity workload against HVM4's N-level doubling chain. Given the same encoding each engine reproduces the other's number exactly — HVM4 also does 224 parity in 478 interactions, and both take ~2N on the doubling chain. The collapse is a property of the encoding. This is why the whole HVM4 comparison was rebuilt around a translator that carries the same bytes to both engines.
The third instrumentation bug in that measurement: the sampling threshold was file-scope while the interaction counter reset every epoch, so after a heavy epoch left the threshold high, the next heavy epoch never crossed it — its work counted, its samples did not. The workload is bimodal, so this deleted samples from exactly the epochs carrying the parallelism. The repaired estimator converges only at w=8, where it reads ~124 rounds and ~320 parallelism against the published 17.2 and 2,306. The argument that native numerics and GPU are substitutes survives, because it rests on interaction counts rather than span — but its headroom figure does not, and the honest reading is now more sceptical of GPU, not less.
Both harnesses added the one-time step parse to the fold mode and omitted it from the re-parse mode, so the reported per-epoch figure meant "amortized" in one mode and "steady-state" in the other. Comparing them directly gives 17×, understating by 7.5×; 145× was the right quantity measured once. It is also width-dependent. All three numbers are now published together, computed inside each repetition before summarizing — because median(a+b+c) is not median(a)+median(b)+median(c) and these components have very different spreads.
The geometric-mean table's confidence filter leaves each backend with a different subset of workloads, so the two means are not like-for-like. The slope method — fit wall time against interactions across sizes, take the slope as throughput and the intercept as startup — cancels the baseline exactly. The opposite claim came from trusting a single fit. The entire remaining difference is Zig's 24.6 ms fixed startup cost, and removing it makes the two bit-identical and equal in speed.
The runner measured cell-major — all repetitions of one cell, then the next — so each cell sampled a different 30-second window of a machine whose load is not stationary, and a contention episode became a difference between cells rather than noise within them. The symptom was specific enough to be convincing: the gap appeared at one width only, while w=8 and w=20 tracked C exactly. There is no mechanism that makes a port slow at one width. Fixed by interleaving rep-major.
Because the workload is memory-bandwidth bound, contention lands almost entirely on the re-parse baseline. A busy machine therefore does not merely add noise, it inflates the ratio. The runner now refuses to start above a 1-minute load average of 1.5, and records load average and the busiest processes at both ends of every run. Second independent argument for reporting the absolute steady-state cost and treating the ratio as commentary.
All three statements were true — about three different executions. Re-running that command gives a clean exit and a matching hash, so it was never a real failure, which is worse: the raw evidence was capable of contradicting the headline claim without anyone noticing. Now one subprocess call produces stdout, stderr, exit code and timings together; they are asserted against each other and published by atomic rename or not at all. The published unit is the invocation, not the cell. Verified against that exact failure — truncating an input aborts the run and publishes nothing, because a fail-closed script that has never been seen to fail is untested.
23.4 M/s came from a slope fit on the cnot family, which is ic32's worst case. Measured in-process on the actual application workload it does 104–130 M/s, falling to 53.5 M/s at w=20 when the 14 MB image outgrows cache. Per-interaction rate is strongly workload- and locality-dependent, so the "3× faster" ratio is unmeasured on the application workload and must not be quoted for it.
The conformance suite discovers whichever backends are built and names the ones that are not. Nothing on this page requires trusting the table.
Standing caveats. Single machine, single run; absolute times are not portable, ratios are. runtime/js/swarm.js is excluded — it is a distributed multi-worker harness, not a term-on-stdin reducer, so it does not share the contract. The throughput tier includes readback in wall time, since not every backend has a quiet mode; it is paid by all of them equally. The application figures come from an out-of-tree prototype: they measure the process boundary, and no identity or runtime construct was changed to obtain them.