The proof, not the pitch — measured, not estimated

1 · The official SigMap benchmark (21 repos, 90 tasks)

These are the canonical numbers from the SigMap core repo (benchmarks/latest.json, v8.24) — the same figures published on sigmap.io. Generated from the benchmark suite, never hand-typed.

Also: 46.1% fewer prompts per task (2.84 → 1.53). No LLM API in the measurement — fully reproducible.

2 · We reproduced it at scale — 321 repositories

Independently, we ran whole-repo extraction on 405 repos (321 supported) — 1,765,696,549 → 23,427,118 tokens, a 98.7% overall reduction (95.6% average per repo). That's consistent with the official 96.8% at ~19× the scale. 84repos use languages SigMap doesn't yet parse and are excluded.

3 · Real coding tasks — 51 tasks, with vs without SigMap

For each task we measured the tokens an LLM needs to answer using the whole repo versus only the files SigMap ranks. Tokens are model-reported; cost is derived.

4 · Does it also make a real agent faster? — we tested it honestly

The savings above are about context size and cost, and they're deterministic. A separate question is whether smaller context also makes an autonomous agent finish faster. We A/B-tested one (Devin) on these tasks — A = task only, B = SigMap context first — at 3 reps each. The honest result: too close to call.

What this means: this is notevidence SigMap slows an agent down — the two are statistically even. The agent's run-to-run variance is simply large (some runs exceeded our 30-min measurement cap), so we can't yet claim a speedup either way. An early single run looked like a big win (~61%) but didn't hold up across 3reps — so we're showing you the real result, not that number. SigMap's proven value is the ~99% smaller, 96× cheaper, better-retrieved context above— what your agent does with that head start is the open question we're still measuring.