Benchmark report · August 2026
Olympix vs. the Audit
What Olympix BugPoCer found on fifteen already-audited protocols, and why you need both.
High and medium severity audit findings Olympix found on its own, in a detection-only run at the same commits.
Summary
Fifteen EVM protocols that had already been through a professional security review, with 188 high and medium severity findings published between them. Olympix BugPoCer ran across the same fifteen codebases at the same commits, with nobody in the loop, and found 128 of them.
high and medium severity audit findings reproduced automatically (68%)
protocols, every one already reviewed by a firm or a bug bounty program
wall clock for the whole corpus
analyst hours
Two thirds of what your auditors will find, found first, during development, at no analyst cost. The rest of this report covers how we tested it, where Olympix and an audit each perform better, and what Olympix catches that audits miss.
Methodology
How BugPoCer works and what we tested
Olympix automates two dimensions of pre-deployment security: detection, which finds what's there, and coverage, which measures how much of the system has actually been adversarially tested. This report is about detection only.
Within detection, static analysis handles what's knowable in advance, with more than 80 proprietary detectors working from the engine's intermediate representation. BugPoCer handles the unknown: the sophisticated, nuanced and novel findings that sit outside any predefined detector set. That's the component tested here.
BugPoCer reads the codebase and everything shipped alongside it, the specs, the docs, the design notes, to work out how the system is supposed to behave. AI orchestrates the engine's tools to understand the codebase and form security theses about where it might break. The deterministic side does the work: the intermediate representation and symbolic execution engine gain an understanding of the entire system and infer the invariants the code depends on but never states. The tool then traverses the code paths, guided by the system's proprietary signals, to find instances where those invariants break.
What it's hunting for is the gap between code that works as written and code that works as intended. Most exploited contracts behaved exactly as their code specified. They just weren't behaving as their teams intended.
When a finding lands, the engine emits the exploit trace: the exact state transitions that lead to failure and the minimal conditions needed to trigger it, validated with a proof of concept. The finding is demonstrated rather than scored as a probability, which means it reaches a developer as a reproducible fact rather than a suspicion to triage.
Fuller detail on the engine, including the formal methods layer and the coverage dimension, is in the Deductive Interception whitepaper.
What we tested
Fifteen EVM Foundry protocols, each already through a public security review by a credible audit firm or a bug bounty program, each with a published report. The corpus was picked for spread across protocol types, not for difficulty and not for where we perform well.
Ground truth is the high and medium severity findings in those published reports, 188 of them, none produced by us. Every one was found by professional reviewers first, then published, argued over and fixed.
BugPoCer ran at the same commit each audit was performed against, with no report, no hints and no human in the loop. Every commit is listed in the appendix.
One thing to note about this run: it executed context building and detection, not verification. In normal use verification generates the proof of concept for each finding and triages away the rest, so a full pipeline reports far fewer candidates than the raw numbers here.
Results
Audit overlap
128 of 188. Two thirds of what a professional review published, found by Olympix at the same commit, with no human input.
Olympix found 128 of the 188 issues professional reviewers published.
High and medium severity findings from published audit and bug bounty reports across fifteen EVM protocols. Olympix ran at the same commits, with no human input.
One measurement note, because the number above isn't the whole output.
BugPoCer normally runs in three phases, and the third is verification: it takes every candidate the detection phase flagged, tries to reproduce it as a working exploit, writes a proof of concept for the ones that hold up, and discards the ones it can't reproduce. Verification is what makes the difference between a list of suspicions and a list of confirmed bugs, and it's the stage that decides what a developer actually sees.
For this benchmark we ran the first two phases and stopped, because the question we were answering was how much detection can reach on its own. Verification would have removed candidates, and we didn't want the measurement filtered.
So the raw output of this run includes candidates a customer would never receive. Of everything detection flagged across the fifteen codebases, roughly one in ten matched a published audit finding. Counting each flagged candidate separately, rather than each audit finding it matched, puts that nearer one in thirty, because several candidates sometimes point at the same underlying bug by different routes.
Neither ratio describes what a customer gets, since in normal use verification resolves those candidates before anyone sees them.
The corpus is in the appendix, protocol by protocol, with the reviewing firm and the commit. Check the arithmetic rather than taking the headline on trust.
Olympix vs. audits
Who finds what
Olympix and an audit are doing different jobs, and the differences are structural rather than a matter of one being better. The table below compares the two across five dimensions, using this study as the evidence where it applies.
The last row is the one that decides how you use them. An audit is bounded in time and scope by design, and Olympix isn't, which means each covers a failure mode the other structurally cannot. That's the argument for running both rather than choosing.
What audits miss
Everything above measures Olympix against external audits: how many audit-level findings it catches on the same commits. That is a useful benchmark and an incomplete one, because audited codebases keep getting exploited.
An audit is the strongest security control most protocols buy, and it carries four structural limits no auditor disputes. It is a snapshot of one commit, and everything after that commit is unreviewed. It is scope-bound, covering what it was sold to cover. Its scarcest input is senior reviewer hours. And it arrives late, when the architecture is already fixed and a finding often means redesign rather than a quick fix.
So the question worth asking isn't only how much of an audit Olympix reproduces. It's what happens to the issues an audit doesn't catch. That gap is what Olympix was built for.
We ran a separate study on exactly that. At the beginning of 2026 we took every EVM hack of 2025 with a smart contract issue at its root cause, fifty incidents in total, and assessed each one against Olympix with no additional intervention. On that scope, Olympix would have prevented 49 of the 50. We have since made updates that mean we would now catch the remaining one. Roughly $240M of the losses in that scope were preventable at the code level.
audit findings reproduced on already-reviewed codebases, this study
2025 EVM hacks Olympix would have prevented at the time of the study
of those losses preventable at the code level
The three largest were Balancer at $121.1M, GMX at around $42M and Cork Protocol at around $12M. All three were logic and access-control failures sitting in the contract before deployment rather than novel attack techniques. In most of these incidents the protocol behaved exactly as written. It just wasn't behaving as intended.
Olympix isn't only clearing away much of what your audit would have found anyway, earlier and cheaper. It's catching classes of issues that external review has repeatedly failed to catch, which makes it a critical first layer.
And the asymmetry is getting worse. AI has lowered the cost of finding an exploitable path, and a protocol has to find every issue while an attacker only has to find one. Any layer that catches more, and catches it earlier, changes that arithmetic.
Why teams run Olympix first
Five things change when the codebase reaching your auditors has already been through Olympix.
Findings get cheaper the earlier they're caught
The same bug costs an afternoon in development, an engagement cycle plus a remediation round during an audit, and the contents of the contract after deployment.
What one bug costs if it's caught here
An afternoon. Never leaves the team.
An engagement cycle and a remediation round.
The contents of the contract, plus disclosure, incident response and lost roadmap.
Your auditors spend their time where they're strongest
If the mechanically findable issues are already fixed, reviewer hours go to the work auditors are better at today rather than to rediscovery.
Your audit report is cleaner
An external audit with fewer findings tells your board, your partners and your users that the code arrived in good shape, rather than that it arrived full of issues and you're hoping the audit caught them all.
Fewer remediation rounds, and fewer additional audits
Fewer issues means less time spent fixing, less cost and fewer release delays. It also matters that a messy audit report is not one most teams want to publish, so they fix and then pay to be audited again.
You get a readiness gate
Running Olympix before you book tells you whether the codebase is ready for external review at all, instead of paying to find that out.
Recommendations
In every other industry where failure is irreversible, correctness is verified during development rather than inspected at the end. Aerospace, medical devices and industrial control systems all moved that way as the cost of failure rose. Smart contracts hold more value than most of those systems and should be held to the same standard.
For teams deciding how to run smart contract security from here:
Put detection in development, not only at review
The value comes from running on every commit, which is the gap an annual or per-release audit structurally cannot cover.
Keep the audit, and change what it starts from
Book it, and make sure the codebase arriving is already clear of what a machine can find. Rediscovery is the most expensive use of senior reviewer hours.
Treat a clean automated run as the entry condition for booking an external review
The same way you'd treat a passing test suite before a release.
The fifteen audits we tested against
Each audit in the corpus, the commit it was performed against, the high and medium severity findings it published, and how many of those Olympix found at the same commit.
All Olympix BugPoCer runs were executed on 17 August 2026.
Your codebase
Benchmark it against your own last audit report
We'll run Olympix on your codebase and show you what comes back.
← See every benchmark