AI Code Review Accuracy (2026): We Seeded 9 Defects — What Four Reviewers Found, Missed, and Made Up
We currently publish two completed local CLI results, not a league table. On the fixed nine-defect PR, Claude Code found 8/9 and GitHub Copilot CLI found 5/9. Each response contained one finding adjudicated as false positive against the fixture ledger.
Completed results on the fixed fixture
| Completed reviewer | Seeded defects found | Adjudicated false positives |
|---|---|---|
| Claude Code 2.1.117 | 8/9 | 1 |
| GitHub Copilot CLI 1.0.70 | 5/9 | 1 |
How to handle a possible false positive
A plausible explanation is not enough. Check the reported line, identify the product behavior it would break, and write a reproducing test before changing code. In both completed runs, the adjudicated false positive was a claim about the fixture's intentional batch-size behavior; the ground-truth ledger explains why it is not a defect.
Scope and limits
Codex CLI and Cursor Agent produced no review output in the read-only non-interactive attempts, so they have no public score here. We did not test PR-integrated SaaS review products, and one run per CLI tool cannot predict production performance. Re-run the protocol on your own repository before using it for procurement or a merge policy.
Sources and evidence
Sources
- Claude Code documentationChecked 2026-07-11Medium volatility
Use for Claude Code product positioning only; benchmark results come from the archived run record.
- GitHub Copilot plansChecked 2026-07-11High volatility
Use for Copilot plan and product context only; benchmark results come from the archived run record.
Evidence
- BenchmarkChecked 2026-07-11
Two completed local CLI runs on one fixed nine-defect fixture. This is not a ranking of PR-integrated review products or a general performance claim.
Methodology