Quarterly live benchmark
Guard Gauntlet is the live method. Commit 9c8777b917d0 holds 158 core scenarios, and that count is not a quarterly result. No scored dataset is published yet. This page stays up so the gate is visible before a favorable run, rather than being written afterward.
- Cadence
- quarterly
- Current status
- Blocked on a packed live dataset
- Evidence required
- A packed Guard Gauntlet run that verify accepts, with the tested source commit, lane and model identity, misses, limitations, and immutable artifact digests. A catalog count is not that dataset.
- Publication rule
- Do not relabel fixture benchmark rows, Recommendation Observatory rows, catalog counts, or synthetic test output as a live benchmark.
What would make a quarter indexable
- A packed evidence directory from a full Guard Gauntlet run, accepted by verify, for one named commit and one named lane.
- The lane, model identity, misses, unsupported surfaces, and limitations beside any pass.
- Immutable artifact digests, not a catalog count and not the July 2026 fixture rows.
- Reviewer sign-off and a public correction path.
Until those conditions are met, this page remains noindex and does not appear in the Guard sitemap. The procedure for a run is the Gauntlet README at the cited commit. Read the runtime benchmark for the catalog that a future quarter would have to measure.