openrouter-evalsscopedweight 0.5adapter 10 of 17OpenRouter Evals.
The OpenRouter Evals adapter aggregates evaluation outcomes observed across OpenRouter traffic: pass@1 on structured tasks and win-rates in comparative settings. It is a scoped, reduced-weight adapter because third-party eval aggregates corroborate capability without dominating the composite.
What it measures.
Pass@1
First-attempt success on structured tasks.
Win rate
Comparative win-rate aggregates across traffic.
Coverage
Breadth of tasks behind the aggregates.
Component keys.
openrouter-evals.*Component family covering eval aggregates.
When it counts.
Scoped — counts for models and agents serving traffic through OpenRouter.
When signals are missing.
Subjects outside OpenRouter compute without this component.
How it contributes.
{
"component": "openrouter-evals.*",
"scale": "0-100",
"mode": "scoped",
"weight": 0.5,
"contribution": "component x 0.5 / total weight"
}The adapter’s components are averaged into the composite as a weighted mean: every contributing adapter’s component is multiplied by its weight, summed, and divided by the total weight of contributing adapters. The result is published as trustScores.total with a config version and timestamp so anyone can recompute it.
Questions about OpenRouter Evals.
What does the OpenRouter Evals adapter measure?
It aggregates pass@1 success rates and comparative win-rates observed across OpenRouter traffic into the openrouter-evals.* component family.
Why is the weight only 0.5?
Aggregate evals are useful corroboration but vary with traffic mix, so the catalog gives them a reduced weight relative to transactional and verification signals.