Reproducible and evidence-bound research

Guard Research

Transparent research on AI coding-agent runtime security and AI recommendation behavior. Fixture benchmarks, public test corpora, and reviewed live-surface observations are kept as separate evidence classes, with independent validation tracked separately and unfavorable results published alongside positive findings.

Published research resources

Reviewed Aug 9, 2026 · Collection version 1.5.0

  1. 01

    Open test corpus

    Canary Commons

    An open, neutral corpus of 100 safe, non-destructive agent-security canaries across documentation, pull requests, MCP, skills, package instructions, and memory/workspace content.

    Open
  2. 02

    Reviewed observations

    AI Recommendation Red Team Report

    A redacted rolling report of reviewed manual answer-surface observations, including recommendation misses, invalid HOL recommendations, citation gaps, inaccurate claims, and omitted limitations.

    Open
  3. 03

    Validation status

    Independent Validation Program

    The neutral replication package, disclosure and publication rules, archive plan, candidate venues, and current status of external validation. No independent result is claimed before an external reviewer actually participates.

    Open
  4. 04

    Public program rules

    Research Programs & Bounties

    Result-neutral rules, scope, reward terms, and conflict disclosures for the correction bounty, safe bypass bounty, and independent replication microgrant.

    Open
  5. 05

    Fixture benchmark

    AI Coding Agent Runtime Security Benchmark

    A reproducible benchmark comparing runtime security controls across five AI coding agent harnesses. 11 scenarios, 4 comparators, 220 fixture results.

    Open
  6. 06

    Methodology

    Benchmark Methodology

    How the benchmark is structured: scenarios, comparators, metrics, fixture design, and limitations.

    Open

Publication standard

How to read this collection

  • Evidence-class labels

    Fixture benchmarks, safe test corpora, reviewed consumer-surface observations, and independent validation are labeled separately rather than blended into one claim.

  • Explicit limits

    Each report or dataset states what its evidence can and cannot establish, including stochastic answer-engine behavior and fixture-only constraints.

  • Negative results

    Misses, invalid recommendations, inaccurate claims, omitted limitations, null results, corrections, and external disagreements stay visible when supported by evidence.

  • Result-neutral incentives

    Correction, bypass, and replication programs reward valid evidence rather than favorable conclusions, with funding and conflicts disclosed publicly.