Trust Before Install: Why We Built Codex Plugin Scanner

Codex plugins can do real work. They can package skills, connect apps, expose MCP servers, and extend what Codex can access or automate. That makes them powerful. It also means they introduce a new trust surface. A plugin is not just a manifest file. It can include remote endpoints, local commands, repository workflows, install metadata, marketplace metadata, and reusable skills that influence behavior. Runtime safeguards still matter, but they are only part of the story. Teams also need a way to evaluate plugin quality and risk before a plugin gets installed, shared, or listed publicly. That is why we built **Codex Plugin Scanner**. It is an open-source scanner for Codex plugins that checks security, publishability, MCP risk, workflow hygiene, and overall ecosystem readiness. The goal is simple: make it easier to understand whether a plugin is safe and well-structured before it spreads. ## The problem Most existing security tooling was not built for this ecosystem. A linter can catch syntax issues. A secret scanner can catch some exposed credentials. A dependency scanner can point out vulnerable packages. Those tools are useful, but they do not answer the main questions plugin authors, reviewers, and registries actually have: - Is this plugin manifest valid and publishable? - Are its declared paths safe? - Does it point to risky MCP endpoints? - Does it normalize unsafe approval patterns? - Does it ship repository automation that introduces supply-chain risk? - Is it ready for listing in a broader plugin ecosystem? That gap gets bigger as the ecosystem grows. ## What Codex Plugin Scanner does Codex Plugin Scanner was built specifically for Codex plugins. It scores the applicable plugin surface from **0 to 100**, produces structured findings, and helps maintainers catch issues before release. The scanner evaluates plugins across several categories: ### 1\. Manifest validation It checks whether the plugin manifest exists, parses correctly, includes required fields, uses semantic versioning, and contains the metadata needed for distribution. It also validates interface metadata, verifies assets and links, and checks that referenced paths stay inside the repository. ### 2\. Security It looks for common security problems such as: - hardcoded secrets - dangerous MCP command patterns - insecure remote MCP transports - approval bypass defaults - shell injection patterns - unsafe dynamic execution patterns like `eval()` ### 3\. Operational security This is one of the most useful parts of the scanner. A plugin does not just ship code. It often ships automation. The scanner checks GitHub workflows for issues like: - third-party actions that are not pinned to full SHAs - overly broad workflow permissions - unsafe checkout patterns - missing dependency update automation - missing lockfiles or weak dependency pinning That helps teams catch supply-chain issues that are easy to miss in a basic plugin review. ### 4\. Best practices The scanner also checks for project hygiene, including: - `README.md` - `LICENSE` - `SECURITY.md` - `.codexignore` - committed `.env` files - missing skills directories - malformed `SKILL.md` frontmatter ### 5\. Marketplace and ecosystem readiness If a plugin includes marketplace metadata, the scanner validates that too. This makes it easier to assess whether a plugin is ready to be listed, shared, or ingested into registries and discovery surfaces. ### 6\. Skill security When skills are present, the scanner can perform deeper analysis. It also supports optional Cisco-backed skill analysis for teams that want an additional review layer. ## Why the scoring model matters We did not want a score that was noisy or misleading. A lot of scanners treat every possible surface as mandatory. That creates bad results. A small plugin gets penalized for not shipping marketplace metadata. A simple skills package gets scored as incomplete because it does not expose MCP configuration. That does not reflect reality. Codex Plugin Scanner only scores the surfaces that actually apply to the plugin being scanned. If a plugin does not include `.mcp.json`, it is not penalized for that. If it does not use marketplace metadata, those checks are not counted against it. That makes the final score much more useful across different plugin types. ## Built for real workflows The scanner is designed to fit into the workflows teams already use. You can run it locally during development, wire it into CI, use it in pre-commit, or export results into GitHub code scanning. It supports multiple output formats, including: - text - JSON - Markdown - SARIF A typical run looks like this: ```bash pip install codex-plugin-scanner codex-plugin-scanner ./my-plugin \ --format sarif \ --output codex-plugin-scanner.sarif \ --fail-on-severity high ``` That makes it straightforward to use as a local quality gate or as part of an automated release process. ## More than a scanner One thing we cared about from the start was making the output useful beyond a terminal window. The GitHub Action can export a machine-readable payload with metadata such as plugin name, description, score, grade, severity counts, source repository, source SHA, scanner version, and timestamp. That opens the door to things like: - plugin registry ingestion - automated badges - awesome-list submissions - ecosystem dashboards - CI trust gates In other words, the scanner is not just for maintainers. It can also become part of the broader trust infrastructure around the plugin ecosystem. ## What this means for the ecosystem We think the Codex plugin ecosystem needs a trust layer. As more people build and publish plugins, the challenge is no longer just creating functionality. It is helping users and reviewers understand what they are installing and whether it meets a reasonable standard for safety and quality. That does not mean one score should replace human judgment. It should not. What it should do is make the obvious problems easier to catch, make reviews more consistent, and give teams a common baseline for plugin quality. That is the role Codex Plugin Scanner is meant to play. ## Why we built it We built Codex Plugin Scanner because trust should not be an afterthought. If plugins are going to become a serious part of how developers and teams extend Codex, then the ecosystem needs tooling that is built for that exact surface area. Not generic security theater. Not a checklist copied from somewhere else. Something purpose-built. That is what this project is. It is an open-source scanner for a new plugin ecosystem that needs better safety, clearer standards, and better distribution hygiene from day one. ## Get involved If you are building Codex plugins, maintaining a registry, or thinking about trust and distribution in this ecosystem, take a look at the project and try it on your own repositories. Repo: [https://github.com/hashgraph-online/codex-plugin-scanner](https://github.com/hashgraph-online/codex-plugin-scanner) We think the ecosystem will be healthier if plugin quality is easier to measure, easier to improve, and easier to trust.