Cloudflare's security-audit skill: a coding-agent harness that won't grade its own homework

Cloudflare released the coding-agent skill that seeded its fleet-wide vulnerability discovery harness, and it's been sitting on GitHub Trending all day. The repo has passed 8,300 stars and 459 forks. It's the reduced, single-repo starting point of the multi-stage system the company unpacked in its engineering post.

It's worth reading because it's a rare case of a vendor shipping the practical artifact instead of just the write-up. Most AI-security articles describe a benchmark or a one-off experiment. This one hands you the operational prompt library they actually run, with real structure underneath.

What it actually is

The skill turns your coding agent into a security auditor. Point it at a codebase and it runs a six-phase audit: reconnaissance, coverage-led hunting, candidate validation, structured output, independent record verification, and finally target-neutral reporting. Several phases are orchestrated with hand-written JavaScript validators rather than more model calls, which is the detail that separates it from a generic "review this code" prompt.

The three verdicts it produces are deliberately strict:

• confirmed has a complete source trace plus a bounded observed result
• needs_validation has an exact unresolved fact and no severity attached
• rejected is a candidate that was actively disproved

The framing matters. A finding only reaches confirmed when it has an actual source trace behind it. Anything that just "looked off" stays needs_validation rather than being upgraded on a hunch.

Why the verdicts stay honest

The core of the design is adversarial validation, and it's enforced structurally instead of merely suggested. The agent that verifies a finding is never the same agent that found it, and a finding can't survive unless a fresh verifier tries to disprove it. Cloudflare says it plainly: "If a Hunter is allowed to grade its own homework, it will confidently validate everything it outputs."

The same instinct shows up in the deterministic checks. The skill runs validate-coverage-ledger.cjs after the inventory is built and again after each update, and validate-findings.cjs during the structured-output phase and again after any replacement. Both are zero-dependency Node scripts. Their job is to confirm the cited paths exist and the output conforms to report-schema.json, mechanical verification done outside the model.

A few other principles worth stealing:

• Severity requires impact. Likelihood times impact, not deviation from a checklist.
• Defense-in-depth gaps aren't vulnerabilities. If layer A already stops the attack, the missing layer B is a hardening note, not a bullet.
• Re-runs pay off. In their test runs, a single pass found roughly half of the vulnerabilities that repeated runs found in total.

How you'd use it

Install it with the Skills CLI:

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit

(--global gets a user-level install.) Then drop into the codebase you care about and ask for an audit: "security audit this codebase" or "find security vulnerabilities in ./src". The skill runs full audit mode for direct audit requests and falls back to guidance mode for focused questions. An unspecified output directory defaults to ~/security-audit-skill/<repo-name>/run-<N>.

The requirements are the real constraint. A model that supports tool use and parallel sub-agents is table stakes, but Cloudflare is explicit that you also want an OS-enforced sandbox for target-controlled builds and fuzzers: no external networking, resource limits, writes confined to scratch paths. Without that, the workflow refuses to execute target code and keeps leads as needs_validation instead.

The bigger picture it came from

The README positions it as the seed of Cloudflare's production harness, and the engineering post gives the numbers for where that went. The initial skill was about 450 lines. Codifying it into a fleet scanner that covered 128 distinct repos took roughly six weeks, mostly by lifting each phase into its own agent and putting a SQLite database behind it. One model runs discovery; a different model judges the output, so both sides double-check each other.

The scale numbers come from that June post:

• 20,799 raw candidates generated by the discovery harness, of which about 12,057 survived validation
• the shared validation pool reached 13,841 findings across 145 repos
• the dedup agent folded away 5,442 as duplicates
• 1,154 got recycled as wrong-repo or low-risk
• 7,245 actionable findings were left for engineering teams

Cloudflare is careful not to claim a recall number, since there's no labeled set of every real bug in a codebase. It's also honest about the practical lessons. It plumbed Semgrep through the whole system and the hunters invoked it zero times in a month of runs, while the "wishlist" mechanism, where agents request a missing tool or build environment, became the most-used tool in the harness. Pay attention to what agents reach for, not what you think they'll want.

The takeaway

This is the most complete open-source look at how one company runs agent-driven security auditing, in a form you can adapt rather than just read. Whether you fork the skill as-is or borrow the adversarial-validation pattern for other agent work, the design principle is portable: the model that discovers a problem should never be the model that decides whether the problem is real.