Deepsec - Agent-Powered Vulnerability Scanner

Vercel-Labs open-source security harness. A selectable Codex, Claude, or Pi backend investigates security-sensitive files, traces data flows, checks mitigations, and emits actionable findings with severity ratings. Runs locally on Kevin's machine or fans out to Vercel Sandboxes for monorepo-scale parallelism.

Current Version Snapshot

Source: GitHub API; npm registry, 2026-08-10

The thesis

Static scanners produce a flood of generic CWE matches that engineers ignore. SAST tools tuned for false-positive minimization miss the subtle, project-specific bugs that actually ship to production - auth condition edge cases, custom rate limiter bypasses, in-house ORM patterns. Coding agents close that gap: they read each candidate site with codebase context and decide whether a real vulnerability exists.

deepsec packages that loop:

1. Regex pre-scan - ~110 matchers identify candidate sites (~15s, no AI calls)
2. AI investigation - the selected Codex, Claude, or Pi model reads each candidate, traces data flow, checks for upstream guards, and emits a finding plus severity
3. Revalidation - second agent run re-checks each finding, including git history, to cut FP rate
4. Enrichment - git committer info attached for ownership routing
5. Export - markdown-per-finding or JSON for downstream tickets

The harness is designed to run on trusted source code (your own repos) but still expects prompt injection from vendored / third-party deps. Sandbox mode isolates worker microVMs from the main process and the API keys.

Pipeline

┌──────────┐  ┌──────────┐  ┌─────────────┐  ┌─────────────┐  ┌──────────┐
│   scan   │→ │ process  │→ │   triage    │→ │ revalidate  │→ │  export  │
│ (regex,  │  │ (Opus,   │  │ (P0/P1/P2,  │  │ (re-read +  │  │ (md / js)│
│  ~15s,   │  │  $$$,    │  │  cheap)     │  │  git hist)  │  │          │
│  no AI)  │  │  iter.)  │  │             │  │             │  │          │
└──────────┘  └──────────┘  └─────────────┘  └─────────────┘  └──────────┘
                                                  ↓
                                           ┌─────────────┐
                                           │   enrich    │
                                           │ (git author │
                                           │  + plugin)  │
                                           └─────────────┘

Every command is idempotent - interrupt a job, restart, deepsec resumes from where it left off. State lives in data/<project-id>/files/ (one JSON FileRecord per scanned source file) plus runs/, project.json, INFO.md.

How it lays itself down

deepsec installs into a .deepsec/ directory at the root of the target repo, checked into git:

my-app/
├── .deepsec/
│   ├── package.json              # pinned deepsec version
│   ├── deepsec.config.ts         # one entry per project to scan
│   ├── AGENTS.md                 # agent prompt for INFO.md filling
│   ├── .env.local                # AI_GATEWAY_API_KEY (gitignored)
│   ├── .gitignore                # excludes data/<id>/files/, runs/
│   └── data/<project-id>/
│       ├── INFO.md               # project-specific context (50-100 lines)
│       ├── SETUP.md               # per-project agent prompt
│       ├── project.json
│       ├── files/                # one JSON per scanned file (gitignored)
│       └── runs/                 # AI run records (gitignored)
└── ...

Config and matchers travel with the code; generated output stays gitignored. To scan multiple repos from one .deepsec/, run pnpm deepsec init-project <path> to register additional projects.

Cost reality

deepsec uses high-reasoning coding agents, and the selected model/harness pair can change with the live benchmark. The harness exists because shallow review misses the project-specific bugs; no historical model name is a permanent operating default.

Costs swing 2–3× based on file complexity. Vercel routinely runs scans that cost low five figures on their monorepos. The bias is: pay once, find the lurking bugs, patch them. Customers told them the cost was worth it for what would otherwise have shipped to prod.

Always start with --limit 50 to calibrate before committing to the full pass. That's $5–15 to learn the per-file rate on a specific codebase. Multiply out, then decide.

Models and benchmark-governed routing

Deepsec 2.3.4 has interchangeable Codex, Claude, and Pi backends. The packaged defaults at this review are Codex/GPT-5.5, Claude/Opus 4.8, Pi/GLM 5.2, and Claude/Sonnet 4.6 for triage. Those are compatibility defaults, not Kevin's permanent ranking.

Interactive and headless setup now read the current DeepSecBench snapshot and support three profiles:

npx deepsec init --yes --model-profile value --output jsonl
pnpm deepsec process --project-id my-app --limit 50 --concurrency 5

Preserve the benchmark JSON, selected model, harness, reasoning level, credential route, provider zero-data-retention posture, and observed limited-run cost/refusal behavior. Direct provider credentials restrict compatible profiles; open-weight and alternate models use the Pi backend. A social or private stealth benchmark changes the candidate set, not the standing default.

The two July bookmarks are useful time-bounded evidence: the undisclosed benchmark favored GPT-5.6 Sol for the strongest baseline and Kimi K3 for continuous price/quality, with GLM 5.2 as a cheaper candidate and Fable showing refusals. The current public DeepSecBench snapshot has already shifted the exact scores and price ratios. Therefore use best once and value/budget for continuous work only after the live snapshot and repo-specific 50-file calibration agree. Source: X/@cramforce and X/@rauchg, 2026-07-18 through 2026-07-19; Deepsec docs/models.md; DeepSecBench snapshot, 2026-08-10

Refusals

After every batch, Deepsec asks the agent for a structured refusal report and stores it in run and file metadata. Refused files remain pending, so another backend can reinvestigate without silently becoming a false negative. The saved benchmark's Fable result is a warning for that snapshot, not a universal model fact; each run must report observed refusals and preserve the selected model and harness.

Vercel Sandbox fanout

Large monorepos can fan work across Vercel Sandbox microVMs:

pnpm deepsec sandbox process \
  --project-id my-app \
  --sandboxes 10 \
  --concurrency 4

The local working tree is tarballed (.git excluded) and uploaded. Vercel runs scans on their own codebases at 1,000+ concurrent sandboxes.

Security model:

• API keys are injected outside the sandbox - cannot be exfiltrated by a compromised worker
• Network egress from worker sandboxes is limited to coding agent hosts - only Anthropic/OpenAI/AI Gateway are reachable during the AI run
• The bootstrap step does have egress (to install deps), but the coding agent doesn't run during bootstrap

This is the standard deepsec story: treat the harness like a coding agent with full shell access on trusted source code, but accept that vendored deps may attempt prompt injection - sandbox mode bounds the blast radius.

INFO.md - the highest-impact tunable

data/<id>/INFO.md is injected into every scan batch's prompt. Vague INFO.md → vague findings.

Vercel's discipline:

• 50–100 lines total. Verbose context dilutes signal.
• 3–5 examples per section, not exhaustive enumeration.
• Name primitives (auth helpers, middleware functions, ORM patterns) but no line numbers (they go stale fast).
• Skip generic CWE categories - built-in matchers already cover those.
• Cover only what's project-specific - custom auth, custom rate limiter, in-house ORM patterns, atypical conventions.

Best practice: open the parent repo in a coding agent, paste the prompt that init printed, and let the agent fill INFO.md after skimming README + AGENTS.md + 5–10 representative files.

Custom matchers (compounding signal)

The killer feature for established codebases: after a first scan, prompt the coding agent to write project-specific matchers.

Inspect previous deepsec runs against ./my-app. Are there custom matchers we should add to find more candidates for vulnerabilities? Look at the auth model, data layer, and team conventions.

Each new matcher widens regex pre-scan coverage without adding AI cost. This is how Vercel built their internal "every authentication path" scanner - running deepsec on their monorepo found subtle auth condition edge cases, and they shipped a custom plugin to catch the same shape going forward.

See docs/writing-matchers.md.

False-positive rate

Vercel reports 10–20% FP rate in their experience. The revalidate step exists specifically to cut this - a second agent run re-reads the code and git history, then emits TP / FP / Fixed / Uncertain. Cuts FP rate by 50%+ on most repos for ~comparable cost to process.

Worth running on the HIGH/CRITICAL set before showing findings to anyone.

Workflow reference

Where deepsec sits in Kevin's security stack

• Brin - Agent Context Security - pre-install gate for npm/PyPI/MCP/skills. Prevents bad supply chain entering the codebase. Before code is in the repo.
• Golf - MCP Agent Security - protocol-level governance over which MCP tools agents can use. Lateral to deepsec.
• Lint-Enforced Agent Guardrails + Ultracite - catch common mistakes during edit/commit. Continuous, low-cost.
• Skill Creator + Auditor + skill-auditor - vet skills before install. Pre-install.
• Security and Review Skills skill - adversarial CTF-style audit with subagents per file. Surgical, attack-surface-focused, $.
• deepsec - complete codebase scan. Periodic, deep, $$$.

The bugs skill and deepsec are complementary: bugs is for "I have 20 critical files I need eyes on now", deepsec is for "I want a complete sweep across all 2,000 files in this monorepo with a real false-positive process".

Current routing signal

The 2026-06-28 bookmark resync surfaced fresh social proof around running deepsec across repos and a reply pointing back to vercel-labs/deepsec. The reviewed local image is a GitHub repository card for deepsec, not a separate source of feature truth; use it as social-proof evidence and the GitHub/npm snapshots above for current tool facts. This does not change the ranking: deepsec stays the complete repo-wide scanner, while bugs stays the cheaper surgical adversarial review. It does reinforce the operational default that production-facing repos should eventually get a calibrated deepsec pass, but not an automatic full scan without explicit cost approval. Source: X/@DavidOndrej1 and local image review, 2026-07-04; Source: X/@tamsi_besson, 2026-06-28

Validation in the wild

"We've been on a lookout for a tool to do security scans on our open source repositories. deepsec's scans have been the most thorough, with the most findings, and good true-positive rate." - James Perkins, Co-founder and CEO @ Unkey Source: Vercel blog, 2026-05-04

"We get a lot of automated security reports, but most of them aren't actionable. deepsec is the first tool that's surfaced the kind of issues we'd actually want a security engineer to flag, and it runs on infrastructure we control." - Steven Tey, Founder and CEO @ dub.co Source: Vercel blog, 2026-05-04

Vercel ran deepsec on dub.co, Unkey, and several other open-source customer/partner repos during development.

Plugin system

deepsec ships with a plugin system for adapting it to a specific codebase. The most common plugin is a custom scanner - regex matchers tuned to a specific auth model, data layer, or team convention. Plugins live alongside the config in .deepsec/. See docs/plugins.md.

Limitations

• Best for applications and services, not libraries/frameworks. The latter would need custom prompts and scanners.
• Costly - full Opus passes are $$$.
• Trusts source code - runs as if invoked by the codebase author. For untrusted vendored code, prefer sandbox mode.
• English-language matchers - some bugs require domain-specific patterns the regex layer doesn't catch. Custom matchers compensate.

Operating playbook

When Kevin says "scan X for vulns" or similar:

1. Brin-check the target if it's an external repo (curl https://api.brin.sh/repo/<owner>/<repo>)
2. Verify Node 22+, pnpm available, and at least one of: logged-in claude/codex, or AI gateway key
3. cd to the target repo
4. npx deepsec init → cd .deepsec → pnpm install
5. Fill INFO.md (use the agent prompt deepsec printed)
6. pnpm deepsec scan → report status to Kevin
7. Preserve the current DeepSecBench output, select best, value, or budget, then calibrate with pnpm deepsec process --limit 50 --concurrency 5 and compute projected total cost and observed refusals
8. Show Kevin the projected dollar amount, get explicit approval
9. Full process, then triage and revalidate on HIGH/CRITICAL
10. pnpm deepsec export --format md-dir --out ./findings
11. Report findings: lead with revalidated CRITICAL/HIGH, group MEDIUM by class, discard pure defense-in-depth LOW
12. Append scan record to wiki/log.md. Draft postmortem if a non-trivial bug surfaced.

The full skill is at ~/.cursor/skills/deepsec/SKILL.md (mirrored to .claude/ and .codex/).

---

Timeline

• 2026-08-10 | Updated to deepsec@2.3.4 and replaced the frozen Opus/GPT route with live DeepSecBench best/value/budget profiles across Codex, Claude, and Pi. The saved stealth-eval rankings now inform a time-bounded baseline-versus-continuous strategy while exact scores, costs, and refusals must be re-proven per snapshot and calibration. Source: X/@cramforce; X/@rauchg; GitHub/npm; Deepsec models docs; DeepSecBench
• 2026-07-04 | Re-checked the source authority during artifact review: deepsec@2.1.2, Apache-2.0, no GitHub release tags observed, repo pushed 2026-07-02, default-branch HEAD still cda4c04f59fdef0bc5c0ab61b7269cb077e4e5e0, 5,067 stars, and 300 forks. The reviewed bookmark image is a GitHub repo card, so it reinforces routing but does not add new technical behavior. Source: GitHub API; npm registry; X/@DavidOndrej1 local image artifact, 2026-07-04
• 2026-07-02 | Refreshed source authority after the X artifact review: deepsec@2.1.2, Apache-2.0, no tags, repo HEAD cda4c04f59fdef0bc5c0ab61b7269cb077e4e5e0, 5,044 stars, and 299 forks. Routing unchanged: use security-doctor-chain for composed security gates and security-scan-workflow for an explicit, cost-approved deepsec run. Source: GitHub API; npm registry, 2026-07-02
• 2026-06-29 | Refreshed the version snapshot: deepsec@2.1.2, Apache-2.0, repo 4.8K stars, no GitHub release tags observed. Source: GitHub API; npm registry, 2026-06-29
• 2026-05-04 | Tool introduced. Vercel open-sourced vercel-labs/deepsec. Brin verdict on repo/vercel-labs/deepsec: caution (score 65, medium confidence, deep scan pending). Source: vercel-labs (established, trusted publisher). Source: Vercel blog, 2026-05-04
• 2026-05-04 | Skill installed. ~/.cursor/skills/deepsec/SKILL.md (mirrored to .claude/ and .codex/). Wired into Kevin's security stack alongside Security and Review Skills, Brin - Agent Context Security, Skill Creator + Auditor, and Lint-Enforced Agent Guardrails. Source: User, 2026-05-04
• 2026-06-28 | Bookmark resync reinforced deepsec as the repo-wide vulnerability-scan route through a new "run deepsec on all your repos" signal plus source reply to vercel-labs/deepsec; routing unchanged, cost gate preserved. Source: X Bookmark Ingest 2026-06-28