Visa Vulnerability Agentic Harness — Agentic Vulnerability Discovery, Remediation, and Validation

VVAH is Visa's open-source harness for autonomous vulnerability discovery, remediation, and validation using large language models, built on learnings from Project Glasswing (Anthropic's initiative for AI-assisted vulnerability research).

VVAH runs as a four-phase pipeline: an S0 static seed plus eleven stages, S1–S11, from code ingest to validated fix. The shipped default.yaml and full.yaml profiles derive that seed from the model; taint.yaml uses rules mode, which needs operator-supplied source/sink YAML. The four phases:

Phase 1 — Discovery & Modeling (S1–S3): map the attack surface and build a threat-aware plan.
Phase 2 — Deep Dive & Verification (S4–S6): run multi-lens analysis and adversarial verification to assess likely exploitability.
Phase 3 — Synthesis & Reporting (S7–S9): deduplicate, chain, and emit structured findings (Markdown + SARIF).
Phase 4 — Remediation & Validation (S10–S11): propose candidate fixes and adversarially validate them before adoption.

Three design choices drive finding quality: threat modeling before analysis focuses the attack surface; multi-agent deterministic voting reduces false positives; and structured triage artifacts compress the lifecycle from AI-discovered weakness to actionable finding. The bottleneck in AI-assisted vulnerability management is triage speed, not discovery. VVAH is designed around that constraint. The primary effectiveness metric is Mean Time to Adapt (MTTA): elapsed time from AI-discovered exploitability to a validated fix in production.

Multi-model by design. Every model-driven role — S0 through S11 — can be pointed at its own model and provider:

Anthropic Claude — native route
OpenAI-compatible — OpenAI models and any OpenAI-compatible gateway
Open-weight — served over Chat Completions-compatible endpoints

No single provider is a hard dependency. Detection and remediation/validation draw on the same set; see docs/models.md for the per-role matrix.

For setup, see docs/SETUP_GUIDE.md. GitHub Issues are open for bug, documentation, and setup reports — see Reporting issues. This repository is not currently accepting external code contributions; see CONTRIBUTING.md for details.

Authorized use only. Run scans only against code you own or have explicit permission to test. Findings and fixes are LLM-generated triage candidates that require human review — see Limitations.

Data egress warning. Any role routed to via: cli, via: sdk, via: openai, or via: deepagents sends prompt data to that model provider endpoint (Anthropic/OpenAI or your configured gateway). Use only approved endpoints and scan targets you are authorized to process.

Docs: SETUP_GUIDE.md — install & configuration · USER_GUIDE.md — commands & options · models.md — model/backend selection · remediation.md · validation.md · Project Glasswing white paper — technical background.

What's new in 1.3

Improved token economics and caching
DeepAgents orchestration across S0–S11
Specialist security lenses expanded from 6 to 11
Stronger threat modelling and verification

Features

From detection to a graded fix — the S0–S11 pipeline combines threat- modeled discovery, deep-dive analysis, adversarial verification, reporting, remediation, and fix validation.
Threat-aware analysis — threat modeling and multi-lens research focus review on the attack surface that matters.
Reachable-code analysis — AST/call-graph seeding focuses model review on relevant code paths rather than the whole repo.
Real data-flow evidence — interprocedural taint analysis complements model-based review across supported application languages.
Coverage backstop — unrecognised files can be added to catch-all review chunks instead of being silently excluded. The backstop is best-effort, not a guarantee; see docs/security.md for coverage guidance.
Actionable output — produce Markdown and SARIF 2.1.0 reports with CVSS, CWE, and per-run diagnostics.
Your models and gateway — use Anthropic or OpenAI-compatible models through cli, sdk, openai, or deepagents routes, configured per role.
Portfolio-scale operations — scan CSV-defined repositories with resumable state and monitor long-running work through stderr output or the optional --s6-progress-file artifact.

See docs/features.md for the full capability reference, including backend and stage details, specialist lenses, taint analysis, and limitations.

Quick start

Prerequisites

Python 3.11 or newer.
Permission to scan the target repository.
One Anthropic credential for the packaged default profile: ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, or ANTHROPIC_SDK_API_KEY. Claude Code CLI auth (run claude, then /login; or claude setup-token) is optional and needed only for a role you configure with via: cli; no shipped profile does.

Credential and profile details are in docs/SETUP_GUIDE.md and docs/models.md.

Install and configure

git clone https://github.com/visa/visa-vulnerability-agentic-harness VisaVulnerabilityAgenticHarness
cd VisaVulnerabilityAgenticHarness

macOS / Linux

python3 -m venv .venv
source .venv/bin/activate
pip install .

Windows PowerShell

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install .

cp .env.example .env
$EDITOR .env                       # set an Anthropic credential
claude                              # optional: only for via: cli roles — /login in the REPL (or: claude setup-token)

pipx install . is an alternative. For platform-specific setup, TLS/proxy notes, editable installs, and profile selection, use docs/SETUP_GUIDE.md.

Check the installation

vvaharness --version
vvaharness setup                    # readiness check; no model spend
vvaharness doctor                   # live backend check; spends model tokens

For the full configuration reference, see docs/configuration.md and docs/models.md.

Run your first scan

Step 1 — Detection only, no code edits. Always start here:

vvaharness scan --repo /path/to/your/repo --stop-after s9

Step 2 — Review the Markdown report and SARIF output in /path/to/your/repo/security-scan/.

Step 3 — When ready, run the full scan with fixes. This edits source files:

vvaharness scan --repo /path/to/your/repo

After a detection-only run, remediate and validate findings independently:

vvaharness remediate --repo /path/to/your/repo --mode report-only  # propose only
vvaharness remediate --repo /path/to/your/repo                     # applies fixes
vvaharness validate --repo /path/to/your/repo                      # validate fixes

For command flags and troubleshooting, see USER_GUIDE.md.

Use with an AI agent (Claude / Copilot / Gemini)

vvaharness setup --install-agents

This writes AGENTS.md (cross-tool) and .github/copilot-instructions.md (Copilot) unconditionally; when the claude CLI is detected it also writes CLAUDE.md + a Claude skill in ~/.claude/skills/ (Claude Code), and when the gemini CLI is detected it writes GEMINI.md (Gemini CLI). Existing files are left untouched. See docs/SKILLS.md for the analysis capabilities.

Pipeline

VVAH implements an S0 static seed plus an eleven-stage workflow. The common operator path is detection-only through S9; the shipped default profile also runs S10 remediation and S11 validation unless you stop at S9.

Stage group	Stages	Purpose
Static seed (optional)	S0	Source/sink callgraph seed for taint-first scanning
Discovery & Modeling	S1–S3	Attack surface mapping, threat modeling, hunting plan
Deep Dive & Verification	S4–S6	Multi-lens research, policy gates, adversarial verification
Synthesis, Chaining & Reporting	S7–S9	Deduplication, chain construction, SARIF emission
Remediation & Validation	S10–S11	Candidate fixes and adversarial fix validation

For stage-by-stage internals, see docs/architecture.md. For every command and flag, see docs/USER_GUIDE.md.

Skills

The pipeline combines stage prompts, language-specific lenses, specialist security lenses, remediation playbooks, and validation personas. See docs/SKILLS.md for the full capability inventory, and docs/features.md for how those capabilities are selected by configuration.

Output

Per target, under <target>/security-scan/:

findings.json — typed FinalReport JSON
<module>_<ts>_report.md — findings + dropped-findings appendix
<module>_<ts>_report.sarif — SARIF 2.1.0
<module>_<ts>_errors.jsonl — non-fatal errors

With the default profile, a scan also writes <target>/security-remediation/<NN_slug>/finding_case.json and edits source files in the target repo (S10 fix mode — see the Quick start warning); pass --stop-after s9 to skip. A timestamped run_manifest_*.json is written to the working directory.

Pipeline checkpoints and resume state are kept outside the scanned repo, in a SQLite state DB at $VVAHARNESS_STATE_DIR/vvaharness.db (default ~/.vvaharness/state/); prune old runs with vvaharness gc.

For report, SARIF, findings.json, remediation, validation, and manifest schemas, see docs/outputs.md.

Limitations (read before you trust output)

LLM-generated, non-deterministic. Findings and fixes are triage candidates, not confirmed vulnerabilities or production-ready patches. Human review is required, and runs may differ.
No compilation or execution. VVAH reads the target and does not build, run, or test it; findings are never confirmed by execution. No shipped profile grants the agent Bash.
No published accuracy numbers yet. Precision and recall have not been published.
Coverage is bounded, not complete. Stage caps, failed or timed-out chunks, unsupported languages, and non-path-sensitive taint analysis can leave code unreviewed. See docs/security.md for coverage guidance.
Modifies the target by default. The default, full, and sdk profiles enable remediation; use --stop-after s9 or taint.yaml for detection-only operation.
Review remediation fixes before you rely on them. VVAH does not build or test patched code; review and test generated fixes before merging.
Elevated privilege. Run VVAH only against authorized, trusted repositories; prompt data may expose sensitive files or credentials to configured providers.
Token-hungry. There is no global spend cap. Run vvaharness estimate first and scope large scans; see docs/configuration.md.

See docs/features.md and docs/security.md for the complete limitations and safety model.

Learn more

Visa Perspectives announcement — the public VVAH announcement.
Project Glasswing white paper — technical background on the approach behind VVAH.

Security

Report vulnerabilities responsibly through the private channels in SECURITY.md. The public issue tracker is open for bug, documentation, and setup reports — never for security vulnerabilities.

Reporting issues

Bugs, documentation, and setup reports go through the issue forms in .github/ISSUE_TEMPLATE/ (blank issues are disabled). See reporter guidance in docs/contributor-issue-guide.md. Report security vulnerabilities privately — see SECURITY.md.

License

Third-party dependencies are installed from PyPI at install time (not bundled in this repository); their licenses are inventoried in THIRD_PARTY_LICENSES.md.

See CHANGELOG.md for release history.