# Pentest Swarm AI — Implementation Plan > **Status**: v2 — planning horizon ~6 months > **Last updated**: 2026-04-19 > **Owner**: @AkhilSharma90 This document is the source of truth for the roadmap from v0.1 (current) to v1.0. Every task is scoped small enough to land in a single PR. Check boxes as you ship. If you're a contributor looking for where to help: look for `P0` (blocking) and `good-first-issue` tags inside each phase. --- ## Table of Contents 1. [North Star & Positioning](#north-star--positioning) 2. [Wave 1 — Credibility Debt (4–6 weeks)](#wave-1--credibility-debt-46-weeks) 3. [Wave 2 — Integrations & Workflows (6–8 weeks)](#wave-2--integrations--workflows-68-weeks) 4. [Wave 3 — Research Frontier (ongoing)](#wave-3--research-frontier-ongoing) 5. [Wave 4 — Researcher Workflow: The XBOW-for-Bug-Bounty Play (6–8 weeks)](#wave-4--researcher-workflow-the-xbow-for-bug-bounty-play-68-weeks) 6. [Wave 5 — Vulnerability Class Coverage (~6 months)](#wave-5--vulnerability-class-coverage-6-months) 7. [Wave 6 — Real-World Operability (~3 months)](#wave-6--real-world-operability-3-months) 8. [Distribution & Community Growth](#distribution--community-growth) 9. [README Rewrite](#readme-rewrite) 10. [Architectural Decisions](#architectural-decisions) 11. [Benchmarks Target](#benchmarks-target) 12. [Research References](#research-references) --- ## North Star & Positioning **Goal**: Pentest Swarm AI is the first open-source pentesting tool that is a *real swarm* — decentralized agents coordinating through shared environment state (stigmergy), producing emergent attack paths no single planner specified. **Non-goals (v1)**: replacing human pentesters; web-only scope. **Long-arc vision (Wave 4)**: a bug-bounty researcher runs **one command** against a HackerOne program they've been accepted to and gets a list of verified, reproducible vulnerabilities with ready-to-submit writeups. Same one-command experience XBOW sells as a SaaS, open-sourced. If researchers file real vulns with this tool, everything else follows — stars, contributors, signal in the market. **Moat vs. competitors**: - vs. **PentestGPT**: we execute, not just suggest - vs. **HackingBuddyGPT**: we have a full agent swarm, not a single agent - vs. **PentAGI**: we are stigmergic, not orchestrated - vs. **HexStrike**: we are not a thin MCP wrapper; we have agent reasoning - vs. **Shannon**: we're open-core-friendly and swarm-native --- ## Wave 1 — Credibility Debt (4–6 weeks) > Make the README true. Make the swarm real. Fix the footguns. ### Phase 1.1 — True Swarm Architecture (the big one) Replace the sequential 5-phase runner (`internal/engine/runner.go:56-298`) with a stigmergic Blackboard pattern. This is the feature that makes the product name honest. **Core design:** - **Blackboard**: a Postgres-backed shared knowledge store (findings, hypotheses, open tasks, blocked tasks). Uses existing pgvector for semantic recall. - **Agents**: long-running workers that subscribe to the blackboard. Each has a *trigger rule* (SQL/vector predicate) that wakes it when relevant state appears. - **Stigmergy**: every agent write includes a *pheromone weight* — how "interesting" the finding is. Weights decay over time. Other agents' trigger rules read weighted state, biasing exploration. - **Scheduler**: no central planner. A thin coordinator rate-limits concurrent agents, enforces scope, and kills on signal. Selection is emergent from trigger rules + pheromones. **Tasks:** - [x] **1.1.1** Define `Blackboard` Go interface in `internal/swarm/blackboard/board.go` - [x] `Write(ctx, Finding) error` — append-only, returns finding ID - [x] `Query(ctx, Predicate) iter.Seq[Finding]` — SQL + vector hybrid - [x] `Subscribe(ctx, Predicate) <-chan Finding` — blocking stream - [x] `Pheromone(findingID) float64` — weight with time decay - [x] Unit tests for write/query/subscribe with fake clock - [x] **1.1.2** Postgres schema migration `migrations/000004_blackboard.sql` - [x] `swarm_findings` table: id, campaign_id, agent_name, finding_type, target, data (jsonb), embedding (vector), pheromone_base, half_life_sec, created_at, superseded_by - [x] Index on (campaign_id, type); pgvector column present; HNSW index to land with first real embeddings - [x] `swarm_pheromone()` SQL function (exponential, half-life configurable per-type) - [ ] Backfill script for existing DB *(deferred — no legacy swarm data to migrate)* - [x] **1.1.3** Agent `Trigger` abstraction via `blackboard.Predicate` - [x] `Predicate` as a composable struct (type match, pheromone threshold, SinceID cursor) - [x] Trigger evaluation is idempotent — each agent commits a cursor after Handle - [x] **1.1.4** Refactor each existing agent to the swarm model *(adapter-wrapper approach, preserves legacy path)* - [x] `internal/swarm/agents/recon.go` — triggers on `TARGET_REGISTERED`, publishes `SUBDOMAIN`, `PORT_OPEN`, `HTTP_ENDPOINT`, `TECHNOLOGY` - [x] `internal/swarm/agents/classifier.go` — triggers on raw findings > 0.2 pheromone, publishes `CVE_MATCH` / `MISCONFIGURATION` - [x] `internal/swarm/agents/exploit.go` — triggers on `CVE_MATCH` with pheromone > 0.5, publishes `EXPLOIT_CHAIN` / `EXPLOIT_RESULT` - [x] `internal/swarm/agents/report.go` — triggers on `CAMPAIGN_COMPLETE` - [x] **1.1.5** New scheduler in `internal/swarm/scheduler.go` - [x] Blackboard-driven dispatch replaces the phase loop (available behind `--swarm`) - [x] Per-agent concurrency caps - [x] Graceful shutdown on SIGINT via runCtx cancellation - [x] Campaign-level budget (agent-hours + tokens) enforced by the budget watcher - [x] **1.1.6** Pheromone tuning - [x] Per-finding-type decay half-lives in `config/pheromones.yaml` (embedded default + override via `Load`) - [x] CLI flag `--exploration-bias {low,med,high}` — scales pheromone base at write time - [ ] **1.1.7** Delete `internal/engine/runner.go` sequential pipeline (keep a `legacy` subcommand behind `--legacy` flag for 1 release) - [x] **1.1.8** Integration test: `tests/integration/swarm_e2e_test.go` — seed → 3 agents → asserts dispatch counts + final board shape - [x] **1.1.9** Observability - [x] `swarm.Tracer` interface + NoopTracer; scheduler wraps every `Agent.Handle` in a span (OTel bridge is a few-line adapter — no SDK dep needed) - [x] Grafana dashboard JSON in `deploy/metrics/grafana-swarm-dashboard.json` (findings-per-type, active agents, budget, error rate) - [x] `blackboard.LoggingBoard` emits structured JSON for every write / cursor commit / budget mutation via zap ### Phase 1.2 — Dashboard Wire-up Current state: `web/` dashboard renders, `SeverityChart` hardcodes zeros, runner never persists to DB or streams to WS. - [ ] **1.2.1** Wire scheduler → DB persistence in `internal/db/findings.go` *(runner emits structured finding events now; DB persist on the server path is next)* - [x] **1.2.2** Wire scheduler → WebSocket `EventHub` for live updates *(server.startCampaign publishes every event via hub.Publish)* - [x] **1.2.3** Replace mock zeros in `SeverityChart` with a live query against `/api/v1/stats` - [ ] **1.2.4** Add "live swarm" view: animated pheromone graph, active agents, findings stream - [ ] **1.2.5** Playwright test: run `pentestswarm scan --dry-run` against mock target, assert chart values non-zero in <10s - [ ] **1.2.6** Add campaign diff view (run N vs run N-1) — foundation for ASM mode ### Phase 1.3 — Safety Fixes - [x] **1.3.1** Wire cleanup registry (was `nil` in `internal/engine/runner.go:211`) - [x] Every exploit that creates artifacts (files, users, sessions) registers a cleanup - [x] Cleanup runs on normal exit, SIGINT, and scheduler crash (detached context survives ctx cancel) - [x] Unit tests: `internal/agent/exploit/shellparse_test.go` + integration via runner defer - [x] **1.3.2** Fix silent LLM fallback in classifier - [x] Surface errors to event stream via `classifier.WithErrorSink` + `recon.WithErrorSink` - [x] Add `--strict` CLI flag — abort on any LLM failure - [x] Default mode: heuristic fallback + WARN event - [x] **1.3.3** Fix recon empty-AttackSurface on JSON parse failure - [x] Retry prompt with narrower schema (already present) - [x] On second failure, emit error event to stream (strict mode promotes to fatal) - [x] **1.3.4** Harden command executor - [x] Replace naive field splitting with quote-aware `parseCommand` - [x] Reject pipes, redirects, backticks, `$(...)`, newlines unless inside quotes - [ ] Sandbox: all exploit commands run in a Docker container by default *(deferred to Wave 2)* - [x] **1.3.5** Scope enforcement audit - [x] All 8 tool adapters now route through `scope.ValidateAndLog(tool, target, def)` instead of the bare `Validate()` so every check is logged - [x] Violations emit `WARN subsystem=scope` with tool + target + allowed scope, then return the error — never log-and-continue - [x] 9 unit tests cover CIDR boundaries, subdomain matching, wildcards, excluded ranges, and ValidateCommand's non-target allow-list ### Phase 1.4 — LLM Layer Upgrades - [x] **1.4.1** Model name is config-driven (`OrchestratorConfig.Model`; per-agent via `AgentsConfig` + `NewAgentProvider`) - [x] Default to `claude-sonnet-4-6` - [x] Support `claude-opus-4-7`, `claude-haiku-4-5-20251001` (any Anthropic model ID) - [x] Per-agent model override (cheap agents on Haiku, reasoning on Opus) via `agents.*.model` - [x] **1.4.2** Prompt caching on Claude - [x] Cache system prompt (tool definitions not yet cached — follow-up) - [ ] Cache per-campaign shared context (scope, objective, recon summary) *(follow-up)* - [x] Emit cache-hit metrics via `Usage.CacheHitRate()` - [x] **1.4.3** Structured tool-use for the classifier - [x] `emit_classified_findings` tool with JSON-Schema enum for severity + confidence - [x] Tool-call path used for providers with `SupportsToolUse()` (Claude); legacy JSON-in-prompt retained as fallback for Ollama / LM Studio - [x] **1.4.4** Token budget per campaign + per agent — hard cap with soft warn - Per-campaign budget already enforced by scheduler (agent-hours + tokens) - Per-agent layer added: `swarm_agent_budgets` table, `AgentBudget` / `ChargeAgent` / `SetAgentBudget` on the Board; scheduler emits `agent_budget_warn` at soft threshold and skips dispatch after hard cap - 3 unit tests cover defaults, warn transition, threshold-raise clears warned flag - [x] **1.4.5** Eval harness at `tests/llm_eval/` - YAML fixture DSL (`severity`, `cvss_min`/`cvss_max`, `*_any_of` allow-lists, `contains_in_description`) - 3 classifier fixtures shipped: critical SQLi, medium XSS, low info-disclosure - MockProvider replays fixture responses through the real classifier code path so the eval tests the *code*, not the LLM - Ready for a `-live` flag extension to run the same rubrics against a real Claude provider - [x] **1.4.6** **OpenAI-compatible provider** — `internal/llm/openai.go` implements the `Provider` interface for any OpenAI-API-compatible endpoint. Covers Together AI (Qwen, Kimi-K2, DeepSeek V3, Llama 3.3 — one key, many models via base-URL config), DeepSeek direct, Moonshot/Kimi direct, Groq, OpenAI itself. Wired into `internal/llm/factory.go` as the fourth provider (`claude` / `openai` / `ollama` / `lmstudio`). Exponential-backoff retry on 429/5xx, immediate fail on 4xx. SSE streaming + non-streaming, native tool-use, health check via `GET /models`. 10 unit tests with httptest — no real-API calls in CI. - [x] Pricing tables in `internal/llm/pricing.go` for Together AI's main tiers (Llama 3.3-70B Turbo, Qwen 2.5-72B Turbo, DeepSeek V3, Kimi K2) + DeepSeek direct (chat, reasoner) + OpenAI direct (gpt-4o, gpt-4o-mini) - [x] `SupportsToolUse()` gating via `DisableToolUse` config — defaults to true; opt-out for models that advertise function-calling but parse erratically; opt-out path strips tools from outgoing requests so callers fall back to the JSON-in-prompt path - [ ] Integration test against a low-cost Together AI model (real-API smoke, not CI — keeps CI spend trivial) - [ ] README "Which provider?" section with the trade-off matrix: Claude (max quality, refuses offensive prompts), Together AI / DeepSeek / Kimi (cheaper, less restrictive, lower CTF capability), Ollama (free, fully local, lowest score) - [ ] **Strategic rationale**: (a) enables the multi-column benchmark in [3.3.1](#phase-33--benchmarks-the-credibility-lever) — Claude headline + cheap-backend second column is more interesting than a single number; (b) addresses Claude's refusal rate on offensive-security prompts that bites researchers mid-engagement; (c) Together AI is ~10× cheaper than Sonnet for debug/iteration cycles (matters for the [Cybench](#phase-33--benchmarks-the-credibility-lever) $200 cap) - [ ] **1.4.7** **Offensive-security prompt scaffolding** (`P1`) — hand-crafted system prompts, few-shot examples, and reasoning templates tuned for offensive-security tasks. Zero training cost; pure prompt engineering. Captures part of the gap between frontier-model baseline (~12% Cybench) and the SOTA fine-tuned approach in [Pentest-R1](https://arxiv.org/abs/2508.07382) (~40% Cybench) without any compute spend. Expected uplift: +3–8 percentage points on Cybench-class benchmarks. Multiplies every benchmark number we publish in [3.3.1](#phase-33--benchmarks-the-credibility-lever). **Infrastructure shipped**; content/eval are follow-ups. - [x] Curated system prompt at `internal/agent/prompts/templates/offsec_system.tmpl` (and strict-reframe variant `offsec_system_strict.tmpl`): authorized-pentest framing, vuln-class priors (auth bugs ~30% of payouts, IDOR, SSRF→cloud-metadata, subdomain takeover), explicit reasoning loop (RECON → HYPOTHESIS → TEST → CONFIRM), anti-patterns list, output discipline. Embedded into the binary via `//go:embed` - [ ] Few-shot example library at `internal/agent/prompts/examples//` — 3 seed examples shipped (web/01_idor, auth/01_jwt_alg_confusion, api/01_graphql_introspection) to validate the loader contract. Need ~15-25 more across crypto / pwn / rev / forensics / cloud / and deeper coverage in web / auth / api before this can claim full library status - [ ] Tool-use templates at `internal/agent/prompts/tooluse/` — exemplary nmap / sqlmap / burp / nuclei invocations paired with the reasoning that led to them - [x] Refusal-handling retry path: `prompts.IsRefusal` detects Claude refusals via opening-150-char heuristic against curated phrase list; `prompts.RetryProvider` wraps any `llm.Provider` with the chain "primary → strict reframe → fallback provider"; `prompts.ErrAllAttemptsRefused` sentinel surfaces when every attempt was refused. Stream/HealthCheck/ModelName etc. pass through transparently. 23 unit tests with a scripted mock provider — no live LLM calls in CI - [ ] Eval-harness measurement: extend `tests/llm_eval/` to run before/after on the same fixtures, report the lift quantitatively rather than vibes - [ ] **Sequencing**: this is Wave 1.4 work and can ship before Cybench Phase 3 ([3.3.1](#phase-33--benchmarks-the-credibility-lever)) so the published numbers reflect the prompt-engineered baseline, not the pre-scaffolding baseline. Otherwise we publish a lower number now and a "better" one later, which looks like backfilling ### Phase 1.5 — README Honesty Pass (see full [README Rewrite](#readme-rewrite) section below) --- ## Wave 2 — Integrations & Workflows (6–8 weeks) > The real pentester's toolbox, wired into the swarm, with named pipelines. ### Phase 2.1 — Core Tool Integrations - [x] **2.1.1** `nmap` adapter `internal/tools/nmap.go` - [x] XML output parser → findings (`PORT_OPEN`, service/version, OS match) - [x] Scope guard in `Run()`; timing flag configurable - [x] Requires `nmap` binary; gated via `IsAvailable()` so missing binary = skip - [x] **2.1.2** `sqlmap` adapter via `sqlmapapi` REST - [x] Full task/new → option/set → scan/start → poll status → data → task/delete lifecycle - [x] Credentials redacted via key-name regex + inline `key=value` redaction before results surface - [x] Defence-in-depth: scope.ValidateAndLog on every call; deferred task delete runs even on timeout - [x] 3 unit tests use httptest-backed fake sqlmapapi so CI needs no sqlmap binary - [ ] Wiring: trigger on classifier `POTENTIAL_SQLI` findings (follow-up — needs a swarm agent adapter) - [x] **2.1.3** `ffuf` adapter `internal/tools/ffuf.go` — FUZZ URL + wordlist, JSON parse via temp file, scope-guarded - [ ] Wordlist registry (SecLists auto-download on first run, cached) *(follow-up)* - [x] **2.1.4** `gobuster` adapter — dir + dns modes, text-line parse - [x] **2.1.5** `trufflehog` adapter — NDJSON stream, `Raw` / `RawV2` secret bodies redacted at ingest - [x] **2.1.6** `gitleaks` adapter — uses `--redact`, additionally scrubs `Secret` field defence-in-depth; exit-1-on-leaks handled - [x] **2.1.7** `semgrep` adapter — p/owasp-top-ten default rule pack, JSON parse, exit-1-on-findings handled - [x] **2.1.8** `amass` adapter — passive by default, active via opt-in flag, NDJSON parse - [x] **2.1.9** `dalfox` adapter — `internal/tools/dalfox.go` wraps hahwul/dalfox in `url ` mode. Supports timeout / workers / deep_dom_xss / skip_bav / blind / cookie / headers / custom_payload options. JSON-array output parsed into ParsedFindings; scope.ValidateAndLog before subprocess spawn. 6 unit tests, no external binary needed in CI. Plan item shipped via PR #8 with Co-Authored-By Bhushan. - [x] **2.1.10** `testssl` adapter — `internal/tools/testssl.go` wraps drwetter/testssl.sh for deep TLS posture audit (weak ciphers, BEAST/CRIME/POODLE/ROBOT/Heartbleed, weak cert chains, expired certs, weak DH). Accepts either `testssl` or `testssl.sh` on PATH (Debian/Homebrew name vs upstream). `--jsonfile` to temp file (stdout would interleave progress). Options: `severity` floor (default `LOW`), `mode` (`fast`/`quick` or full), `starttls` (smtp/imap/pop3/ldap/ftp/mysql/pgsql). Parser drops OK/INFO entries to keep findings signal-heavy. Registered in `NewCoordinator()`; installed via `apt-get install testssl.sh` in Dockerfile. 4 unit tests. - [x] **2.1.11** `jwt_tool` adapter — `internal/tools/jwt_tool.go` wraps ticarpi's jwt_tool. Default mode `-M at` (all tests); options expose `token`, `mode`, `wordlist` (HMAC crack), `pubkey` (asymmetric attacks), `cookie`, `header`. `parseJWTToolOutput` scans verbose plaintext for `(VULN)` / `[+]` markers and emits structured findings with `severity=high`. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.12** `kube_hunter` adapter — `internal/tools/kube_hunter.go` wraps aquasecurity/kube-hunter for Kubernetes misconfig scanning. Default `remote` mode (--remote ); also supports `cidr`, `interface` (in-pod), `active` (includes exploit attempts — only use with safe-mode off). `--report json -o `; parser walks the `vulnerabilities` array and emits one finding per entry with vid/category/severity/location/evidence/hunter. Severity passes through from kube-hunter directly. Binary on PATH uses hyphen (`kube-hunter`); adapter name uses underscore to match the plan id. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.13** `arjun` adapter — `internal/tools/arjun.go` wraps s0md3v/Arjun for HTTP parameter discovery. Writes JSON via `-oJ` to a temp file (stdout interleaves progress). Options: `method`, `threads`, `stable` (passive), `wordlist`, `cookies`, `headers`, `delay`. `parseArjunJSON` flattens `{url: {params: [...]}}` into one finding per discovered parameter. Registered in `NewCoordinator()`. 5 unit tests. - [x] **2.1.14** `crlfuzz` adapter — `internal/tools/crlfuzz.go` wraps dwisiswant0/crlfuzz for CRLF-injection fuzzing (CR/LF + Unicode-encoded variants in path/query → detect header echo, Set-Cookie reflection, response splitting). Writes matches to a temp file via `-o` (stdout interleaves). Options: `concurrent`, `method`, `cookies`, `headers`, `user_agent`, `proxy`. Each matched URL becomes one HIGH-severity finding categorised as `crlf_injection`. Registered in `NewCoordinator()`; installed in Dockerfile via `go install github.com/dwisiswant0/crlfuzz/cmd/crlfuzz@latest`. 4 unit tests. - [x] **2.1.15** `nikto` adapter — `internal/tools/nikto.go` wraps sullo/nikto for classic web server misconfiguration scanning (dangerous files, outdated server software, missing security headers, default paths). `-Format json -o ` (stdout interleaves progress); nikto predates severity scoring so every finding surfaces at `low` — the classifier promotes after correlating context. Options: `ssl`, `tuning`, `user_agent`, `max_time`. Registered in `NewCoordinator()`; installed via `apt-get install nikto` in Dockerfile. 5 unit tests. - [x] **2.1.16** `wpscan` adapter — `internal/tools/wpscan.go` wraps wpscanteam/wpscan for WordPress-specific vulnerability surface (core CVEs, vulnerable themes/plugins, exposed users, wp-config backups). `--format json --output `; parser walks core/plugin/theme vulnerability slots and emits one HIGH finding per CVE, plus low/high disclosure findings for users + config backups. Options: `api_token` (wpscan.com — required for CVE data), `enumerate` (default `vp,vt,u`), `stealthy`, `user_agent`, `disable_tls`. Gating note: agent should fingerprint as WordPress before invoking. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.17** `droopescan` adapter — `internal/tools/droopescan.go` wraps SamJoan/droopescan for non-WordPress CMS enumeration (Drupal default; Joomla/SilverStripe/Moodle via `cms` option). Different scope from wpscan: enumerates (version, plugins, themes, sensitive files) but doesn't ship a CVE catalogue — the classifier agent correlates discovered versions/plugins against NVD as a follow-up. `--output json`; emits an `info`-severity version finding + low-severity findings per plugin/theme/interesting-url. Options: `cms`, `threads`, `enumerate` (default `a` = all), `user_agent`. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.18** `crackmapexec` adapter — `internal/tools/crackmapexec.go` wraps crackmapexec / its maintained successor NetExec (`nxc`) for multi-protocol internal enumeration (SMB/LDAP/MSSQL/SSH/WinRM/RDP/FTP). Binary resolution prefers `crackmapexec`, falls back to `nxc`/`netexec`. No first-class JSON export, so `parseCrackMapExecText` parses the line-oriented ` [marker] ` stdout: `[+]` → valid_credentials (HIGH if `(Pwn3d!)` → admin_access), plus `signing:False` → smb_signing_disabled and `SMBv1:True` → smbv1_enabled. Options: `protocol`, `username`, `password`, `hash`, `domain`, `local_auth`, `shares`, `users`, `sessions`, `extra`. Internal-scope only; scope-gated. Registered in `NewCoordinator()`; NetExec installed via venv pip. 5 unit tests. - [x] **2.1.19** `bloodhound` + `SharpHound` adapter — `internal/tools/bloodhound.go` drives the Linux `bloodhound-python` collector (SharpHound is the Windows-native equivalent; both emit the same JSON object schema so one parser serves either). Full graph analysis lives in neo4j; the adapter surfaces the high-signal facts readable straight from object `Properties`: AS-REP roastable users (`dontreqpreauth`) and Kerberoastable users (`hasspn`) from `*_users.json`, unconstrained-delegation computers (`unconstraineddelegation`) from `*_computers.json`, plus summary counts. Options: `username`, `password`, `hash`, `dc`, `nameserver`, `collect`, `ldaps`. Internal-scope only; scope-gated. Registered in `NewCoordinator()`; `bloodhound-python` installed via venv pip. Powers [5.15.8](#phase-515--network-services-p3--for-internal-scope-programs). 5 unit tests. - [x] **2.1.20** `ysoserial` adapter — `internal/tools/ysoserial.go` wraps frohoff/ysoserial as a payload generator (not a scanner). `target` is the OS command to execute on the deserializing host; options pick `gadget` (default `CommonsCollections5`) and `encoding` (`raw` / `base64` / `hex` for JSON round-trip survival). Returns the serialized payload as a single ParsedFinding with `tool`, `gadget`, `command`, `encoding`, `payload`, `size`. Registered in `NewCoordinator()`. `ysoserial.net` adapter still pending — file a separate issue when needed. 2 unit tests. - [x] **2.1.21** `pacu` adapter — `internal/tools/pacu.go` wraps RhinoSecurityLabs/pacu in non-interactive mode (`--module-name --exec`). Pacu modules are too output-varied for per-module structured parsing; adapter surfaces the raw module output as a single coarse-grained `aws_enumeration` finding the LLM reasons over directly. Options: `session` (default `pentestswarm`), `module` (REQUIRED, e.g. `iam__privesc_scan`), `module_args`, `region`. Returns an actionable error when `module` is unset. Registered in `NewCoordinator()`. 3 unit tests. - [x] **2.1.22** `prowler` adapter — `internal/tools/prowler.go` wraps prowler-cloud/prowler v3+ with `--output-formats json-ocsf --output-directory `. Parser walks the OCSF detection_finding events, filters to FAIL entries (status_code 2 / status_detail "FAIL"), and emits one finding per failure with check_id, severity, message, first resource UID/region/type, and remediation hint. Soft-handles prowler's non-zero exit on findings (signal-not-error). Options: `provider` (aws|gcp|azure|kubernetes), `profile`, `region`, `severity_min`, `checks`, `services`, `compliance`. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.23** `scoutsuite` adapter — `internal/tools/scoutsuite.go` wraps nccgroup/ScoutSuite for multi-cloud posture review (aws|azure|gcp|aliyun|oci|kubernetes). scout emits its report as a `.js` file (`scoutsuite_results = {...};`); parser strips the JS-assignment wrapper, decodes the JSON, walks per-service `findings` map, and emits one finding per non-empty check with severity `high` for `danger` / `medium` for `warning`. Empty `flagged_items` checks filtered. Options: `provider`, `profile`, `regions`, `services`. Binary on PATH is `scout`, not `scoutsuite`. Registered in `NewCoordinator()`. 4 unit tests. - [x] **2.1.24** `cloudsplaining` adapter — `internal/tools/cloudsplaining.go` wraps salesforce/cloudsplaining for AWS IAM policy analysis. Two-step usage (download + scan) collapsed into one Run by accepting `target` as either an existing policy file path OR the literal `"download"` to do both. Parser walks per-policy results and emits one finding per non-empty risk category: `privilege_escalation` / `data_exfiltration` / `resource_exposure` (high), `credentials_exposure` (critical), `infrastructure_modification` (medium). Options: `profile`, `include_aws_managed`. Registered in `NewCoordinator()`. 5 unit tests. - [x] **2.1.25** `checkov` adapter — `internal/tools/checkov.go` wraps bridgecrewio/checkov for IaC scanning across Terraform, CloudFormation, Kubernetes, Helm, Dockerfile, Serverless, ARM templates. Uses `-d -o json --soft-fail` (soft-fail prevents the swarm chain from aborting on findings). Parser handles both single-framework (top-level object) and multi-framework (top-level array) shapes; each failed_check becomes one finding tagged with framework/check_id/file_path/line_range/resource. Severity falls back to "medium" when checkov omits it. Options: `framework`, `skip_check`, `soft_fail`. Registered in `NewCoordinator()`. `tfsec` adapter (separate codebase, lighter Terraform-only scanner) deferred to a follow-up issue. 4 unit tests. - [x] **2.1.26** `interactsh-client` adapter — `internal/tools/interactsh.go` wraps ProjectDiscovery's OOB callback client. Time-window oriented: `Run` listens for `timeout` seconds (default 30), captures the assigned OOB URL from stderr banner and any received DNS/HTTP/SMTP/LDAP interactions from stdout JSONL, then SIGTERMs the child cleanly (deadline-exceeded exit is the normal termination path, not an error). First finding is a `type=payload` with the assigned URL + usage instructions for the LLM; subsequent findings are received interactions with protocol/remote_address/raw_request. Options: `timeout`, `server` (custom OOB server), `token` (auth for private servers). Registered in `NewCoordinator()`; installed in Dockerfile. 6 unit tests. - [x] **2.1.27** `gxss` adapter — `internal/tools/gxss.go` wraps KathanP19/Gxss as a fast reflected-XSS pre-screener. Takes a URL or a newline-separated list (piped via stdin to match the upstream `cat urls.txt | Gxss` shape), prints URLs where at least one parameter reflects unfiltered. Accepts either `Gxss` (upstream case) or `gxss` (lowercase package-manager variant) on PATH. Each reflected URL becomes one MEDIUM finding with `category=reflected_input` and a note pointing the agent at dalfox for confirmation. Options: `urls` (stdin batch), `threads`, `payload`. Registered in `NewCoordinator()`. `bxss` (blind XSS finder) deferred. 4 unit tests. - [x] **2.1.28** `dotdotpwn` adapter — `internal/tools/dotdotpwn.go` wraps wireghoul/dotdotpwn, a directory-traversal fuzzer (permutes `../`, `..\`, URL/Unicode/double-encoded variants at varying depths, confirms via a keyword match in the response). Perl tool with no structured output, so `parseDotDotPwnText` scans stdout + the `-r` report file for the inline `VULNERABLE` marker and emits one HIGH-severity `path_traversal` finding per confirmed traversal (deduped). Runs non-interactively via `-q` (suppresses the banner + Enter prompt that would hang) and `-C` (continue on error). Options: `module`, `depth`, `file`, `pattern`, `os_type`, `ssl`, `port`, `break_on_first`, `extra`. Registered in `NewCoordinator()`; installed via git clone + Perl CPAN deps in Dockerfile. 5 unit tests. ### Phase 2.2 — Heavyweight Integrations - [x] **2.2.1** **Burp Suite MCP** bridge (official PortSwigger MCP) - [x] JSON-RPC 2.0 HTTP client at `internal/integrations/burp/client.go` with bearer auth - [x] Burp tool constants + helpers: `StartActiveScan`, `GetIssues`, `ListTools`, `Ping` - [x] Swarm agent `agents.BurpAgent` triggers on `HTTP_ENDPOINT` findings above 0.5 pheromone, publishes Burp issues as `CVE_MATCH` - [x] 5 unit tests use httptest-backed JSON-RPC server, so CI runs with no Burp install - [x] **2.2.2** **Metasploit** via msfrpcd - [x] HTTP/JSON client at `internal/integrations/metasploit/client.go` (msgpack was too opaque; msfrpcd supports JSON fine) - [x] `auth.login` token cached, transparent refresh on 401 - [x] `module.execute`, `session.list`, `session.stop`, `job.stop` primitives - [x] 5 unit tests use httptest-backed fake msfrpcd; covers token refresh, session lifecycle - [ ] Swarm agent that registers every session with the cleanup registry (follow-up, needs an exploit-agent extension) - [x] **2.2.3** **OWASP ZAP** REST API - [x] Client at `internal/integrations/zap/client.go` with API-key query param - [x] Spider + active-scan primitives, status polling, alerts endpoint - [x] 5 unit tests (spider + active lifecycle, alert parsing, API-key enforcement, URL encoding regression) - [x] **2.2.4** **Nuclei template author agent** - [x] `NucleiAuthorAgent` triggers on high-pheromone novel findings (no CVE match), uses structured tool-use (`emit_nuclei_template`) so output is always parseable - [x] Drafts land at `./drafts/nuclei/-.yaml` for human review - [x] Optional `-validate` pass via the nuclei binary when present; rejects ill-formed drafts before publishing - [x] `NUCLEI_TEMPLATE_DRAFT` finding type added to blackboard + tuning (12h half-life for reviewer workday) ### Phase 2.3 — Swarm Playbooks (Named Pipelines) Ship as YAML files in `playbooks/` — like Nuclei templates but for full swarm behaviors. - [x] **2.3.1** `playbooks/bug-bounty.yaml` - [x] **2.3.2** `playbooks/external-asm.yaml` - [x] **2.3.3** `playbooks/ci-cd-security.yaml` - [x] **2.3.4** `playbooks/internal-network.yaml` - [x] **2.3.5** `playbooks/ctf-solver.yaml` - [x] **2.3.6** Playbook schema + validator (`internal/plugins/validate.go`) - Name + version-semver check, duplicate-phase detection, variable-type whitelist, tool-known check, `{{ var }}` reference to undeclared variables fails, `required + default` combo warns - CLI `pentestswarm playbook validate ` now reports full error/warning lists with exit code on errors - 9 unit tests cover each rule - [x] **2.3.7** `pentestswarm playbook run ` CLI wiring *(already in `cli/playbook.go`)* - [ ] **2.3.8** "Playbook marketplace" page on site (v1: just a listing; v2: submit PRs) ### Phase 2.4 — CI/CD & Ecosystem - [x] **2.4.1** GitHub Action (composite action in `deploy/github-action/action.yml`) - [ ] Publish to GitHub Marketplace *(external — tag + GH submit)* - [ ] SARIF output integrates with Code Scanning *(wired in action, emitter still pending)* - [x] Fail-PR-on-critical flag (`fail-on` input) - [x] **2.4.2** Jira adapter — `internal/integrations/jira/client.go`; severity → Jira priority map (Highest…Lowest), Basic auth or Bearer, labels include attack-category - [x] **2.4.3** Slack adapter — `internal/integrations/slack/client.go`; both incoming-webhook and `chat.postMessage` (Bot token), thread-per-campaign tracking, `PostFinding` + `PostEvent` surface - [x] **2.4.4** SARIF 2.1.0 export (covers the SARIF slice of SIEM); CEF + STIX deferred as separate PRs since each needs schema work - [x] **2.4.5** Webhook dispatcher — HMAC-SHA256 signing, exponential-backoff retry, DLQ channel, permanent-error classification for 4xx (no wasted retries), exported `Verify()` helper for receivers - [x] **2.4.6** Censys API client — `internal/osint/censys.go` queries the Search v2 `dns.names:` endpoint with HTTP Basic (`api_id:api_secret` per the v2 spec). Returns normalized Assets: 1 host per hit + 1 service per discovered port + dedup'd subdomains. Env: `PENTESTSWARM_CENSYS_API_ID` / `PENTESTSWARM_CENSYS_API_SECRET`. Degrades gracefully (IsAvailable=false) when either half is missing. 5 unit tests. - [x] **2.4.7** Shodan API client — `internal/osint/shodan.go` runs a `hostname:` search via `/shodan/host/search`. Returns 1 "service" asset per match (IP:port + product/version metadata) and dedup'd "subdomain" assets across hostnames. Env: `PENTESTSWARM_SHODAN_API_KEY`. 6 unit tests. - [x] **2.4.8** GitHub code search client — `internal/osint/github_search.go` runs a small built-in probe set (`"{{target}}" api_key`, `secret`, `password`, `BEGIN RSA PRIVATE KEY`) against `/search/code`. Each hit becomes a "secret" Asset with the HTML URL as the click-through value and repo/path/matched-query in metadata. Partial-failure tolerant (per-query rate limits don't black-hole the rest). Env: `PENTESTSWARM_GITHUB_SEARCH_TOKEN` (any PAT, no scopes required for public code search). 6 unit tests. - [x] **2.4.9** GitLab code search client — `internal/osint/gitlab_search.go` parallels the GitHub variant (shares the same `defaultGitHubQueries` probe set so leak coverage is symmetric across the two ecosystems). Queries `/search?scope=blobs` with the `PRIVATE-TOKEN` header; resolves each hit's `project_id` via `/projects/` (cached per Lookup) to compose a clickable URL of the form `/-/blob//#L`. Falls back to a project-id URL if the project lookup fails so leak findings aren't dropped. Endpoint defaults to `gitlab.com/api/v4` but is overridable for self-hosted GitLab instances. Bitbucket variant deferred to a follow-up issue. 7 unit tests. --- ## Wave 3 — Research Frontier (ongoing) > Pick 2–3, don't chase all. Each item has a research lineage — see references. ### Phase 3.1 — RAG / Experience Memory - [ ] **3.1.1** CVE corpus ingestion - [ ] Nightly NVD dump → pgvector with CVSS + CWE metadata - [ ] Indexed by vuln type and affected-product - [ ] **3.1.2** Nuclei template corpus — embed all templates, retrievable by finding signature - [ ] **3.1.3** ExploitDB corpus - [ ] **3.1.4** Experience Memory (pattern from AutoAttacker) - [ ] Redacted trace of each campaign stored at finish - [ ] Retrieval on new campaign: "similar attack surfaces → what worked" - [ ] Opt-in telemetry (never on by default) → shared intelligence network - [ ] **3.1.5** Agent-specific graph memory (arXiv:2511.07800 approach) - [ ] Exploit agent learns chain success/failure edges - [ ] Encoded as trainable graph, beats flat vector for multi-step reasoning ### Phase 3.2 — Fine-tuned Pentest-Swarm Model Reproduce Pentest-R1 (arXiv:2508.07382) on Qwen3 or Llama 3.3 base. - [ ] **3.2.1** Training data pipeline (`training/`) - [ ] Phase A: offline walkthroughs (HackTricks, HTB writeups, PortSwigger Academy solutions) - [ ] Phase B: online RL traces from CTF solving - [ ] **3.2.2** Fine-tune Qwen3-32B-Instruct with LoRA - [ ] **3.2.3** GGUF quantized release (`Pentest_LLM.gguf` already exists — v2 supersedes) - [ ] **3.2.4** HuggingFace Hub publishing - [ ] **3.2.5** Ollama modelfile for one-command local install - [ ] **3.2.6** Distillation target: 7B model that runs on 16GB VRAM laptops ### Phase 3.3 — Benchmarks (the credibility lever) Publish numbers in README. Update on every release. This is how XBOW built its brand. - [x] **3.3.1** Cybench runner scaffold at `tests/bench/cybench/`: `Challenge` / `Result` types, `Runner` interface, `Score()` sum function, `LoadChallenges()` fails loudly until fixtures vendored. Pin shape so the harness can plug in a Docker sandbox + Cybench fixture loader without re-architecting. - [ ] Vendor Cybench fixtures into `tests/bench/cybench/fixtures/` (gitignored) - [ ] Implement Docker-sandboxed Runner (per-challenge container, mount the swarm binary). Hard per-challenge `--budget` cap (≤ $5) to prevent runaway spend - [ ] CI job to run against a small subset on every PR - [ ] **Multi-backend benchmark run** — publish a 2- to 3-column results table: Claude Sonnet 4.6 (headline, max quality), Together AI cheap tier via [1.4.6](#phase-14--llm-layer-upgrades) (cost differentiation), Ollama-Llama-3.3 if competitive (fully-local story). No single competitor publishes all three — this is the differentiator - [ ] First-pass workflow: Phase 1 debug on Haiku (~$30), Phase 2 5–10 challenges on Sonnet to measure real per-challenge cost (~$40), Phase 3 full 40 on Sonnet (~$120). $200 hard budget cap - [ ] **3.3.2** AutoPenBench runner - [ ] **3.3.3** CVE-Bench runner - [ ] **3.3.4** HackTheBox subset — retired boxes only (legal, reproducible) - [ ] **3.3.5** Results dashboard at `benchmarks.pentestswarm.ai` — live updated - [x] **3.3.6** Competitor table shipped in `README.md` (Comparison section): us vs PentestGPT / HackingBuddyGPT / PentAGI / Shannon / HexStrike / Pentest-R1, columns for architecture, executes-vs-suggests, memory, tools, MCP, swarm-or-pipeline. - [ ] **3.3.7** **XBOW validation-benchmarks runner** (`P2`, future) — harness for the [xbow-engineering/validation-benchmarks](https://github.com/xbow-engineering/validation-benchmarks) suite (~104 Dockerized web-exploit challenges, `XBEN-0XX-24`). Sibling package `tests/bench/xbow/`, mirroring the `tests/bench/cybench/` shape (`Runner` interface, `LoadChallenges()` reading each `benchmark.json`, `Score()`). Per challenge: `make build FLAG=` → `make run` (`docker compose up --wait`) exposes the target; the swarm runs in `--mode ctf` against it; solve = exact-match the injected flag in campaign artifacts; teardown `make stop && make clean`. **Run in the "hint-free, source-aware" (white-box) mode** — feed challenge source to the swarm (semgrep + source path in objective), no vuln hints — the same mode [Shannon](https://github.com/KeygraphHQ/shannon) reported 96.15% (100/104) on; XBOW's own black-box figure is ~85%. **Label the mode exactly on any published number** or it reads as inflated. Extend `Result` with a structured **failure taxonomy** (recon-miss / tool-missing / exploit-stalled / flag-not-extracted / model-refusal / budget-exhausted) so a run yields a prioritized fix-list, not just a score — the goal is *knowing where we stand*, not beating Shannon. Known prerequisite for a competitive number: the browser-driven exploit-and-verify loop (our capability gap vs. Shannon). Reuse the [3.3.1](#phase-33--benchmarks-the-credibility-lever) staged-budget model. Fixtures gitignored under `tests/bench/xbow/fixtures/`. ### Phase 3.4 — Agent Robustness (the underserved moat) Memory poisoning and inter-agent-comm attacks are real. Be the first tool to market as *hardened*. - [x] **3.4.1** `internal/swarm/provenance` — Ed25519 keypair per agent, `Sign(canonical bytes)` + `Verify(pub, sig, ...)`. Detects payload tamper AND agent-name impersonation (signing under a different keypair fails verification). Tests cover roundtrip, tamper, impersonation, malformed key/sig. Wiring through Board.Write is a follow-up. - [x] **3.4.2** `internal/swarm/blackboard/injection_test.go` — three MINJA tests (type-isolation, pheromone-flood clamp, MinPheromone gate). The pheromone-flood test surfaced a real defense gap: `MemoryBoard.Write` was accepting `PheromoneBase=9999`. Now clamped to [0, 1]. - [x] **3.4.3** `internal/swarm/memorygraft` — `Scan(ctx, board, cfg)` watchdog flags four memory-graft signals: burst writes, repeat-title fingerprints, byte-identical Data payloads, and type-mismatch (an agent emitting a finding under a type owned by another agent). Pure-read; conservative defaults. - [x] **3.4.4** `internal/swarm/ratelimit` — per-agent token-bucket limiter, wired via `swarm.WithAgentRateLimit(name, perSec, burst)`. Defends against pathological self-feeding loops. No external deps. Agents without a configured limit are uncapped (opt-in tightening). - [x] **3.4.5** `docs/security/swarm-hardening.md` — full writeup of the four-layer defense (clamp / provenance / heuristic detector / rate limit) framed as a blog post. Doubles as the script for a BSides / Black Hat Arsenal talk once benchmark numbers land. ### Phase 3.5 — Symbolic Execution Hybrid (stretch) - [ ] **3.5.1** `angr` wrapper for binary targets - [ ] **3.5.2** LLM-guided symex — LLM proposes paths, angr validates - [ ] **3.5.3** Benchmark on canonical CTF binaries --- ## Wave 4 — Researcher Workflow: The XBOW-for-Bug-Bounty Play (6–8 weeks) > **The goal of this wave is the only thing that actually matters for adoption:** a real researcher should be able to `brew install`, point the tool at a HackerOne program they've been accepted to, and walk away with a shortlist of verified, reproducible vulnerabilities they can paste straight into the platform's submission form. > > XBOW built this as a commercial SaaS and drove its brand off a public HackerOne leaderboard. Our job is to match the experience (one command, real bounties) and beat them on friction (zero setup, zero cost floor, local control of data). Every task in this wave is judged against one question: "Does this make it more likely that a researcher reports a real vuln found via this tool in the next 30 days?" > > Everything in earlier waves feeds this. Everything in this wave is user-facing. ### Phase 4.1 — Frictionless Onboarding Zero-to-first-scan under 60 seconds. - [x] **4.1.1** `pentestswarm init` — interactive, one-shot setup - [x] API key stored via `internal/keychain` (go-keyring — macOS Keychain / linux-secret-service / Windows Credential Manager) - [x] Tool probe via shared `internal/toolprobe` — same data powers `doctor` - [x] Writes `~/.pentestswarm/config.yaml` WITHOUT the API key (keychain owns the secret; config file is safe to commit) - [x] **4.1.2** First-run bootstrap — `scan` prompts once instead of failing when TTY is attached; offers to stash the key in the keychain - [ ] **4.1.3** One-line installer: `curl -sSL https://install.pentestswarm.ai | sh` *(external: needs `install.pentestswarm.ai` hosted)* - [x] **4.1.4** [`Armur-Ai/homebrew-tap`](https://github.com/Armur-Ai/homebrew-tap) created and live. `Formula/pentestswarm.rb` is a build-from-source formula pinned to upstream commit `9d2bbc2` with `head` block for `--HEAD` users. Audit passes, `brew install --dry-run Armur-Ai/tap/pentestswarm` resolves cleanly. Will be auto-rewritten to use binary releases by upstream's release workflow once v0.1.0 is tagged. - [ ] **4.1.5** "Zero to First Finding in 60 seconds" landing page *(external: needs site)* - [x] **4.1.6** Three-stage `Dockerfile` ships a 779MB image with all 16 security tools pre-installed (subfinder, dnsx, httpx, naabu, katana, gau, nuclei, ffuf, gowitness, amass via Go install; nmap, sqlmap, gobuster, chromium via apt; trufflehog + gitleaks pinned binaries; semgrep via venv). `GOTOOLCHAIN=auto` insulates the build from upstream Go-version bumps. `.github/workflows/docker.yml` builds linux/amd64 + linux/arm64 and publishes to `ghcr.io/armur-ai/pentestswarm` (`:edge` on main push, `:latest`+`:vX.Y.Z` on tags). `docker run pentest-swarm-ai doctor` shows 16/16 tools green. - [x] **4.1.7** `pentestswarm doctor --fix` auto-installs Go-installable tools; prints copy-paste commands for brew/apt tools (safer than running package managers for the user) - [ ] **4.1.8** `pentestswarm tutorial` (`P1`) — interactive walkthrough against a bundled sidecar Docker target (OWASP Juice Shop). Researcher sees a real scan end-to-end on their first run, no API key required for the demo path (uses local Ollama as fallback) - [ ] **4.1.9** `pentestswarm scan --demo` (`P2`) — runs against an embedded Juice Shop / DVWA / WebGoat container so first-time users see what success looks like before pointing at a real target ### Phase 4.2 — HackerOne + Bugcrowd Scope Auto-Import No one should have to hand-type a 200-asset scope list. - [x] **4.2.1** `pentestswarm scope import h1 ` — pulls structured_scopes from HackerOne's v1 API; URL/WILDCARD/DOMAIN → domains, CIDR/IP → CIDRs; Basic auth for private programs - [x] **4.2.2** `pentestswarm scope import bugcrowd ` — v4 engagements/targets endpoint with Token auth - [x] **4.2.3** `pentestswarm scope import intigriti ` — external researcher API with Bearer auth - [ ] **4.2.4** `pentestswarm scope import synack ` *(deferred: Synack's API is gated; revisit when a researcher with access can test)* - [ ] **4.2.5** Credential-per-program config *(partial: tokens stored in keychain via init; per-program override not yet wired)* - [x] **4.2.6** `pentestswarm scope diff ` — coloured +/- diff or JSON, non-zero exit when changes found (shell-friendly) - [x] **4.2.7** Scope drift guard — `scope.Watcher` reloads the YAML on a ticker, emits `Diff`s via a channel for the scheduler to warn on / abort in-flight scans ### Phase 4.3 — Verified-PoC Gate (The Signal-vs-Noise Moat) XBOW's reputation rides on low false-positive rate. So does ours. - [x] **4.3.1** `pipeline.Reproduction` struct on ClassifiedFinding: Command / HTTPRequest / ExpectedIndicator / Tools - [x] **4.3.2** `ConfirmationAgent` — re-runs Reproduction (shell command or HTTP request), supersedes with pheromone 0.1 when indicator is absent - [ ] Rotated IP (outbound proxy) per-run *(follow-up: needs a proxy-pool integration)* - [x] **4.3.3** `pipeline.CrossValidate` — 2+ tools on same target+category promote to High confidence; 0/1 tool without reproduction downgrades to Unverified - [x] **4.3.4** FP feedback loop at `~/.pentestswarm/fp-cache.jsonl` — append-only JSONL, wildcards by target/category, auto-suppresses matching findings on future scans - [x] **4.3.5** NVD CVSS sanity-check: `internal/pipeline/nvdcheck` fetches v3.1 base score, disk-cached, flags mismatches > 2.0 CVSS points - [x] **4.3.6** `--publish-unverified` flag — default (0.5 threshold) only publishes confirmed findings; with the flag, threshold drops to 0.1 and suspected findings ship too ### Phase 4.4 — Report & Submission Automation Turn a campaign into a ready-to-paste submission. - [x] **4.4.1** HackerOne template at `internal/agent/report/templates/hackerone.md.tmpl` (Summary · Steps · Impact · Recommendation · Severity · CVEs · Verified-by) - [x] **4.4.2** Bugcrowd template — VRT / Vuln-Details / Steps / PoC / Remediation layout - [x] **4.4.3** Intigriti template — 'Type of weakness' field, PoC above Impact - [x] **4.4.4** `pentestswarm submit` writes per-finding drafts to `./submissions/`; `--live` is gated with a 'not yet implemented' error (can't accidentally post an AI-generated report) - [x] **4.4.5** H1 dedup — `Client.Reports()` + `dedup.FindDuplicates` (Jaccard, target-boost); drafts get a `> ⚠ Possible duplicate of: #…` callout prepended - [x] **4.4.6** `hackerone.Client.PublicReports(slug, limit)` queries the H1 hacktivity feed scoped to one program; `cli/submit.go.loadPriors` now merges both sources (researcher's own reports prefixed `own:`, public disclosed prefixed `public:`) into the dedup pass. Public source works without credentials. Bugcrowd / Intigriti scrapers TBD. - [x] **4.4.7** Quality gate at `internal/agent/report/qualitygate`: LLM rubric on clarity / impact / reproducibility; threshold 6.0, structured tool-use so parse failures are impossible; `submit --quality-gate` wires it - [x] **4.4.8** Evidence capture at `internal/agent/report/evidence`: `.http` files (Burp-importable) + optional gowitness screenshots; missing gowitness = silent skip - [x] **4.4.9** `pentestswarm report polish ` — re-runs the rubric on a hand-edited draft; non-zero exit when below threshold (shell-friendly) - [ ] **4.4.10** Per-finding video walkthrough (`P2`) — auto-generated asciinema or screen-capture of the reproduction step. Massive submission-quality lift on programs that value clear PoCs - [ ] **4.4.11** "Explain this finding" mode (`P1`) — LLM teaches the researcher the vuln class with refs to OWASP / PortSwigger Academy / HackTricks. Educational + builds trust - [ ] **4.4.12** Bounty calculator with program payout history (`P2`) — fetch disclosed reports from H1 hacktivity for the program, compute median payout per severity. Replaces the public-market fallback in [4.5.6](#phase-45--cost--risk-controls) for programs with disclosed history - [ ] **4.4.13** CVSS with environmental score (`P2`) — beyond base; incorporates business-impact context from the affected feature - [ ] **4.4.14** Markdown → PDF report rendering (`P3`) — some programs prefer PDF - [ ] **4.4.15** Re-test workflow (`P1`) — after vendor claims fix, `pentestswarm retest ` re-runs the reproduction and updates the report - [ ] **4.4.16** Disclosure timeline tracker (`P2`) — 90-day clock for coordinated disclosure; nudges the researcher - [ ] **4.4.17** CVE coordination helper (`P3`) — MITRE CVE assignment workflow for novel findings ### Phase 4.5 — Cost + Risk Controls - [x] **4.5.1** `--estimate` flag at `cli/scan.go`: short-circuits before config validation, prints USD range (`Pricing.EstimateUSD(class)` heuristic buckets), exits without touching network. Works pre-init (no API key required). - [x] **4.5.2** Live cost meter: `internal/llm/meter.go` (`Meter` + `meteredProvider` decorator wraps `Provider.Complete` to record `Usage`); `internal/engine/swarm_runner.go` ticks every 15s emitting a structured cost event; final spend prints on campaign end. - [x] **4.5.3** Pause-on-budget: scheduler writes CAMPAIGN_COMPLETE to the blackboard on budget exhaustion instead of cancelling, so the report agent renders a partial report from whatever was found before the cap. Researcher can extend the budget + re-run for more. - [x] **4.5.4** `--safe-mode` flag wired through `cli/scan.go` → `engine.CampaignConfig.SafeMode` → `exploit.Executor.WithSafeMode(true)`. Rejects destructive tokens (rm/kill/chmod/drop/truncate/...) at parse time with an actionable error; checks each word in quoted args (`'drop table users'` is caught). - [x] **4.5.5** `internal/scope/programterms` heuristic parser pulls rate limits, banned techniques (brute-force / DoS / social / physical), required headers, and disallowed paths out of policy prose. `pentestswarm program inspect h1:` fetches the policy via the H1 API and prints them; `--yaml` emits a config fragment for shell pipelines. - [x] **4.5.6** `internal/agent/report/bounty` — `Estimate(finding, programStats)` returns a USD range, preferring per-program stats when supplied and falling back to a conservative public-market table. `Total()` sums across findings. - [x] **4.5.7** `internal/agent/report/roi` — `Calculate(spend, findings, stats)` returns a Result with green/yellow/red Verdict (>10× / 2–10× / <2× ratio). `Result.Footer()` is appended to `PentestReport.ROIFooter` by the swarm report agent and rendered at the bottom of every campaign report. ### Phase 4.6 — Community Gravity (The Compounding Flywheel) Once the tool finds real bugs, word-of-mouth does more than any marketing. - [ ] **4.6.1** Public findings gallery on `pentestswarm.ai` — researchers opt-in to publish a redacted writeup once the program closes the report - [ ] **4.6.2** Bounty leaderboard — total $ earned with the tool, self-reported with an H1 URL as proof (manual verification for now; automated via H1 webhooks later) - [ ] **4.6.3** Template marketplace — share + install playbooks and nuclei templates via `pentestswarm market install `. Signed + reviewed before merging into the main `armur-templates` repo - [x] **4.6.4** `--assist` flag wired through `engine.WithAssistConfirmer` → `exploit.Executor.WithConfirm`. CLI prompts y/N/a (approve-all-remaining) before every executed step; bare-enter skips (fail-closed); EOF on stdin aborts the campaign. Skipped steps surface as `[SKIPPED by assist mode]` results instead of failures. - [ ] **4.6.5** Weekly office-hours Discord event — researchers bring their scans, we review live. Pure community signal, no marketing overlay - [x] **4.6.6** `internal/pipeline/fpcache.Anonymize` + `Store.ExportShare` strip target hostnames + reason free-text and SHA-256 hash a normalized title-token-set so equivalent FPs across researchers collide. `pentestswarm fp share` writes the anonymised payload to `~/.pentestswarm/fp-share.jsonl` for manual review before upload. - [x] **4.6.7** `internal/plugins.Overlay(base, overlay)` merges per-program tuning on top of a base playbook with structured rules: scalars overlay-wins, tags union-deduped, variables per-key, phases replace-by-name (else append), includes union-deduped. Example overlay shipped at `playbooks/programs/example.yaml`. ### Phase 4.7 — Benchmarks That Matter to Researchers Not Cybench. The numbers bug-hunters actually care about. - [ ] **4.7.1** Public scoreboard tracking, across opt-in users: total vulns filed, total vulns triaged, total $ earned, active researchers - [ ] **4.7.2** Case-study page: "It found this" — before/after for public programs and retired HTB boxes, with full reproduction chains - [ ] **4.7.3** Triage-rate KPI: target >60% of auto-filed reports triaged as Valid within 3 months of the tool going public - [ ] **4.7.4** Time-to-first-finding KPI: median time from `pentestswarm init` to first verified finding — target under 15 minutes on a HackerOne program - [ ] **4.7.5** False-positive rate KPI: target <20% over a rolling 30-day window across all opt-in runs ### Phase 4.8 — Simplicity Guards (enforced, not aspired to) - [ ] **4.8.1** "One-command" invariant: `scan` is the only command a researcher needs. Everything else is optional polish. Any PR that adds a required step to the scan flow gets rejected. - [x] **4.8.2** Root `--help` lands at exactly 30 lines after hiding `mcp`/`config`/`explain`/`ctf` (cobra `Hidden: true` — still functional, just out of the index). `pentestswarm --help` works as before. - [x] **4.8.3** Config-load + campaign-failure errors in `cli/scan.go` end with a next-step (`run pentestswarm init`, `re-run with --strict`). Audit + completion across the rest of the CLI is a follow-up. - [x] **4.8.4** `internal/engine/zero_postgres_test.go` pins the smoke-test invariant: `NewRunner(emptyConfig)` constructs cleanly with the in-memory cleanup registry; the swarm path uses an in-memory blackboard by default. - [x] **4.8.5** `cli/scan.go`: scan without `--scope` defaults to scanning the target itself, with a `[scope]` notice. Single-target scope is conservative — it can't accidentally reach a sibling domain. --- ## Wave 5 — Vulnerability Class Coverage (~6 months) > The depth play. From "finds the easy bugs" to "finds the bugs only this tool would catch." Each phase specializes in a vulnerability class with dedicated agents, payloads, and tool wiring. Coverage target: match or exceed Shannon / Strix / PentAGI. > > **Priority tags**: `P0` = ship first (highest bug-bounty ROI), `P1` = ship second tier (high value), `P2` = niche but valuable, `P3` = community-contribution candidates. > > **Out of scope by explicit decision**: AI/ML system security (LLM prompt injection in apps, model extraction, vector-DB leakage, AI safety-filter bypass) — we use LLMs internally; we don't test them as targets. ### Phase 5.1 — Auth & Session Specialization (`P0`) The highest-payout class in bug bounty. Most real money flows through account takeover chains and IDOR. - [ ] **5.1.1** Multi-user auth-state harness — sessions A + B + admin held concurrently in the blackboard - [ ] **5.1.2** JWT vulnerability scanner — algorithm confusion, `none`-alg, weak secret, `kid` injection, JKU/JWK URL abuse - [ ] **5.1.3** OAuth / OIDC misconfig — `state` param, `redirect_uri` bypass, PKCE downgrade, implicit-flow drift - [ ] **5.1.4** Account-takeover chain detector — password reset, email change, SSO bind, 2FA reset paths - [ ] **5.1.5** 2FA bypass patterns + WebAuthn / FIDO2 implementation flaws - [ ] **5.1.6** SAML attacks — XML signature wrapping (XSW), comment-in-NameID, signature stripping - [ ] **5.1.7** Session fixation, session puzzling, session-cookie scope confusion - [ ] **5.1.8** Password-reset poisoning (Host header trick); magic-link / passwordless flow bugs - [ ] **5.1.9** Username / email enumeration via response timing + content diff - [ ] **5.1.10** Credential-stuffing posture check (rate-limit on auth endpoints) — read-only, never actually stuffs ### Phase 5.2 — IDOR & BOLA Orchestration (`P0`) OWASP API #1. Easy money when present. - [ ] **5.2.1** Two- and three-account orchestrator (A vs B vs admin in parallel) - [ ] **5.2.2** Numeric / sequential ID enumeration with auth state - [ ] **5.2.3** UUID v1 timestamp inference; UUID v4 entropy assessment - [ ] **5.2.4** GraphQL BOLA via introspection - [ ] **5.2.5** Indirect IDOR (user A modifies user B's resource via shared `order#` etc.) - [ ] **5.2.6** Tenant / org parameter swap auth bypass - [ ] **5.2.7** IDOR confidence scoring — FP-prone class, needs careful gating ### Phase 5.3 — Server-Side Injection (`P0`) - [ ] **5.3.1** SSRF with out-of-band detection (depends on [5.8](#phase-58--oob-infrastructure-p0)) - [ ] **5.3.2** SSTI multi-engine payload library — Jinja2, Twig, ERB, Velocity, Freemarker, Handlebars, Mustache - [ ] **5.3.3** XXE with OOB exfiltration + parameter entities - [ ] **5.3.4** NoSQL injection — MongoDB operator injection, Redis CONFIG via SSRF chain - [ ] **5.3.5** Command injection through HTTP params, blind via OOB - [ ] **5.3.6** LDAP injection - [ ] **5.3.7** XPath injection - [ ] **5.3.8** ORM injection (HQL, JPQL) - [ ] **5.3.9** ESI (Edge-Side Includes) injection — Akamai / Varnish / Fastly surfaces - [ ] **5.3.10** Server-side prototype pollution (Node.js apps) - [ ] **5.3.11** CSV formula injection (`=cmd|...` in Excel-opened CSV exports) - [ ] **5.3.12** LaTeX / Markdown injection chains ### Phase 5.4 — Client-Side & Browser-Required (`P1` — depends on [6.1](#phase-61--browser-driven-testing-p0)) - [ ] **5.4.1** DOM XSS scanner via browser instrumentation (source/sink tracking) - [ ] **5.4.2** Client-side prototype pollution → gadget chain to XSS / RCE - [ ] **5.4.3** `postMessage` misuse detection - [ ] **5.4.4** Service Worker hijacking - [ ] **5.4.5** CORS misconfig (advanced — credentialed origin reflection, null origin, regex bypass) - [ ] **5.4.6** Open-redirect chains (auth-bypass payload routing) - [ ] **5.4.7** Clickjacking + frame-ancestor analysis; tabnabbing (`rel=noopener` missing) - [ ] **5.4.8** Mixed-content downgrade; Subresource Integrity gaps - [ ] **5.4.9** localStorage / sessionStorage abuse → XSS-to-ATO chains - [ ] **5.4.10** WebSocket origin confusion + auth-state inheritance ### Phase 5.5 — API & GraphQL Security (`P1`) Under-tested attack surface across most programs. - [ ] **5.5.1** OpenAPI / Swagger spec ingestion → endpoint corpus - [ ] **5.5.2** GraphQL introspection + schema fuzzing - [ ] **5.5.3** GraphQL depth / complexity DoS - [ ] **5.5.4** GraphQL batching attack + alias DoS - [ ] **5.5.5** GraphQL field-suggestion leak ("did you mean…") - [ ] **5.5.6** GraphQL persisted-query bypass - [ ] **5.5.7** REST mass assignment (extra fields in JSON body) - [ ] **5.5.8** API rate-limit bypass patterns (X-Forwarded-For, header case, version path) - [ ] **5.5.9** Server-Sent Events (SSE) auth bypass - [ ] **5.5.10** gRPC reflection enumeration + protobuf fuzzing - [ ] **5.5.11** JSON-RPC method enumeration - [ ] **5.5.12** AsyncAPI / event-driven API testing - [ ] **5.5.13** HATEOAS metadata over-disclosure ### Phase 5.6 — Cloud & Infrastructure (`P0`) Subdomain takeover and exposed cloud creds are the easiest high-rate finds in modern programs. - [ ] **5.6.1** Subdomain takeover detector (dangling CNAME → 40+ known services) - [ ] **5.6.2** S3 / GCS / Azure Blob bucket enumeration + permission probing - [ ] **5.6.3** AWS IAM privilege-escalation path finder (Pacu-style) - [ ] **5.6.4** AWS-specific: Cognito misconfig, API Gateway custom-authorizer bypass, Lambda Function URL exposure, SSM Parameter Store leak, exposed snapshots (EBS / RDS) - [ ] **5.6.5** GCP-specific: Cloud Run public exposure, GKE workload-identity misconfig, IAM `allUsers` bindings - [ ] **5.6.6** Azure-specific: AD app-registration issues, Storage Account public access, managed-identity scope leakage - [ ] **5.6.7** Cloud credentials in JS bundles / GitHub / npm packages (depends on [6.7](#phase-67--osint--passive-intelligence-p1)) - [ ] **5.6.8** Kubernetes API server exposure (kube-hunter wrapper) - [ ] **5.6.9** Service mesh misconfig (Istio AuthorizationPolicy gaps, Linkerd mTLS bypass) - [ ] **5.6.10** TLS / SSL deep audit (testssl.sh wrapper: weak ciphers, BEAST, CRIME, Heartbleed, Logjam, ROBOT) - [ ] **5.6.11** Email security: SPF / DKIM / DMARC spoofability check - [ ] **5.6.12** DNS misconfigs: AXFR, DNSSEC chain breaks, NS subdomain takeover ### Phase 5.7 — Race Conditions & Logic Bugs (`P2`) Hard to find, very high payout when present. - [ ] **5.7.1** Concurrent-request orchestrator (single-packet attack / last-byte sync, per PortSwigger research) - [ ] **5.7.2** TOCTOU detector on state-changing endpoints (payments, refunds, role changes) - [ ] **5.7.3** State-machine bypass (skip intermediate states in multi-step flows) - [ ] **5.7.4** Business-logic heuristics — price manipulation, coupon stacking, refund abuse, negative quantity, decimal precision loss, currency conversion arbitrage - [ ] **5.7.5** Workflow abuse — subscription renewals, loyalty points, friend referrals - [ ] **5.7.6** Idempotency-key reuse / collision - [ ] **5.7.7** Cart desync (different cart states across sessions / devices) ### Phase 5.8 — OOB Infrastructure (`P0` — enabler) Single integration unlocks blind detection for SSRF, XXE, SSTI, command injection, blind XSS. - [ ] **5.8.1** Self-hosted `interactsh` server (ProjectDiscovery's open Burp-Collaborator-equivalent) - [ ] **5.8.2** Per-campaign unique OOB domain + correlation IDs - [ ] **5.8.3** Auto-correlate inbound OOB hits to outbound payloads → confirmed finding event on the blackboard - [ ] **5.8.4** Inbound webhook receiver mode (handles OAuth callbacks for [6.6](#phase-66--oauth--auth-callback-infrastructure-p0), SSRF echo, blind-XSS) ### Phase 5.9 — Deserialization & Object Injection (`P1`) RCE-tier when present. - [ ] **5.9.1** Java deserialization detector + ysoserial gadget chains - [ ] **5.9.2** .NET deserialization (ysoserial.net) - [ ] **5.9.3** PHP deserialization (PHPGGC chains) - [ ] **5.9.4** Python pickle / pickle-over-HTTP - [ ] **5.9.5** Ruby Marshal deserialization - [ ] **5.9.6** Node.js insecure deserialization (`node-serialize`) - [ ] **5.9.7** YAML deserialization (`yaml.load`) - [ ] **5.9.8** BSON deserialization edge cases - [ ] **5.9.9** Generic detector — serialized magic bytes in HTTP params / cookies → fuzz ### Phase 5.10 — File Upload & Processing (`P1`) Common surface, frequent RCE. - [ ] **5.10.1** Extension / content-type / magic-byte bypass enumeration - [ ] **5.10.2** ImageTragick-class (ImageMagick) exploits - [ ] **5.10.3** SVG with embedded XSS / XXE - [ ] **5.10.4** Office documents with macros / external relationships - [ ] **5.10.5** ZIP slip / path traversal in archive extraction - [ ] **5.10.6** Polyglot files (PDF+JS, image+phar) - [ ] **5.10.7** File upload → RCE via processor chain - [ ] **5.10.8** ZIP bomb / decompression DoS detection (safe probing only) ### Phase 5.11 — HTTP Protocol Quirks (`P1`) XBOW-style high-impact, often headline-payout. - [ ] **5.11.1** HTTP request smuggling — CL.TE, TE.CL, TE.TE, HTTP/2 downgrade - [ ] **5.11.2** HTTP/2 request tunneling - [ ] **5.11.3** HTTP/2 header injection (CRLF in HPACK-encoded headers) - [ ] **5.11.4** Web cache poisoning (CDN cache-key inconsistency) - [ ] **5.11.5** Web cache deception (`/account.json` vs `/account.json/style.css`) - [ ] **5.11.6** Host header injection - [ ] **5.11.7** CRLF response splitting / header injection - [ ] **5.11.8** HTTP Parameter Pollution (HPP) - [ ] **5.11.9** HTTP method override (`X-HTTP-Method-Override`, `_method` param) - [ ] **5.11.10** SMTP smuggling (recent class — line-ending parser drift) ### Phase 5.12 — Supply Chain & Dependency (`P2`) - [ ] **5.12.1** Dependency confusion (npm, PyPI, Maven, RubyGems, NuGet) — name-squat detection - [ ] **5.12.2** Typosquatting detection in installed deps - [ ] **5.12.3** Lockfile injection detection in PR diffs - [ ] **5.12.4** npm install-script review - [ ] **5.12.5** Outdated deps with known CVEs (correlates with CVE corpus from [3.1.1](#phase-31--rag--experience-memory)) - [ ] **5.12.6** Abandoned-package takeover risk assessment - [ ] **5.12.7** SBOM consumption + CVE matching ### Phase 5.13 — CI/CD Pipeline Security (`P2`) - [ ] **5.13.1** GitHub Actions vulns: pwn-requests (untrusted PR triggers), self-hosted runner takeover, secrets in logs, cache poisoning, `workflow_dispatch` input injection - [ ] **5.13.2** GitLab CI equivalent classes - [ ] **5.13.3** Jenkins: unauthenticated `/script` Groovy console, outdated plugin CVE sweep - [ ] **5.13.4** Drone, CircleCI, Buildkite parallels - [ ] **5.13.5** ArgoCD / FluxCD config-drift detection - [ ] **5.13.6** Docker registry auth-bypass enumeration - [ ] **5.13.7** Container image supply chain (typosquat / dep-confusion via `FROM` line) ### Phase 5.14 — Container & Kubernetes Runtime (`P2`) - [ ] **5.14.1** Container-escape pattern detection (privileged containers, mounted Docker socket) - [ ] **5.14.2** `hostPath` / `hostNetwork` / `hostPID` misuse - [ ] **5.14.3** K8s service-account token over-scoping - [ ] **5.14.4** K8s Dashboard / etcd exposure - [ ] **5.14.5** Container runtime CVE matching (runc / containerd / Docker) - [ ] **5.14.6** Helm chart secret leakage ### Phase 5.15 — Network Services (`P3` — for internal-scope programs) - [ ] **5.15.1** LDAP / AD enumeration - [ ] **5.15.2** SMB — null sessions, signing disabled, vuln versions - [ ] **5.15.3** RDP exposure + NLA enforcement check - [ ] **5.15.4** VNC unauthenticated detection - [ ] **5.15.5** IPMI exposure (Supermicro etc.) - [ ] **5.15.6** NTP amplification / monlist - [ ] **5.15.7** SNMP default-community sweep - [ ] **5.15.8** BloodHound integration for AD attack-path analysis ### Phase 5.16 — Mobile Application (`P2`) - [ ] **5.16.1** Android APK static analysis — exported components, content providers, WebView UXSS / JS-bridge abuse, intent-filter abuse, debug / backup enabled, strings.xml secrets, Network Security Config bypass - [ ] **5.16.2** iOS IPA — URL scheme hijacking, Universal Links validation, Keychain leakage, backup secrets - [ ] **5.16.3** Hardcoded API keys / tokens / endpoints in mobile bundles - [ ] **5.16.4** Frida-driven runtime instrumentation for cert-pinning bypass (in-scope mobile programs only) ### Phase 5.17 — Information Disclosure & Sensitive Data Exposure (`P1`) Low payout per finding, but cheap to automate, high hit-rate. Volume play. - [ ] **5.17.1** Source-map exposure (`.js.map`) → unredacted source + secrets - [ ] **5.17.2** `.git` / `.svn` / `.DS_Store` directory exposure - [ ] **5.17.3** Backup-file fuzzing (`.bak`, `~`, `.swp`, `.old`, copy patterns) - [ ] **5.17.4** Debug endpoints (`/debug`, `/health`, `/actuator/*`, `/api/swagger`, `/api-docs`) - [ ] **5.17.5** Stack traces / error-page disclosure - [ ] **5.17.6** Version disclosure → version-specific CVE lookup - [ ] **5.17.7** robots.txt / sitemap.xml mining - [ ] **5.17.8** Hidden form fields / commented endpoints in HTML - [ ] **5.17.9** Secrets in response headers (`X-Powered-By`, custom debug headers) ### Phase 5.18 — Crypto / TLS Implementation (`P2`) - [ ] **5.18.1** Padding oracle (CBC modes) - [ ] **5.18.2** Length-extension attacks - [ ] **5.18.3** ECDSA nonce reuse detection - [ ] **5.18.4** Weak RNG output detection - [ ] **5.18.5** TLS cipher-suite weakness enumeration - [ ] **5.18.6** HSTS bypass / preload-list gaps ### Phase 5.19 — Browser Extension Security (`P3` — niche) - [ ] **5.19.1** Manifest V2 / V3 permission analysis - [ ] **5.19.2** Content-script isolation gaps - [ ] **5.19.3** Native messaging host issues - [ ] **5.19.4** Cross-extension communication abuse --- ## Wave 6 — Real-World Operability (~3 months) > The friction-removal play. A real scan against a real HackerOne program hits rate limits in minutes, needs an authenticated session, has stateful flows, and gets banned without back-off. Wave 6 makes the swarm survive contact with reality. > > Wave 6 is largely **independent of Wave 5** — both can ship in parallel by different contributors. Wave 5 needs the agents; Wave 6 needs the harness. ### Phase 6.1 — Browser-Driven Testing (`P0`) Prerequisite for [5.4](#phase-54--client-side--browser-required-p1--depends-on-61), parts of [5.1](#phase-51--auth--session-specialization-p0), [5.5](#phase-55--api--graphql-security-p1). - [ ] **6.1.1** Playwright integration wired through the swarm scheduler - [ ] **6.1.2** Auth-state persistence (cookies, localStorage, sessionStorage, IndexedDB) - [ ] **6.1.3** DOM source/sink instrumentation hook for client-side bug detection - [ ] **6.1.4** Stateful crawling (form submission, navigation chains) - [ ] **6.1.5** Screenshot + video evidence per finding (replaces gowitness for client-side) - [ ] **6.1.6** Headless-detection bypass (for legit reachability where targets soft-block headless) ### Phase 6.2 — Continuous & Differential Scanning (`P1`) Turns one-shot scans into a service. - [ ] **6.2.1** `pentestswarm watch ` long-running asset monitor - [ ] **6.2.2** Diff-based re-scanning — only test new endpoints since last scan - [ ] **6.2.3** Cron / scheduled mode - [ ] **6.2.4** Asset-change notifications routed via Slack / Discord / email (severity-routed) - [ ] **6.2.5** Long-term findings DB at `~/.pentestswarm/findings.db` (SQLite); searchable across all historical scans - [ ] **6.2.6** Certificate Transparency log monitoring → new-subdomain alert ### Phase 6.3 — Rate Limiting & Anti-Detection (`P0`) Without this, every real scan gets the researcher's IP banned. - [ ] **6.3.1** Adaptive rate limiter — back off on 429, respect `Retry-After`, learn the limit empirically - [ ] **6.3.2** BYO proxy pool (HTTP / SOCKS5) - [ ] **6.3.3** User-agent rotation - [ ] **6.3.4** Request timing jitter (avoid bot-detection fingerprints) - [ ] **6.3.5** Robots.txt respect with `--ignore-robots` override - [ ] **6.3.6** Off-hours mode (only scan 2-5am target-local time for noise-averse programs) - [ ] **6.3.7** WAF detection (Cloudflare / Akamai / Imperva) → log + adjust strategy (does not attempt evasion) - [ ] **6.3.8** CAPTCHA detection → graceful back-off + researcher notification (never fake-solves) ### Phase 6.4 — Distributed Execution (`P3` — later) Premature until single-machine is mature. - [ ] **6.4.1** Federated blackboard across multiple workers - [ ] **6.4.2** Kubernetes Helm chart for large scans - [ ] **6.4.3** Cloud VPS bootstrap (Terraform module for clean-IP runner) - [ ] **6.4.4** Worker-pool scheduling across geographic regions (geo-locked targets) ### Phase 6.5 — Resume, Replay & Reproducibility (`P1`) - [ ] **6.5.1** `pentestswarm resume ` — pick up after Ctrl-C or crash - [ ] **6.5.2** Replay full reasoning trace for any finding (audit + debugging) - [ ] **6.5.3** Deterministic mode — same seed → identical actions - [ ] **6.5.4** Audit log of every tool invocation + LLM call (compliance + debugging) ### Phase 6.6 — OAuth / Auth-Callback Infrastructure (`P0`) Unlocks [5.1](#phase-51--auth--session-specialization-p0) end-to-end testing. - [ ] **6.6.1** Swarm acts as a real OAuth client — hosts the callback URL, handles code exchange - [ ] **6.6.2** TOTP / passkey support when researcher provides the secret - [ ] **6.6.3** API key rotation handling for short-lived creds - [ ] **6.6.4** Session-timeout auto-refresh mid-scan - [ ] **6.6.5** MFA challenge handling — manual prompt or automated if creds provided ### Phase 6.7 — OSINT & Passive Intelligence (`P1`) Cheap signal sources; high hit-rate for cloud creds, exposed assets, leaked endpoints. - [ ] **6.7.1** Shodan API client (already-exposed asset discovery) - [ ] **6.7.2** Censys API client - [ ] **6.7.3** GitHub code search (leaked creds, vulnerable code patterns) - [ ] **6.7.4** GitLab / Bitbucket code search equivalents - [ ] **6.7.5** Wayback Machine deep dive (historical JS bundles → unredacted secrets) - [ ] **6.7.6** Public bug-bounty report scraping (similar programs → known vuln patterns) - [ ] **6.7.7** Paste-site monitoring (pastebin / ghostbin / rentry) for leaked creds matching scope - [ ] **6.7.8** Cert Transparency log queries for subdomain discovery ### Phase 6.8 — Session & State Management Extended (`P1`) - [ ] **6.8.1** Cookie jar with multi-domain awareness - [ ] **6.8.2** JWT refresh-token loop handling - [ ] **6.8.3** CSRF token auto-extraction + replay - [ ] **6.8.4** Anti-bot challenge detection (hCaptcha, reCAPTCHA, Cloudflare Turnstile) → back off; never attempts to fake-solve ### Phase 6.9 — Import / Export & Interop (`P2`) Researchers don't start from zero — let them bring their work in and our work out. - [ ] **6.9.1** Burp session import (`.burp` file) - [ ] **6.9.2** HAR file ingestion (browser-captured session) - [ ] **6.9.3** Postman collection import - [ ] **6.9.4** OpenAPI / Swagger spec import as endpoint corpus - [ ] **6.9.5** Findings export: HAR, Postman, curl-script bundle, Burp request file - [ ] **6.9.6** Cross-tool resume: import Nuclei JSON, Subfinder list, etc., into a campaign ### Phase 6.10 — Manual Override / Human-in-the-Loop (`P2`) - [ ] **6.10.1** `pentestswarm pause` mid-campaign; researcher does manual testing; `pentestswarm resume` with new context - [ ] **6.10.2** "Inspect blackboard" CLI — dump current findings, hypotheses, pheromones - [ ] **6.10.3** Inject manual findings via CLI for the swarm to chain off - [ ] **6.10.4** Interactive shell into a specific finding's reproduction step ### Phase 6.11 — Exploit Chain Planner (`P1`) Reasoning-depth upgrade. Most real exploits are 3-5 vulns chained. - [ ] **6.11.1** Dedicated `ChainPlanner` agent — takes a set of findings, reasons about chain-ability - [ ] **6.11.2** Graph-based "low-priv access → admin" path finding - [ ] **6.11.3** Lateral-movement modeling for multi-host scope - [ ] **6.11.4** Vuln correlation across services (SSRF on foo.example.com + unauth metadata on bar.example.com = combo) ### Phase 6.12 — Attack-Tree Visualization (`P2`) - [ ] **6.12.1** Live dashboard view of the hypothesis tree (tried, failed, current frontier) - [ ] **6.12.2** Helps the researcher trust the swarm + understand why it's not finding bugs in a given direction - [ ] **6.12.3** Export to graphviz / mermaid for writeups and conference talks --- ## Distribution & Community Growth > Trivy, Nuclei, and Gitleaks all got distribution right early. We're following the same playbook. > > Priority calls are driven by **where our users actually are**: pentesters skew Kali / Arch / macOS heavily. That makes Kali + BlackArch the highest-leverage channels, not Snap or npm. Each tier below is roughly ordered by ROI for *this* audience. ### D.1 — macOS - [x] **D.1.1** Homebrew third-party tap — `Armur-Ai/homebrew-tap` live, binary-release formula for darwin/linux × arm64/amd64, auto-bumps on upstream release (see [4.1.4](#phase-41--frictionless-onboarding)) - [ ] **D.1.2** Tap polish pass (`P1`) - [ ] ~~Wire `repository_dispatch` from upstream `release.yml` → tap's `update-formula.yml`~~ *(obsolete once [D.6.3](#d6--direct-install) brews: block is enabled — GoReleaser pushes the formula directly on each release, no polling needed)* - [x] `brew audit --strict ./Formula/pentestswarm.rb` in tap CI — workflow at `homebrew-tap/.github/workflows/ci.yml`. Fails the build on style / lint regressions before they reach users - [x] `brew install + brew test` job in tap CI on macos-latest. Downloads the real release binary, validates SHA, runs the formula's `test do` block. *(Follow-up: extend matrix to macos-13 + linuxbrew on ubuntu-latest)* - [x] Ship bash/zsh/fish completions — Cobra auto-registers `pentestswarm completion ` (hidden from --help via `CompletionOptions.HiddenDefaultCmd`). Homebrew formula uses `generate_completions_from_executable` to install; AUR PKGBUILD shells out to the just-installed binary and writes to the canonical Arch completion paths. No release-asset bloat, no version mismatches between binary and completion scripts. *(Man page deferred — separate session, needs cobra/doc wiring + `cmd/gen-docs/`.)* - [ ] Sign + notarize macOS binaries (Apple Developer ID $99/yr) OR ship `xattr -d com.apple.quarantine` caveat as stopgap - [x] Fix `caveats`: `pip install semgrep` → `brew install semgrep`; document `~/.pentestswarm/` lifecycle (config.yaml, fp-cache.jsonl, findings.db are not removed by `brew uninstall`). Mirrored into `update-formula.yml`'s heredoc so future release-driven regenerations don't revert it. - [ ] Versioned formula (`pentestswarm@0.1.rb`) once 2-3 versions are out - [ ] `head` block for `brew install --HEAD` early-adopter path - [ ] **D.1.3** Homebrew core submission (`post-v1.0`) — rewrite as source-build formula, PR to `Homebrew/homebrew-core`. Drops the `Armur-Ai/tap/` prefix → `brew install pentestswarm`. Wait until API stabilizes; pre-1.0 churn through external review is friction we don't need ### D.2 — Linux (distro-specific) — the high-leverage tier - [ ] **D.2.1** **AUR PKGBUILD** (`P0` — ship now) — PKGBUILD + `.SRCINFO` for the binary-release variant `pentestswarm-bin` authored at [`packaging/aur/pentestswarm-bin/`](packaging/aur/pentestswarm-bin/), mirrored in-tree for PR review. License declared as `AGPL-3.0-only`. Targets the v0.1.0 GitHub release binaries with verified SHA-256 sums. `packaging/aur/README.md` documents the submission + update workflow. **Remaining**: maintainer pushes the PKGBUILD to `ssh://aur@aur.archlinux.org/pentestswarm-bin.git` (requires the maintainer's SSH key + AUR account, which Claude can't set up). Once submitted, `yay -S pentestswarm-bin` covers Arch / Manjaro / EndeavourOS / BlackArch users. Follow-up: GitHub Action to auto-bump pkgver + sha256sums on each upstream release tag, same pattern as `homebrew-tap`'s `update-formula.yml` - [ ] **D.2.2** **BlackArch package** (`P1` — submit 1–2 months after AUR) — submit via [BlackArch packaging guide](https://blackarch.org/guide.html). Smaller, more forgiving review culture than Kali; good place to learn the downstream-maintainer relationship dance. Use AUR `PKGBUILD` as the starting point — most of the work transfers - [ ] **D.2.3** **Kali Linux repos** (`P0` priority, **gated on cadence**) — single biggest leverage point. Submit via [bugs.kali.org](https://bugs.kali.org). Goal: `apt install pentestswarm` on every fresh Kali install - [ ] Gate: ship 3 consecutive monthly releases without a user-visible breaking change (CLI flags, config keys, output formats) before submitting. The old "after v0.5" gate was a version-number proxy for cadence stability; we're tracking the underlying thing directly - [ ] Commit publicly that Kali tracks our **stable tags only** — never alphas / RCs - [ ] Start `BREAKING-CHANGES.md` (or a `breaking:` label in release notes) so "three clean releases" is auditable, not vibes - [ ] **D.2.4** **ParrotOS package** (`P2`) — smaller pentest distro, but loyal user base. Submit upstream - [ ] **D.2.5** **APT repo** for Debian/Ubuntu (`P2`) — host via [Cloudsmith](https://cloudsmith.com) free tier or [packagecloud.io](https://packagecloud.io); fallback `apt` channel for non-Kali Ubuntu users - [ ] **D.2.6** **Fedora COPR** for RPM (`P2`) — community build service, free; covers Fedora/RHEL/CentOS users - [ ] **D.2.7** **Nixpkgs derivation** (`P3`) — niche but very active power-user audience; submit PR to `nixos/nixpkgs` - [ ] **D.2.8** **AppImage** (`P3`) — portable single-file binary; fallback for "weird distro" users. GoReleaser produces this for free ### D.3 — Windows - [ ] **D.3.1** **Scoop** manifest (`P1`) — easiest Windows channel; JSON manifest in `Armur-Ai/scoop-bucket` repo. `scoop install pentestswarm`. Dev-friendly Windows crowd - [ ] **D.3.2** **Winget** submission (`P2`) — Microsoft's official; PR to `microsoft/winget-pkgs`. Growing fast. Slightly painful submission process - [ ] **D.3.3** **Chocolatey** package (`P2`) — larger reach than Scoop but heavier process (nuspec, moderation queue). Submit after Scoop is stable ### D.4 — Containers - [x] **D.4.1** GHCR multi-arch image — `ghcr.io/armur-ai/pentestswarm:latest` + `:vX.Y.Z` + `:edge`, linux/amd64 + linux/arm64, 16 tools pre-installed (see [4.1.6](#phase-41--frictionless-onboarding)) - [ ] **D.4.2** Docker Hub mirror (`P3`) — some users default to Docker Hub; cheap to mirror via a workflow that pushes after each GHCR build ### D.5 — CI / automation channels - [ ] **D.5.1** **GitHub Action on Marketplace** (`P1`) — `uses: Armur-Ai/pentestswarm-action@v1` in user CI. Use case: continuous bug-bounty scans, drift detection. (Originally D.4 / see Phase 2.4.1) - [ ] **D.5.2** GitLab CI component (`P3`) — GitLab's equivalent. Smaller audience but trivial to mirror once the GitHub Action exists ### D.6 — Direct install - [x] **D.6.1** GitHub release binaries — `release.yml` builds for darwin/linux/windows × amd64/arm64, publishes with checksums on `v*` tag - [ ] **D.6.2** `curl | sh` installer (`P1`) — `curl -sSL install.pentestswarm.ai | sh`; detects OS/arch, fetches the right binary. Already [4.1.3](#phase-41--frictionless-onboarding); needs `install.pentestswarm.ai` hosted - [x] **D.6.3** **GoReleaser pipeline** — `.goreleaser.yaml` replaces the hand-rolled matrix build in `release.yml`. Builds 5 targets (darwin / linux × amd64 / arm64, plus windows / amd64), ships raw binaries matching the existing `pentestswarm--` naming (Homebrew formula + AUR PKGBUILD keep working unchanged), generates `checksums.txt` (SHA-256), auto-renders a categorized changelog from conventional-commit messages. `make snapshot` runs a local dry-run; `make release-check` validates the config. Local snapshot build verified end-to-end. `brews:` and `aurs:` blocks staged but not yet enabled — they require `HOMEBREW_TAP_TOKEN` (PAT to the tap repo) and `AUR_KEY` (SSH deploy key to `aur:pentestswarm-bin`) secrets respectively. Once those are added, GoReleaser pushes formula + PKGBUILD updates directly on each release tag, eliminating the 6h cron in `homebrew-tap/.github/workflows/update-formula.yml` and any manual AUR maintenance. - [ ] **D.6.4** SLSA / `cosign` signatures (`P2`) — sign release artifacts; publish provenance. High trust signal for a *security* tool. GoReleaser has a `signs:` block - [ ] **D.6.5** Reproducible builds (`P3`) — pin Go toolchain, `-trimpath`, fix `SOURCE_DATE_EPOCH`; lets third parties verify binary ↔ source. OpenSSF Best Practices badge points ### D.7 — Community + marketing - [ ] **D.7.1** `pentestswarm.ai` landing site - [ ] Value prop + demo GIF - [ ] Benchmark numbers (after Phase 3.3) - [ ] Playbook marketplace listing - [x] Discord invite — live server at [discord.gg/6qtkhpW8tk](https://discord.gg/6qtkhpW8tk), linked from README badge + Community section (see [D.9.7](#d9--activation--positioning-the-try-it-funnel)) - [ ] **D.7.2** Discord community — server is live; still to do: dedicated channels (#bugbounty, #ctf, #asm, #playbooks), pinned "run your first scan" guide, and `good first issue` cross-post - [ ] **D.7.3** Monthly release cadence with release notes framed as "what's new for the swarm" - [ ] **D.7.4** Conference talks - [ ] Submit to Black Hat Arsenal 2026 - [ ] Submit to DEF CON Demo Labs - [ ] BSides circuit for regional credibility - [ ] **D.7.5** Weekly content: one tutorial / blog / video per week during Wave 1+2 - [ ] **D.7.6** Launch posts on each major channel addition (Kali, BlackArch, AUR, Homebrew core acceptance) — LinkedIn + r/netsec + Twitter; each new package channel is a free PR moment - [x] **D.7.7** **Adopters list** — `ADOPTERS.md` + a "Who's using Pentest Swarm?" README section inviting orgs to add themselves via PR (empty to start — honest, fills in as real users appear; not GitHub's auto "Used by" widget, which only counts library imports and stays sparse for a CLI). Follow-up: logo wall on the landing site once there are named, permissioned adopters. - [x] **D.7.8** **Enterprise & commercial support** — README "Enterprise & Commercial Support" section (services-only for now: managed deployment on customer infra incl. air-gapped, integration/customization, priority support + SLAs, training). Email contact. Follow-ups: dedicated services page on the landing site; revisit commercial/dual-licensing later if demand warrants (deliberately deferred). ### D.8 — Explicitly skipped (with rationale) These are intentionally **not** on the roadmap. Adding them is unforced effort. - **Snap / Flatpak** — sandbox confinement breaks tools that scan arbitrary network targets and spawn other binaries. `--classic` workarounds make Canonical reviewers unhappy. Wrong shape for a pentest tool - **npm wrapper** — pattern works (`esbuild`, `prisma`, `swc` do it), but pentesters aren't `npm i`-ing security tools. Wrong audience - **pip wrapper** — only justifiable if we ship Python bindings later - **MacPorts** — audience overlap with Homebrew users is ~100%, MacPorts coverage adds nothing - **Pentoo (Gentoo)** — audience too small to justify - **openSUSE OBS** — audience too small to justify - **Pre-commit hook** — wrong shape; scans take minutes, not seconds - **Helm chart** — only relevant if we ship a long-running dashboard/server; current product is CLI-first ### D.9 — Activation & positioning (the try-it funnel) > Packaging (D.1–D.7) gets people to the repo. This tier is what happens in the first five minutes: try it free, see it work, understand why it's different. Highest-leverage for converting a star into a user. - [x] **D.9.1** **Bundled legal lab target** — `pentestswarm scan --lab` (`cli/lab.go`) spins up OWASP Juice Shop via an embedded docker-compose, polls it ready host-side, points the swarm at `http://localhost:3000` (scope `127.0.0.1/32,localhost`), and tears it down on exit — the genuine "watch it find a real vuln in 2 minutes" run. No target/scope args needed, and **no API key** when paired with `--provider ollama` (the `--provider` override now also takes effect for the pre-flight key check). Human-auditable copy at `deploy/lab/docker-compose.yml`. Verified end-to-end: up → ready → scoped → teardown, no leftover container (finding an actual vuln needs a running model, which the user supplies). Follow-ups: add DVWA as a second lab service; a `--lab-target` picker. - [x] **D.9.2** **$0 / no-API-key local-first onboarding** — README Quick Start now leads with the fully-local (Ollama / LM Studio) path: zero key, 100% on-box; cloud is the "max quality" upsell, not the default. Follow-ups: `doctor` nudges toward a recommended local model; a `--provider ollama` smoke path. - [x] **D.9.3** **"Harness, not the model" positioning** — README hero + LLM Providers now frame Pentest Swarm as the harness that gives *any* model hands (frontier, OpenAI-compatible, fully-local, and the new security-tuned open models like Pentest-R1) — explicitly *not* competing with those models. Follow-up: provider presets + a `docs/models.md` "which model?" matrix incl. the security-tuned options. - [ ] **D.9.4** **Speed benchmark — swarm vs. sequential** (`P1`) — the most on-brand, easiest-to-produce number: run `--swarm` and the sequential runner over the same target set and publish the wall-clock ("same findings, N× faster"). Proves the "concurrent / machine-speed / *real* swarm" claim without needing to win an exploitation benchmark. Harness at `tests/bench/speed/`; chart in `docs/benchmarks.md` + README. Pairs with D.9.1 (run it against the lab target). **Runner shipped**: `ShellRunner` scans a target both ways, times each, counts findings from the JSON report; env-guarded live test (`SPEEDBENCH_TARGET=…`) prints the `Summarize` headline with a `FindingsParity` honesty guard. Remaining: run it against the lab with a model and publish the chart. - [ ] **D.9.5** **MCP discovery** (`P1`) — list `pentestswarm mcp serve` in the MCP registries (Smithery, mcp.so, Cursor / Claude Desktop directories) and ship a copy-paste MCP quickstart. Being a great MCP server is live distribution to every Claude Desktop / Cursor user. - [ ] **D.9.6** **Contributor funnel** (`P1`) — `good first issue` / `help wanted` labels on the smallest plan items (tool adapters are the perfect on-ramp — the pattern is ~30 lines), an "add a tool in 30 lines" section in `CONTRIBUTING.md`, and a fiercely-guarded `scripts/setup.sh` (a broken dev setup kills contributors). We're already merging community PRs; this widens the top of the funnel. **Started**: closed 5 stale good-first-issues (adapters already merged: nikto/droopescan/crackmapexec/bloodhound/dotdotpwn) and opened 4 fresh ones — #60 `whatweb`, #61 `wafw00f`, #62 `feroxbuster`, #63 `docs/models.md` — so the README's good-first-issue link points at real open work. Added the **"Add a tool adapter (~30 lines)"** section to `CONTRIBUTING.md` (copy nikto.go → 6 steps → acceptance), linked from the good-first-issues. CI now has a **Shell scripts** job (`bash -n` + `shellcheck` over `scripts/*.sh`) so a broken dev setup can't reach contributors. Done. - [x] **D.9.7** **Star + Discord CTA in README** — clickable Discord badge, a Community section with copy pointing at the live server ([discord.gg/6qtkhpW8tk](https://discord.gg/6qtkhpW8tk)), and a tasteful "drop a star" ask. ### D.10 — Standardization & findings quality (enterprise credibility) > Security teams expect findings in the industry's language: OWASP / CWE labels, CVSS, evidence, repro steps, low false-positive rate. Audit (2026-08): most of this is **already shipped** — CVSS v3.1 (`classifier/scorer.go`), validation evidence (`report/evidence/evidence.go` — HTTP req/resp + screenshots), custom attack chains (`EXPLOIT_CHAIN` + `AttackStep`), reproducible steps (`Reproduction` + `report/submission.go`), and a multi-layer false-positive stack (`fp_filter` + `dedup` + `qualitygate` + `fpcache` + `cross_validate` + the verified-PoC publish threshold). SAST (semgrep/gitleaks/trufflehog/checkov) **and** DAST (nuclei/sqlmap/dalfox/nikto/crlfuzz/gxss/arjun/wpscan) both ship. The gaps below close the rest. - [x] **D.10.1** **OWASP Top 10 + CWE tagging on findings** — new `internal/taxonomy` package maps a finding's (free-form, LLM-produced) attack category + title onto an OWASP Top 10 2021 category + CWE id, with whole-word matching so short tokens don't false-match (e.g. `rce` inside "resource"). `ReportFinding` gains `owasp`/`cwe` fields, populated in `report/agent.go`; surfaced as a **Classification** line in the markdown report, as fields in the JSON report, and as `external/cwe/cwe-nnn` + `owasp/…` rule tags in the SARIF emitter (Code Scanning links CWE tags). ~25-entry taxonomy table; taxonomy + renderer tests. Tagged at report time (single clean integration point) rather than in the classifier — same visible outcome, lower blast radius. The single most-asked-for standardization signal. - [ ] **D.10.2** **Honest "novel-vulnerability discovery" framing** (`P1`) — we do **not** do "zero-day detection" and must never claim it (it invites ridicule). What we actually have: novel-vuln discovery via active fuzzing (`crlfuzz`/`gxss`/`arjun`/`dalfox`) + LLM reasoning, and the **nuclei-template-authoring agent** (`internal/swarm/agents/nuclei_author.go`) that codifies newly-found high/critical issues into reusable detections. Document it accurately as "finds and codifies novel issues", never "zero-day". - [x] **D.10.3** **"Steps to Reproduce" in every report path** — the default report dropped repro (renderer only emitted Description/Evidence/Remediation; `ReportFinding` had no repro field and `ReportAgent.Generate` never copied `f.Reproduce`). Fixed: added `Reproduce *Reproduction` to `ReportFinding`, wired it through the report agent, and `renderer.go` now renders a **Steps to Reproduce** block (shell command + raw HTTP request + expected indicator), guarded so empty reproductions print nothing. JSON report carries it too. 2 renderer tests. ### D.11 — Enterprise adoption (getting onto the approved-tools list) > Four gates get a tool adopted by a security team: **speak the standards**, **plug into their workflow**, **satisfy governance**, and **report in a form execs consume**. Items already tracked elsewhere are cross-referenced so nothing is duplicated; the checkboxes below are the gaps. **Speak the standards** — every finding carries the metadata enterprises triage by. Complements [D.10.1](#d10--standardization--findings-quality-enterprise-credibility) (OWASP/CWE). *Already shipped: SARIF 2.1.0 export ([2.4.4](#phase-24--integrations--distribution)); Code-Scanning emitter wiring still pending (see the SARIF sub-item under 2.4).* - [x] **D.11.1** **MITRE ATT&CK mapping** — extended `internal/taxonomy` with an ATT&CK technique per class (e.g. injection/RCE/SSRF/deserialization → `T1190 Exploit Public-Facing Application`, XSS → `T1059.007`, path traversal → `T1083`, privesc → `T1068`, auth → `T1078`, crypto → `T1040`). `ReportFinding.ATTACK` surfaces in the markdown **Classification** line, JSON, and an `attack/Txxxx` SARIF tag. ATT&CK is TTP-oriented and many web-vuln classes don't map cleanly, so unmapped classes (CSRF, open redirect, headers, info-disclosure, logging) are left **blank rather than invented**. taxonomy + renderer tests. - [ ] **D.11.2** **CISA KEV enrichment** (`P1`) — flag any finding whose CVE is on the CISA Known-Exploited-Vulnerabilities catalog ("actively exploited in the wild") via a nightly KEV feed pull. Instant, defensible prioritization — very low effort, high perceived value. - [ ] **D.11.3** **EPSS score enrichment** (`P1`) — attach FIRST EPSS exploit-probability scores to CVE findings and let triage sort by them. Pairs with KEV as the modern prioritization signal. - [ ] **D.11.4** **Compliance & control mapping — the GRC buying accelerator** (`P1`, raised from `P2`) — the fastest path to a "yes" from **both** sides of the market: enterprise procurement (GRC has to justify every tool against a control framework) **and** startups racing to their first SOC 2 / ISO 27001 cert to close their own enterprise deals. Two layers, both pure enrichment over data we already produce — **no new scanning**: **(a)** every finding is stamped with the specific controls it touches, and **(b)** the whole campaign is framed as *audit evidence* for the controls that **mandate** security testing. Built on the existing `internal/taxonomy` (every finding already resolves to OWASP + CWE + ATT&CK): add one mapping table keyed by CWE/OWASP → control ids, a report section, and JSON/SARIF fields. Prioritization signals ([D.11.2](#d11--enterprise-adoption-getting-onto-the-approved-tools-list) KEV, [D.11.3](#d11--enterprise-adoption-getting-onto-the-approved-tools-list) EPSS) feed the "which controls, how urgent" story. - [x] **D.11.4.1** **Mapping engine** — `internal/compliance` package (sibling to `internal/taxonomy`) shipped: `Map(owasp, cwe, enabled) → []Control` keyed off the finding's already-resolved OWASP class (with per-CWE overrides where a CWE maps more precisely, e.g. security-headers → PCI 6.4.1); `Mandate(enabled)` returns the campaign-level "testing is required" controls (PCI 11.4.1/11.3.1, SOC 2 CC4.1/CC7.1, ISO A.8.8/A.8.29, NIST CSF ID.RA-01/ID.IM-02); `ParseFrameworks("pci,soc2,…")` for a selectable set; `Control.Tag()` emits `pci-dss/…`/`soc2/…`/`iso27001/…`/`nist-csf/…` for SARIF/JSON; `Disclaimer` const carries the honesty notice. Table-driven (adding a framework is new rows), leaf package (no deps), unmapped classes return nil — never invented. 10 tests, all green. **Remaining (rolls into D.11.4.6):** the `--compliance` CLI/config flag and the caller that walks findings into the report. - [ ] **D.11.4.2** **PCI DSS v4.0** (`P1`) — map finding classes to requirements: injection/XSS/access-control → **Req 6.2.4** (secure coding) + **Req 6.3** (fix known vulns); TLS/crypto → **Req 4.2.1**; auth/session → **Req 8**; misconfig/headers → **Req 2.2 / 6.4.1**; outdated components → **Req 6.3.3**. **The wedge:** position the campaign itself as evidence for **Req 11.4.1–11.4.5 (penetration testing)** and **Req 11.3.1/11.3.2 (internal & external vulnerability scanning)** — the requirements that literally *require* this activity quarterly / after significant change. We don't just "touch" PCI; we produce the artifact a QSA asks for. - [ ] **D.11.4.3** **SOC 2 (AICPA Trust Services Criteria)** (`P1`) — map to the Common Criteria, anchored on **CC7.1 (vulnerability identification / security monitoring)** as requested, plus **CC4.1** (monitoring of controls), **CC6.1 / CC6.6** (logical access — auth/authz findings), **CC6.7** (transmission — TLS/crypto), **CC7.2** (anomaly monitoring), **CC8.1** (change management — CI/CD & config findings). The report frames the pentest as **CC4.1 / CC7.1** evidence covering the Type II audit period. - [ ] **D.11.4.4** **ISO/IEC 27001:2022 (Annex A)** (`P1`) — map to **A.8.8 (management of technical vulnerabilities — the anchor)**, **A.8.29 (security testing in development and acceptance)**, **A.8.24** (use of cryptography), **A.8.5 / A.8.2** (secure authentication / privileged access), **A.8.9** (configuration management), **A.5.7** (threat intelligence — for CVE/KEV-tagged findings). Campaign framed as **A.8.8 + A.8.29** evidence for the ISMS. - [ ] **D.11.4.5** **NIST CSF 2.0** (`P1`) — map to Function → Category → Subcategory: **ID.RA-01** (vulnerabilities in assets identified), **ID.RA-05** (prioritization — pairs directly with CVSS + EPSS + KEV), **DE.CM-09** (computing hardware/software monitored for events), **PR.PS-*** (platform security — config/patch findings), **PR.AA-*** (identity/auth). CSF 2.0 is the language US enterprises and their boards use for posture. - [ ] **D.11.4.6** **Report + machine outputs** (`P1`) — a **Compliance Coverage** section in the markdown/PDF report: per selected framework, a table of touched controls with the count and max-severity of findings mapped to each, plus a plain-English "what this means for your audit" line per control. JSON report gains a `compliance` block (`framework → control → [finding ids]`). SARIF rules gain `tags` (`pci-dss/11.4`, `soc2/CC7.1`, `iso27001/A.8.8`, `nist-csf/ID.RA-01`) so GitHub Code Scanning and GRC importers ingest them. Builds on [D.11.11](#d11--enterprise-adoption-getting-onto-the-approved-tools-list) (exec report) + [4.4.14](#phase-44--reporting--evidence) (PDF). - [ ] **D.11.4.7** **More frameworks — demand-ordered** (`P2`/`P3`) — same table, more rows: **OWASP ASVS** (appsec teams request verification-level mapping by name), **CIS Controls v8** (maps cleanly from CWE), **NIST 800-53 Rev5** (RA-5, SI-2, SC-8, IA-* — the FedRAMP substrate), **HIPAA Security Rule** (§164.308(a)(1) risk analysis, §164.312 technical safeguards — healthcare buyers), **GDPR Art. 32** (security of processing — EU), with stubs for **HITRUST CSF**, **FedRAMP**, **DORA** (EU financial), and **Essential Eight** (AU gov). Ship the four named above first; the rest land incrementally. - [ ] **D.11.4.8** **Honesty guardrail** (`P1`, non-negotiable) — every compliance section states findings *map to* / *provide evidence toward* a control, **never** "you are compliant." Compliance is an audit outcome, not a scan result; over-claiming is both false and a legal liability. A standing disclaimer ships in every compliance output — same discipline as [D.10.2](#d10--standardization--findings-quality-enterprise-credibility) (no "zero-day" claims). **Plug into the workflow** — findings that don't reach the team's existing tools don't get fixed. *Already shipped: Jira ([2.4.2](#phase-24--integrations--distribution)); Slack ([2.4.3](#phase-24--integrations--distribution)); SARIF/SIEM slice ([2.4.4](#phase-24--integrations--distribution), with CEF + STIX deferred there).* - [ ] **D.11.5** **DefectDojo import** (`P1`) — push findings to DefectDojo, the de-facto open-source appsec vuln-management hub. The single biggest lever for landing inside an existing appsec workflow. - [ ] **D.11.6** **ServiceNow ticketing + dedup** (`P2`) — a ServiceNow adapter at parity with the Jira one, plus finding-level dedup so re-scans don't re-file the same issue (applies to Jira too). - [ ] **D.11.7** **Native SIEM push + Teams** (`P2`) — Splunk HEC + Microsoft Sentinel connectors (beyond the SARIF/CEF slice), and Microsoft Teams notifications at parity with Slack. **Satisfy governance** — the offensive-tool-specific gate. *Already shipped/tracked: audit log of every tool + LLM call ([6.5.4](#phase-65--resume--replay-p2)); signed releases + SLSA provenance ([D.6.4](#d6--direct-install)); on-prem / air-gapped / local-model is the core product story.* - [ ] **D.11.8** **Rules-of-Engagement enforcement** (`P1`) — an authorization record per campaign (scope + written-approval reference), **blackout windows**, global rate / blast-radius caps, and a **kill switch**. Packages the existing scope enforcement, `--safe-mode`, cleanup registry, and rate limiter into the RoE model that security governance requires before approving an *offensive* tool. - [ ] **D.11.9** **SBOM on releases** (`P2`) — emit a CycloneDX / SPDX SBOM in the release pipeline (extends [D.6.4](#d6--direct-install) signing). A recurring procurement / vuln-management requirement. - [ ] **D.11.10** **Enterprise / air-gapped deployment guide** (`P2`) — a docs page packaging the on-prem, no-data-egress, local-model story explicitly for security buyers (the capability exists; this makes it legible to procurement). **Report in a form execs consume** — *Already tracked: Markdown→PDF ([4.4.14](#phase-44--reporting--evidence)); run N vs N-1 diff ([1.2.6](#phase-12--dashboard-wire-up)).* - [ ] **D.11.11** **Executive report** (`P2`) — a one-page exec summary (risk posture, severity breakdown, trend vs. last run, top remediations) rendered to branded PDF. Builds on [4.4.14](#phase-44--reporting--evidence) (PDF) + [1.2.6](#phase-12--dashboard-wire-up) (diff). - [ ] **D.11.12** **White-label / branded reports** (`P3`) — logo + org name on reports, for consultancies / MSSPs delivering to their own clients. --- ## README Rewrite The current README overclaims in specific places. Fix after Wave 1.1 is merged (not before — we need to be able to keep the new claims true). - [x] **R.1** Hero block (title + tool demo GIF + architecture GIF + badges) - [x] **R.2** New section: *"What makes this a swarm?"* — stigmergy / emergence / decentralization - [x] **R.3** Swarm diagram rewritten around the blackboard - [x] **R.4** Competitor table (us vs. PentestGPT / HackingBuddyGPT / PentAGI / Shannon / HexStrike / Pentest-R1) - [ ] **R.5** Benchmark numbers inline (after Phase 3.3 ships) - [x] **R.6** "5-agent architecture" claim replaced with accurate description - [x] **R.7** "ReAct loop" replaced with stigmergic-swarm framing - [x] **R.8** Tool claim updated to 8 (ProjectDiscovery stack + nmap) - [x] **R.9** Feature-status table with stable / beta / alpha / planned labels - [x] **R.10** Credits & research section with inspiration links --- ## Architectural Decisions ### AD-1: Build the swarm natively in Go, not on Google ADK / CrewAI / AutoGen **Options considered:** | Framework | Language | Pattern | Fits? | |---|---|---|---| | **Google ADK** | Python (Java/Go partial) | Orchestrator + agents | No | | **CrewAI** | Python | Role-based orchestration | No | | **AutoGen** | Python | Conversational multi-agent | No | | **LangGraph** | Python/JS | State-machine graph | Partial | | **Eino** (ByteDance) | Go | Orchestration-oriented | Partial | | **Custom Go + Blackboard** | Go | Stigmergy-native | ✅ | **Decision**: build native. Reasons: 1. **Language alignment.** The codebase is Go. Bridging to Python ADK via gRPC or subprocess adds latency, complexity, and breaks the "single binary" distribution promise that's central to the Go-native pitch. 2. **Swarm intelligence ≠ multi-agent orchestration.** All the frameworks above assume a central planner dispatches to specialist agents. Stigmergy is the *opposite* — no central planner, coordination via shared environment. Bolting stigmergy onto an orchestrator framework is more work than writing it from scratch. 3. **Blackboard is ~800 LoC.** We already have Postgres + pgvector + an event bus + agent abstractions. What's missing is the *trigger semantics* and *pheromone decay* — both are small, self-contained. 4. **We can always adopt ADK later** for specific features (e.g., ADK's evaluation framework) without buying its whole programming model. **Trade-off accepted**: we don't get the community plugins / integrations those frameworks ship with. Counter: our integrations are shell-level (tools, not agents), so framework plugins aren't a fit anyway. ### AD-2: Postgres as the Blackboard store Alternatives: Redis Streams, NATS JetStream, in-memory only. **Decision**: Postgres + pgvector. We already run it. Transactional writes matter for exploit-cleanup correctness. Query flexibility (SQL + vector) matters for trigger rules. Redis Streams would be faster but give up the query expressiveness. ### AD-3: Pheromone decay is per-finding-type Different finding classes have different half-lives. A `PORT_OPEN` is valid for hours; a `SESSION_TOKEN` is valid for minutes. Config-driven. ### AD-4: Agents are goroutines, not containers For v1. Containerization per agent is Phase 1.3.4 (executor sandbox), not per-agent-process. Easier to debug, smaller footprint, fine until we need strong isolation between untrusted agent code. --- ## Benchmarks Target Aim for v1.0: | Benchmark | Current (est.) | v1.0 Target | Leader | |---|---|---|---| | Cybench (full) | Unknown | ≥ 35% | Pentest-R1 at ~40%+ with Claude Sonnet | | AutoPenBench | Unknown | ≥ 30% | xOffense 79% (Qwen3-32B tuned) | | CVE-Bench (one-day) | Unknown | ≥ 60% | GPT-4 87% with advisory | | HackTheBox retired (easy) | Unknown | ≥ 50% | — | Publish results per-release in `docs/benchmarks.md`. --- ## Research References Living list — add as you read. **Swarm intelligence foundations:** - [LLM-Powered Swarms (arXiv:2506.14496)](https://arxiv.org/pdf/2506.14496) - [Multi-agent systems powered by LLMs: swarm intelligence (Frontiers AI 2025)](https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2025.1593017/full) - [Ledger-State Stigmergy (arXiv:2604.03997)](https://arxiv.org/abs/2604.03997) **State of the art in AI pentesting:** - [Pentest-R1 (arXiv:2508.07382)](https://arxiv.org/abs/2508.07382) — 2-stage RL - [Cybench (arXiv:2408.08926)](https://arxiv.org/abs/2408.08926) - [CAIBench meta-benchmark (arXiv:2510.24317)](https://arxiv.org/html/2510.24317v1) - [Benchmarking LLM-driven Offensive Security (arXiv:2504.10112)](https://arxiv.org/html/2504.10112) - [Cloak, Honey, Trap — USENIX Security 2025](https://www.usenix.org/conference/usenixsecurity25/presentation/ayzenshteyn) **Memory / reasoning techniques:** - [From Experience to Strategy — trainable graph memory (arXiv:2511.07800)](https://arxiv.org/html/2511.07800v1) - [SAILOR symbolic + LLM (arXiv:2604.06506)](https://arxiv.org/abs/2604.06506) **Agent security (for Phase 3.4):** - [MemoryGraft (arXiv:2512.16962)](https://arxiv.org/abs/2512.16962) - [MINJA memory injection (arXiv:2503.03704)](https://arxiv.org/html/2503.03704v2) - [Dark Side of LLMs — agent takeover (arXiv:2507.06850)](https://arxiv.org/html/2507.06850v3) **MCP ecosystem:** - [PortSwigger Burp MCP](https://portswigger.net/bappstore/9952290f04ed4f628e624d0aa9dccebc) - [PentestMCP (arXiv:2510.03610)](https://arxiv.org/html/2510.03610v1) - [Top MCP Servers for Cybersecurity 2026 (Levo)](https://www.levo.ai/resources/blogs/top-mcp-servers-for-cybersecurity-2026) --- ## Revision Log - **2026-04-18**: v1 draft. Wave 1 / 2 / 3 structure, Blackboard architecture, ADK decision. - **2026-04-19**: v2 adds **Wave 4 — Researcher Workflow**. Long-arc vision is now the XBOW-for-bug-bounty play: one command, real bounties, open source. Wave 4 is the only wave judged on whether researchers file real vulns in the 30 days after release — everything else earns its keep by feeding that outcome. - **2026-05-12**: Restructured **Distribution & Community Growth** from a flat D.1–D.10 list into eight tiered subsections (macOS / Linux-distro / Windows / Containers / CI / Direct-install / Community / Skipped). Reflects accurate done-status for Homebrew tap (4.1.4) and GHCR image (4.1.6); documents what we're explicitly *not* shipping (Snap, Flatpak, npm, MacPorts) and why. Replaces the old blanket "after v0.5" gate on Kali/BlackArch with per-channel sequencing: AUR ships now (we own the PKGBUILD — no maintainer to burn), BlackArch follows 1–2 months later, Kali is gated on "3 consecutive monthly releases with no user-visible breaking change" rather than a version number. Adds a `BREAKING-CHANGES.md` tracking corollary so the Kali gate is auditable. - **2026-05-12**: Adds **1.4.6 OpenAI-compatible provider** — single-file implementation covering Together AI (Qwen / Kimi / DeepSeek / Llama via one key), DeepSeek direct, Groq, OpenAI. Strategic enabler for (a) the multi-column Cybench benchmark in 3.3.1, (b) addressing Claude's refusal rate on offensive-security prompts, (c) ~10× cheaper iteration cycles for benchmark debugging. Extends 3.3.1 with a multi-backend benchmark sub-task and the agreed 3-phase $200 Cybench budget plan (Haiku debug → Sonnet subset → Sonnet full). - **2026-05-12**: Adds **1.4.7 Offensive-security prompt scaffolding** — Flavor C from the Pentest-R1 analysis. Curated system prompts, few-shot examples per Cybench category, tool-use templates, refusal-handling retry path. Zero compute cost; pure prompt engineering. Captures part of the frontier→SOTA gap (~12% → ~40% on Cybench) without any model training. Sequenced to ship before Cybench Phase 3 so published numbers reflect the engineered baseline. Flavors A (light SFT on Qwen-7B) and B (full Pentest-R1 reproduction on Qwen3-32B) deliberately deferred; both remain conceptually in Phase 3.2. - **2026-05-12**: Adds **Wave 5 — Vulnerability Class Coverage** (19 phases, ~6 months) and **Wave 6 — Real-World Operability** (12 phases, ~3 months). The depth-and-friction play to match Shannon / Strix / PentAGI coverage. Wave 5 specializes by vuln class: auth (5.1), IDOR/BOLA (5.2), server-side injection (5.3), client-side (5.4), API/GraphQL (5.5), cloud (5.6), race/logic (5.7), OOB infrastructure (5.8), deserialization (5.9), file upload (5.10), HTTP protocol quirks (5.11), supply chain (5.12), CI/CD (5.13), containers (5.14), network services (5.15), mobile (5.16), information disclosure (5.17), crypto/TLS (5.18), browser extensions (5.19). Wave 6 handles real-world friction: browser-driven testing (6.1), continuous monitoring (6.2), rate limiting (6.3), distributed execution (6.4), resume/replay (6.5), OAuth/auth-callback infrastructure (6.6), OSINT (6.7), session management (6.8), import/export (6.9), human-in-the-loop (6.10), exploit chain planner (6.11), attack-tree visualization (6.12). Also extends Phase 2.1 with 20 new tool adapters (dalfox, testssl, jwt_tool, kube-hunter, arjun, ysoserial, pacu, prowler, interactsh-client, etc.), Phase 2.4 with 4 OSINT integrations (Censys, Shodan, GitHub/GitLab code search), Phase 4.1 with tutorial + demo modes, Phase 4.4 with video walkthroughs / re-test / bounty calc / disclosure timeline. Every item has a P0–P3 priority tag for navigability. **Explicitly out of scope**: LLM/AI application targeting (prompt injection in user-facing LLM apps, model extraction, AI safety-filter bypass) — we use LLMs internally; we don't test them as targets.