> **English** | **[简体中文](lang/zh-CN/ARCHITECTURE.md)** | **[日本語](lang/ja/ARCHITECTURE.md)** | **[한국어](lang/ko/ARCHITECTURE.md)** # ATLAS Architecture System architecture for ATLAS V3.1.3. Two-layer design: an outer agent loop handles tool-call orchestration, and an inner V3 pipeline generates diverse code candidates with build verification and energy-based selection. --- ## 1. System Overview ```mermaid graph LR User["User"] --> TUI["atlas-tui\n(Bubbletea)"] TUI --> Proxy["atlas-proxy\n:8090"] subgraph outer["Outer Layer"] Proxy -->|"grammar JSON"| LLM["llama-server\n:8080"] Proxy -->|"T2 files"| V3Service["v3-service\n:8070"] end subgraph inner["Inner Layer"] V3Service --> LLM V3Service --> Lens["geometric-lens\n:8099"] V3Service --> Sandbox["sandbox\n:30820"] Lens --> LLM end style User fill:#333,color:#fff style TUI fill:#1a3a5c,color:#fff style Proxy fill:#1a3a5c,color:#fff style LLM fill:#5c1a1a,color:#fff style V3Service fill:#2d5016,color:#fff style Lens fill:#2d5016,color:#fff style Sandbox fill:#2d5016,color:#fff ``` Services run as containers via Docker Compose (recommended) or as local processes via the `atlas` launcher. Only llama-server uses the GPU. Everything else runs on CPU. The chat front-end is the **atlas-tui** (Bubbletea): a native Go terminal UI consuming `/v1/agent` (per-turn chat SSE) and `/events` (global typed-envelope feed for the pipeline pane). Launch with `atlas` (interactive default) or `atlas tui` (explicit). Pipeline pane shows V3 stages live; chat pane renders assistant markdown via glamour; slash commands `/add /diff /commit /run` etc. handle local file context and shell-out. Mode-aware input (chat / `!bash` / `/slash`) with a hint dropdown. Third-party clients that want tool calls + V3 pipeline target `/v1/agent` directly; `/v1/chat/completions` is a passthrough to llama-server (see §3). The contract is documented in [API.md](API.md). ### 1.1 Supported Accelerators llama-server is the only GPU-using service; every other ATLAS service runs on CPU (proxy is Go, v3-service / geometric-lens / sandbox are Python). That keeps the multi-backend surface small — adding a new accelerator means a new Dockerfile + an entrypoint env-var branch, not changes to the pipeline. | Backend | Status (V3.1.x) | Image / build path | Compose override | Tested cards | |---|---|---|---|---| | **CUDA** (NVIDIA) | Supported (since V3.1.0) | `inference/Dockerfile.v31` → `atlas-llama` | (default) | RTX 5060 Ti 16GB (canonical). The published image is compiled for Blackwell (compute capability 12.0/12.1) only; earlier generations need a local rebuild — see [SETUP.md](SETUP.md) | | **ROCm / HIP** (AMD) | Community-tested (since V3.1.1) | `inference/Dockerfile.rocm` → `atlas-llama-rocm`, built on the host (`pull_policy: build`; no GHCR image) | `docker-compose.rocm.yml` | RX 7900 XTX (community smoke-test, GH #26) | | **Metal** (Apple Silicon) | Supported ([#32](https://github.com/itigges22/ATLAS/issues/32)) | Hybrid: native llama-server (Metal) + Docker for the rest (macOS can't passthrough GPU to containers) | `docker-compose.macos.yml` | M-series; Q4_K_M on ≤16 GB, Q6_K on ≥24 GB unified | | **Vulkan** (cross-vendor fallback) | Preview | `inference/Dockerfile.vulkan` → `atlas-llama-vulkan` | `docker-compose.vulkan.yml` | lavapipe CPU boot path (smoke-tested); no real-GPU validation yet | | **SYCL** (Intel Arc) | Roadmap — Intel Arc uses `vulkan` today | TBD | TBD | — | **Backend selection happens at install time, not runtime.** `atlas init` runs `tier.detect_gpu()` (see `atlas/commands/tier.py`), picks the largest-VRAM GPU across all detected vendors (override with `ATLAS_GPU_VENDOR` / `ATLAS_GPU_INDEX`), and writes `ATLAS_BACKEND={cuda|rocm|metal|vulkan}` into `.env`. Detection resolves to the packaged native backend when one exists: CUDA for NVIDIA, ROCm for AMD on x86_64, the hybrid Metal path on macOS. When no native backend is packaged for the host (Intel Arc, AMD on arm64, unrecognized vendors), the wizard offers the Vulkan universal fallback (default-yes): one image covers AMD, Intel, Adreno, MoltenVK, and the lavapipe CPU rasterizer, at roughly 20–40% below a tuned native backend. It refuses — rather than writing a `.env` that won't boot — only when nothing usable exists. Each backend has its own image — CUDA and Vulkan prebuilt on GHCR, ROCm compiled on the AMD host at first `up` — so users don't run a fat image that ships every backend's libraries. **Bring-your-own-model surface (V3.1.1).** `atlas lens check` is a cheap pre-flight against a running llama-server that reports whether the loaded model is Lens-compatible. `atlas lens build --samples ` wraps `geometric-lens/geometric_lens/training.py` to train fresh C(x) (`cost_field.pt`) **and** G(x) (XGBoost) artifacts at the model's native embedding dim. Together they let users swap in non-default GGUFs without forking the lens code — the C(x) constructor accepts arbitrary `input_dim`, so the only thing that changes per-model is the trained weights. See [CLI.md § atlas lens](CLI.md#atlas-lens) for the user-facing flow; `atlas lens publish` (or the combined `atlas publish`) uploads the artifacts to HuggingFace and opens the registry PR that pins their hashes. **What's vendor-agnostic** (works on every backend): grammar-constrained JSON, self-embeddings (`/embedding`), per-layer hidden states, ASA control vectors (loaded by llama.cpp's `control_vector_load` regardless of backend), KV cache quantization, the entire outer agent loop, V3 pipeline, Geometric Lens, and sandbox. **What differs per backend:** - **Flash attention.** CUDA + ROCm: full support. Metal: limited (llama.cpp Metal backend supports flash-attn for some head sizes; defaults to off if unsupported). Vulkan: driver-dependent. - **Pinned host memory.** `GGML_CUDA_NO_PINNED` applies to CUDA + ROCm (HIP mirrors the CUDA path at the GGML compat layer). Metal/Vulkan don't use the CUDA/HIP pinning path. - **Multi-GPU + tensor parallelism.** V1 supports single-GPU only on every backend; multi-GPU is GH #34, not bound to a specific vendor. - **Apple unified memory.** macOS shares GPU+system memory; "VRAM" math is actually "16 GB total minus OS + apps." See §7. The K3s deployment path (`scripts/install.sh`, manifests in `templates/`) is CUDA-only as of V3.1.1 — ROCm K8s recipe is deferred to the V3.2 infra list (needs `/dev/kfd` + `/dev/dri` hostPath mounts and `render`/`video` group membership, the cluster-level equivalents of `docker-compose.rocm.yml`). --- ## 2. Services | Service | Port | Language | Purpose | |---------|------|----------|---------| | **llama-server** | 8080 | C++ (llama.cpp) | LLM inference (CUDA / ROCm / Metal / Vulkan; SYCL on roadmap — see §1.1), grammar-constrained JSON, self-embeddings, per-layer residual hidden states | | **atlas-proxy** | 8090 | Go | Agent loop, tool-call routing, tier classification, `/v1/agent` SSE, `/events` typed SSE, `/cancel`. `/v1/chat/completions` forwards to llama-server with only a `max_tokens` clamp applied (see API.md). | | **atlas-tui** | (client) | Go | Bubbletea TUI; consumes `/events` and `/v1/agent` SSE streams. | | **v3-service** | 8070 | Python | V3 pipeline HTTP wrapper (PlanSearch, DivSampling, PR-CoT, etc.) | | **geometric-lens** | 8099 | Python (FastAPI) | Internal `/internal/*` scoring service: C(x) energy scoring, G(x) XGBoost quality prediction, per-step scoring, plus the pattern cache (read + write); owns the SQLite state store (`SQLITE_DB_PATH` on the `lens-state` volume) backing the pattern cache and co-occurrence graph | | **sandbox** | 30820 (host) / 8020 (container) | Python (FastAPI) | Isolated code execution, compilation, linting, test running | --- ## 3. atlas-proxy (Outer Layer) The proxy is the entry point for chat front-ends. It accepts user messages on `/v1/agent` (typed event stream — what the TUI uses) and runs an internal agent loop that calls llama-server, parses tool calls, executes them, and streams events back. The `/v1/chat/completions` endpoint is a transparent passthrough to llama-server; it is kept for SDK compatibility and does not run the agent loop. See [API.md](API.md) for the full event-type catalogue. The proxy is 12 Go files, one concern each: | File | Owns | |---|---| | `main.go` | HTTP server, routes, auth, passthrough, error envelope, private-value log filter | | `agent.go` | The agent loop: turn state, LLM calls, plan generation, pattern-context injection, stuck-loop breakers | | `tools.go` | The 14 tool definitions + executors, tier classification, tool-call grammar | | `gates.go` | Honesty/plan gates: claim-check, structural, syntax, embedded-script, plan-adherence, plan-reminder, asset lint | | `detectors.go` | Stuck-pattern detectors: tool repetition, reasoning repetition, traceback localization | | `context.go` | Context enrichment: symbol index, project scan, workspace containment, session file manifest | | `permissions.go` | Permission gate (`/v1/permission`), trust mode, hard-blocked patterns | | `lens.go` | Lens scoring calls, lens-sample banking (`/feedback`), calibration status | | `guardrails.go` | Per-tool steering guards (shrinkage, missing-command/module steers, doctype strip) | | `events.go` | Typed-envelope broker (`/events`) and SSE plumbing | | `v3_bridge.go` | SSE client for v3-service `/v3/generate` + `/v3/plan` | | `types.go` | Shared types, tiers, turn caps | ### Agent Loop Flow ```mermaid flowchart LR Start["User msg"] --> Build["Build prompt"] --> Call["llama-server"] --> Parse["Parse JSON"] Parse --> Route{Type?} Route -->|"tool_call"| Tier{"T2?"} Tier -->|"Yes"| V3["V3 Pipeline"] --> Result["Append result"] Tier -->|"No"| Exec["Execute tool"] --> Result Result --> Budget{"Budget?"} Budget -->|"< 4"| Call Budget -->|"4"| Warn["Nudge: write now"] --> Call Budget -->|"5+"| Esc["Escalated nudge"] --> Call Route -->|"text"| Stream["Stream"] --> Call Route -->|"done"| Done["End"] style Start fill:#1a3a5c,color:#fff style Done fill:#333,color:#fff style V3 fill:#2d5016,color:#fff ``` ### Grammar Enforcement Every model output is constrained toward one of three valid JSON shapes: ```json {"type": "tool_call", "name": "", "args": {...}} {"type": "text", "content": ""} {"type": "done", "summary": ""} ``` In the default `strict` mode the proxy sends a full JSON schema — `oneOf` with `additionalProperties: false`, tool names enumerated from the registry — which llama-server enforces as a grammar during token generation. Grammar constraints make malformed output rare, not impossible: `ATLAS_GRAMMAR_MODE=loose` sends `{"type":"json_object"}` only (valid JSON, no shape enforcement — some models require it), and the response token cap can truncate mid-JSON. The proxy treats parsing as fallible — it recovers JSON from prose/`reasoning_content`, detects truncated tool args before execution, feeds targeted parse-failure descriptions back, and breaks the loop after three consecutive failures. ### Tools 15 tools registered in `proxy/tools.go`: | Tool | Purpose | Read-only | |------|---------|-----------| | `read_file` | Read file contents (with optional offset/limit) | Yes | | `outline_file` | List a file's top-level functions/classes with line ranges, no bodies (tree-sitter for `.py`, best-effort scan otherwise). The surgical-read entry point: outline first, then `read_file` with offset/limit | Yes | | `write_file` | Create a NEW file (rejected for existing files >5 lines — see safety limits) | No | | `edit_file` | Surgical inline string replacement (old_str/new_str) for ≤10-line changes | No | | `insert_after` | Insert new lines after a given line number — the line numbers `read_file` prints. For ADDING code (a branch, function, import) where nothing existing changes: there is no `old_str` to reproduce, which is the step that fails on long spans | No | | `structural_edit` | Whole-function/class/HTML-element rewrite via tree-sitter selector (`function:NAME`, `class:NAME`, ``); REQUIRED over edit_file for whole-node swaps. GH #39, .py/.html/.htm only in v1 | No | | `delete_file` | Delete file or empty directory (forces loop exit after) | No | | `move_file` | Move or rename a file within the workspace (e.g. `index.html` → `templates/`). Pure relocation — bypasses the V3/surgical-edit gate, refuses to clobber an existing destination. The supported path for "reorganize the files" since shell `mv`/`cp` are refused | No | | `find_file` | Regex search by file **name** / path (cheap existence + locate). Distinct from `search_files` which greps inside file contents. | Yes | | `search_files` | Regex search across file contents (max 200 matches, skips .git/node_modules) | Yes | | `list_directory` | List directory contents with type and size | Yes | | `run_command` | Execute shell command via sandbox container; 5 min timeout cap | No | | `run_background` | Start a long-running process (e.g. `python app.py`) in the sandbox; returns a `job_id` immediately | No | | `tail_background` | Fetch new stdout/stderr from a backgrounded job by `job_id` | Yes | | `stop_background` | SIGTERM/SIGKILL a backgrounded job by `job_id` | No | ### Tool-selection bias mitigations A measured reference deployment showed a bias toward `edit_file` over `structural_edit` even when `structural_edit` was correct (BiasBusters arxiv 2510.00307 — embeddings of nearby tool names compete; descriptions matter more than names). Four model-independent defenses compose in the proxy: 1. **Description rewrite** (`proxy/tools.go`). edit_file's description warns against whole-file/whole-function use; structural_edit's description says REQUIRED for >10-line / whole-node swaps; write_file's says NEW files only. 2. **Conditional GBNF grammar** (`proxy/tools.go`, `proxy/agent.go:stepExclusions`). When a write_file is rejected on an existing .py/.html/.htm file >5 lines, the next LLM call is constrained by a GBNF grammar that bans edit_file and write_file from the tool-name production. The model physically cannot emit them. Restriction expires after one decision. 3. **Per-step tool-list filter** (same trigger). An ephemeral `[system note]` user message is injected reminding the model that structural_edit is the only structural-edit tool for this step. 4. **ASA steering vectors** (`geometric-lens/asa_calibration/`). Activation steering shifts the residual-stream distribution upstream so structural_edit is preferred even on first-attempt decisions before any rejection has fired. Auto-loaded by `inference/entrypoint-v3.1.sh` from `/models/ast_edit_steering.gguf` only when its `.model` sidecar matches the selected model—always-on after a compatible build via the workflow in `geometric-lens/asa_calibration/README.md`. Override path/scale/ layer-range via `ATLAS_CONTROL_VECTOR*` env vars. **Per-model coupling.** Each ASA vector is trained against a specific model's residual-stream geometry. No cross-model fallback is safe. `atlas asa check` verifies the `.model` sidecar, probes the loaded embedding dimension, parses GGUF layer metadata, and reports `compat` / `needs-build` / `incompatible`. `atlas asa build` derives the extraction layer from the loaded model, writes the vector and marker, and runs inside the lens container. `atlas asa publish` refuses missing or mismatched markers before upload. See [CLI.md § atlas asa](CLI.md#atlas-asa). ### Per-File Tier Classification Each `write_file`/`edit_file` call is classified independently: | Tier | Max Turns | Action | |------|-----------|--------| | T0 (Conversational) | 5 | Text response only | | T1 (Simple) | 0 (uncapped) | Direct write — no V3 overhead | | T2 (Feature) | 0 (uncapped) | V3 pipeline fires | | T3 (Hard) | 0 (uncapped) | V3 pipeline fires | The two columns above belong to two different classifiers, and the table reads as one only by coincidence. **Turns** comes from the message tier (`proxy/agent.go:classifyAgentTier`), which scores what the user typed. **Behavior** comes from the file tier (`proxy/tools.go:classifyFileTier`), which scores the file being edited and is what actually gates V3 — the message tier is forwarded to v3-service but only lands in a log line. Because the turn cap is the same for T1/T2/T3, the message tier has exactly one decision to make: conversational or not. T0 caps at 5 turns and skips Plan Mode; every other value behaves identically. It therefore requires positive evidence to call something conversational — a sub-12-character greeting or a question shape — and treats everything else as work. The asymmetry is deliberate: misreading conversation as work costs one wasted planner call, while misreading work as conversation caps a real request at 5 turns and fails it. Tier caps are 0 (uncapped); the detector stack inside the loop decides when to break: lens regression (`agent_lens_intervention`), reasoning repetition (`agent_reasoning_intervention`), tool-call repetition (`agent_repeat_intervention`), path-aware error breaker, done-without-action gate, claim-check gate, plan adherence threshold, and the empty-response fallback. Operators can override with `ATLAS_MAX_TURNS=` for one-off "fix the entire app" prompts — see `proxy/types.go::envOverrideMaxTurns`. Two of those gates decide whether to fire from what the run observed rather than from how the request was worded, because request wording is an open vocabulary that no list completes: - **Verification gate** — blocks `done` when the user asked for a repair, *or* when a test or build command actually exited non-zero and nothing has passed since. The second condition catches a failing test the model introduced on its own, which no reading of the user's message could have predicted. - **Done-without-action gate** — blocks `done` when the request carries explicit action wording, *or* when the model opened the project on a non-conversational message and nothing landed on disk. That covers verbs absent from the intent list (`remove the debug logging` matches none of them), while questions stay exempt: they are conversational, and answering one by reading files and writing nothing is correct. Classifier in `proxy/tools.go` (`classifyFileTier`); logic-pattern matcher in the same file (`hasLogicIndicators`). **Always T1 (direct write):** - Config files matched by name (e.g. `package.json`, `go.mod`, `pyproject.toml`, `dockerfile`, `docker-compose.*`) - Data files by extension (`.json`, `.yaml`, `.yml`, `.toml`, `.csv`, `.xml`, `.env`) - Style files (`.css`, `.scss`, `.less`) - Documentation (`.md`, `.txt`, `.rst`) and shell scripts (`.sh`, `.bash`) - Trivially-tiny files under **10 lines** (V3 has nothing to meaningfully diversify on at that size) - Unknown extensions with no logic indicators The exact config-file list and extension sets live in `proxy/tools.go:classifyFileTier`. **T2 (V3 pipeline)** — file qualifies if it's ≥10 lines AND either: - `hasLogicIndicators(content)` returns true — **2+ matches** across pattern families covering function/method definitions, control flow, error handling, Flask/FastAPI/Django routing, Express/Node API, React state/data, validation, database calls, JSX/React component patterns, and imports (the literal token list is in `proxy/tools.go:hasLogicIndicators`) - OR the file has a recognized source-code / markup extension (`.py`, `.go`, `.rs`, `.ts`, `.tsx`, `.js`, `.jsx`, `.html`, `.htm`, …) and no logic indicators fired — gets the benefit of the doubt at T2 (covers minimal-but-real files like a 12-line component shell) **T3 (Hard)** — currently classifier never emits T3 by itself; the cyclomatic-complexity refiner (`refineTierWithCC` via GH #39 point 2's `/internal/cyclomatic_complexity`) *escalates* on McCabe CC: to T2 at CC ≥ 8 (including from T1) and to T3 at CC ≥ 16. Never downgrades. ### Plan Mode (per-turn pre-flight) Plan mode is a pre-flight planning step that runs once per agent turn before the first tool call: the planner samples candidate plans, scores them heuristically, and renders the winner into the system prompt, where an adherence gate auto-revises when the model thrashes off-plan. It cuts discovery thrashing and blocks no-evidence `done` by guarding on the plan's verify step. See [PLAN_MODE.md](PLAN_MODE.md) for the full flow, components, tunables, skip conditions, cost, and testing matrix. ### Safety Limits Operator-facing limits and the knobs that tune them. Internal steering guards (traceback localization, missing-module/missing-command/broken-inline-script/case-mismatch steers, symbol grounding, no-op/empty-content/syntax gates, doctype strip) live in `proxy/guardrails.go` and `proxy/agent.go`; the structural gate (refuses a `.py` write that introduces an unresolved direct call — a would-be `NameError` — on `edit_file`, `structural_edit`, and every `write_file` branch; under BypassV3 only the non-iterating T0/T1 direct `write_file` skips it, so the demo baseline pane shows the raw model, while the edit paths and the iteration fast-path stay gated in all modes) lives in `proxy/gates.go`. The embedded-script gate lives there too: it parses the JavaScript (and brace-balances the CSS) inside `