1. Runtime role
LNR is the primary long-horizon execution path in scienceflow/solver/lnr/. It maintains one or more persistent ScienceAgent workers within a wall-clock budget, preserves validated experimental states, periodically reassesses the active route, and produces a set of evaluated final artifacts.
2. Worker and workspace layout
The task root owns shared logs and resource state. Each worker receives an indexed execution root and a workspace containing the flat dataset view, source files, artifacts, memory, ledger, and snapshots.
task_root/ ├── task_logs/ │ ├── lhr_stage_performance.csv │ ├── lhr_events.jsonl │ └── resource/resource_events.jsonl ├── workers/ │ ├── w00/ │ │ ├── workspace/ │ │ │ ├── dataset/ │ │ │ ├── .run_results.md │ │ │ └── .agent_memory/stage_memory/ │ │ ├── snapshots/ │ │ └── logs/ │ └── w01/... └── merge/
lnr.init_workspace.workspace_path can copy ordinary files from a prior workspace before the new Git baseline is created. Runtime-control directories and the dataset view are not blindly copied. If present, the prior .logs/initial_workspace_state.md is included in the first-user context.
3. Candidate-to-Stage transaction
GateService is the authoritative entry point. In the default primary mode, exactly one evaluation outcome is required and only GateDecision.accepted=true may create a Stage. Missing artifacts, duplicate outcomes, unknown policies, invalid configuration, evaluator exceptions, and failed invariants fail closed.
stage_source_mode | Behavior |
|---|---|
primary | The Gate outcome is the sole Stage-admission decision. |
shadow | Write evaluator events for audit without changing Stage creation. |
adjudicate | Merge evaluator facts into an existing Stage record rather than making evaluator evidence the only source. |
4. Stage commit and hidden ledger
After the Gate accepts a candidate, the controller requests a short text-only Stage card. The temporary LNR_STAGE_COMMIT_REQUEST uses tool_choice=none; tool calls are rejected so bookkeeping cannot execute new work.
| Field | Rule |
|---|---|
STAGE_ID | Normalized Stage identifier. |
METRIC | Metric tied to the accepted evaluator event. |
METRIC_VALIDITY | Evidence quality used by selection and later audit. |
BRIEF / WHY | Concise result and rationale for the research trajectory. |
FILES | Required workspace-relative files. The controller does not infer this list from a full workspace scan or recent mtime. |
The append-only .run_results.md ledger is durable audit state and is not exposed as an ordinary agent-readable file. By default, the normalized Stage entry and simulated append result are projected into agent memory, while the temporary commit prompt itself is not persisted. If model-authored metadata remains malformed after retry, deterministic ledger-valid fallback metadata preserves the accepted evaluator evidence.
Optional settings support JSON Stage metadata, the active agent context, automatic tool selection, and a compact experiment-state view. The SciModelingBench profile enables these modes while the default text-only commit remains unchanged.
5. Snapshots and effective facts
solver/lnr/snapshot_store.py captures the source and artifact state associated with an accepted Stage. Workspace snapshots are enabled by default and verify content-addressed objects. Large weights or feature files can be stored once in the object store while compatibility views retain the expected selection/audit paths.
The ledger remains append-only. Decision-time consumers use effective Stage facts: snapshot source_event metadata may lower metric validity, change selection eligibility, or attach a reason code after the original commit without rewriting history.
6. ESTRA decisions
ESTRA periodically evaluates the current trajectory from structured Stage evidence, score summary, peer evidence, recent context, and bottleneck information. Each decision is made on two axes. The start point selects the execution base for the next research segment: the current workspace files (current_workspace), or a selected historical Stage snapshot (previous_stage). The intent selects the direction from that base: continue keeps deepening the current route when it is still reasonable, while redirect steers the next segment around the stated bottleneck when recent work has become shallow or repetitive. A redirect decision carries a short redirect_focus note naming the new focus.
| Start point | Intent | Normalized action | Effect |
|---|---|---|---|
current_workspace | continue | keep_current | Compact context and deepen the current route. |
current_workspace | redirect | keep_but_redirect | Keep files but refocus the next experiment. |
previous_stage | continue | switch_stage | Restore a named Stage snapshot, then continue from that state. |
previous_stage | redirect | switch_stage | Restore a named Stage snapshot, then refocus the next experiment from that state. |
When the two axes are combined, the start point dominates: any decision anchored at a previous_stage normalizes to switch_stage, while the recorded intent still steers how the next research segment is focused after the restore.
The default trigger interval is two accepted Stages. Context-limit and context-hygiene paths can also request ESTRA or deterministic compaction. Invalid decisions fall back to a safe current-workspace compact action rather than corrupting the run.
7. Stage-memory folding
solver/lnr/stage_memory.py builds a prompt-safe view from effective Stage cards. The raw ledger remains the source of truth; folding is a persistent cache for active context.
- If all Stage cards fit within
stage_memory_context_budget_chars(24,000 by default), render them directly. - Otherwise keep the most recent three cards raw and identify verification-critical target/best cards.
- Fold the older prefix into an
L-level summary stored under.agent_memory/stage_memory/. - If the view is still too large, fold summaries into a
G-level summary before clipping. - Persist summary identifiers, verification Stage ids, the current segment, and folding events for reuse and audit.
protected_eda_mode selects rule-based facts, the raw prefix, or an agent-generated EDA summary. Per-round runtime context can also expose the remaining wall-clock budget and effective Bash timeout.
8. Resume
| Level | Current behavior |
|---|---|
| Task resume | ParallelRunner reuses the task workspace and derives the remaining/fresh budget from task_logs/state.json. |
| Agent-memory resume | LNR reloads persisted ScienceAgent memory and chooses between a new LLM round and a pending tool continuation. |
| Tool-cursor resume | If the memory tail contains one assistant tool call without a result, the tool call is executed directly and its result is written before the next LLM turn. |
Resume events are audit records and are not appended as synthetic user messages. Multiple pending tail tool calls are detected and audited but are not automatically replayed. scienceflow replay-prepare can create an isolated reproduction for a selected pending call.
9. Multi-worker reduction and finals
Workers share task-level score and resource evidence but retain separate workspaces and agent memories. At finalization, global_merge collects selection-eligible Stage artifacts, creates a bounded merge workspace, and asks the live owner worker (W00 by default) to produce the configured number of distinct finals. Every final directory is evaluated through the same Gate-owned evaluator service, and structured fallback promotes ranked valid candidates when necessary.
Default selection settings enable merge, reserve up to 900 seconds for global merge, and request exactly three finals. The output is a set under merge/finals/final_*, not a single mutable winner file.
When merge is disabled, the opt-in final_artifact_mode: best_stage materializes the highest-ranked eligible Stage instead.
10. Resource integration
Each worker BashTool publishes job facts to the shared task-local resource runtime. LNR injects task and worker CPU lists, current budget context, resource feedback contracts, and resource observer hooks without moving research-route decisions into the control plane. Stage and score evidence are available to the resource arbiter as facts when it evaluates opportunity cost or a bounded proof window.