ScienceFlow CLI and Configuration

1. Command surface

The package entry point is scienceflow = scienceflow.cli:main. The Click group loads repository and working-directory environment files and installs process-group cleanup handlers before dispatching a command.

Command	Purpose	Implementation
`prep`	Prepare a flat workspace dataset. It runs the data-preparation agent by default; `--link-only` exposes input entries as symlinks.	`solver/data_prep.py`, `solver/lnr/prep_fs.py`
`run`	Run the REPL-native long-horizon solver. The current supported type is `lnr`.	`core/orchestrator.py`, `solver/lnr/solver.py`
`repl`	Run an interactive ScienceAgent session from explicit paths or a single-task manifest, with optional automatic first turn.	`core/orchestrator.py`
`parallel`	Launch manifest tasks with bounded concurrency and per-task CPU/GPU environment isolation.	`core/parallel_runner.py`
`monitor`	Render a Rich dashboard from one task log directory or a parallel manifest.	`ui/monitor/`
`monitor-trace`	Build a self-refreshing HTML trend view from a parallel manifest.	`ui/monitor_trace/`
`resource-summary`	Aggregate one or more `resource_events.jsonl` files as a table or JSON.	`resource_runtime/unified_store.py`
`replay-prepare`	Copy a saved LNR run into an isolated replay directory and truncate memory to a selected pending tool call.	`solver/lnr/replay_prepare.py`

2. Configuration model

scienceflow/config/default.yaml is a small include file. Stable defaults are split by responsibility under scienceflow/config/defaults/, and LNR defaults are further split under defaults/lnr/.

Config
├── agent       # code and feedback model settings, routing, price metadata
├── exec        # debug and task CPU/GPU lists
├── workspace   # input root, task root, task identity, tracing
├── repl        # interactive profile, step budget, workspace Git behavior
├── tool        # sandbox, tool concurrency, output and memory limits
├── lnr         # workers, stages, ESTRA, folding, snapshots, resources, merge
├── evaluator   # candidate, metric, backend, command environment, event logs
├── gate        # trusted Gate policy and configurable parameters
└── profile_overrides

Configuration is represented by dataclasses in config/settings.py. A run starts from the default include graph or a custom overlay, then applies task profile and explicit manifest/CLI overrides. Task packages remain authoritative for their artifact, metric, evaluator, and Gate declarations.

3. Current default behavior

Setting	Default	Meaning
`lnr.wall_clock_budget_sec`	`3600`	One-hour task budget unless the manifest overrides it.
`lnr.max_steps`	`500`	Maximum LLM/tool rounds for a worker session.
`lnr.num_workers`	`1`	One persistent worker by default.
`lnr.omp_threads_cap`	`8`	Maximum OpenMP threads assigned to each worker CPU slice.
`lnr.estra_enabled`	`true`	Enable evidence-based continue, redirect, and previous-stage restoration decisions.
`lnr.stage_memory_folding_enabled`	`true`	Fold old stage cards into persistent summaries when the stage-memory view exceeds its budget.
`lnr.resource_control_mode`	`resource_smart_llm`	Enable the full resource runtime, admission LLM, arbiter LLM, owning-agent advisory, queue, and GPU sharing.
`evaluator.task_profile` / `backend`	`auto` / `auto`	Resolve the effective profile and evaluator from a registered task package.
`evaluator.stage_source_mode`	`primary`	Use the Gate outcome as the sole Stage-admission decision.
`repl.repl_profile` / `repl_tool_preset`	`lite` / `bash_write`	Use the compact REPL prompt and write files through bash rather than legacy write/edit tools.

4. Manifest shape

max_concurrent: 1
time_limit: 3900
resume: false

lnr:
  wall_clock_budget_sec: 3600
  num_workers: 2
  omp_threads_cap: 8
  resource_control_mode: resource_smart_llm

defaults:
  config: scienceflow/config/default.yaml
  workspace_base: /path/to/runs

tasks:
  - exp_id: nomad2018-predict-transparent-conductors
    run_id: nomad-example
    input_data_dir: /path/to/task/data
    cpu_list: "0-15"
    gpu_list: cpu
    lnr:
      seed: 2222
      resource_gpu_pool: []

ParallelRunner resolves a task workspace from explicit task fields or workspace_base/run_id/exp_id, applies task-level CPU/GPU boundaries, and passes the structured configuration patch to the child process. The maintained examples are in scripts/lnr.yaml, scripts/repl.yaml, scripts/prep.yaml, and scripts/lnr_two_tasks.yaml.

5. Evaluator and Gate configuration

The evaluator defaults are intentionally task-neutral. At runtime, TaskPackageSpec resolves the effective profile, candidate path, artifact kind, metric name and direction, evaluator entry point, and timeout from tasks/**/task.yaml.

Field	Current role
`evaluator.backend`	`auto` resolves registered tasks to `task_package`. `artifact_command` remains available for command-template integration.
`evaluator.candidate`	Defines a workspace-relative artifact path, kind, scan mode, and optional SHA requirement for command-based evaluation.
`evaluator.metric`	Defines the metric name, direction, parser fields, and whether selection requires a known direction.
`evaluator.command`	Defines the evaluator command, isolated Python/environment path, working directory, timeout, environment variables, and output tails.
`evaluator.query_budget_scope`	Selects a task-wide or worker-local evaluator query budget.
`evaluator.stop_on_query_budget_exhausted`	Stops research after an accepted evaluation uses the final available query.
`evaluator.expose_wall_clock_remaining_sec`	Adds the current task time budget to evaluator feedback when enabled.
`gate.policy`	Selects a trusted registered Gate policy. Runtime configuration may tune permitted parameters but cannot disable core invariants.

The built-in default policy requires validation, candidate readiness, a finite primary metric, required direction metadata, and sufficient metric validity. The opt-in optimization_feasibility policy adds a constraint-violation bound.

The optional scienceflow/config/sci_modeling_bench.yaml profile combines worker-local query budgets with runtime context, structured Stage commits, agent-generated EDA summaries, and best-Stage finalization.

6. Resource-control presets

Mode	Expanded behavior
`off`	Disables the resource runtime, review state, queue, arbiter, advisory, observation, and sharing.
`resource_smart_policy`	Enables deterministic runtime, queue, review state, and policy arbiter without admission LLM, owning-agent advisory, or GPU sharing.
`resource_smart_llm`	Enables the complete control plane, including admission LLM, LLM arbiter, owning-agent advisory, and GPU sharing.

Expert fields can override individual switches after selecting a coarse mode. The canonical expansion lives in scienceflow/config/resource_modes.py.