ScienceFlow Selection and Observability

1. Structured run artifacts

Path	Producer	Purpose
`task_logs/lhr_stage_performance.csv`	LNR Stage capture and adjudication	Candidate rows, metric evidence, validity, selection eligibility, artifact identity, Gate fields, and worker/ESTRA context.
`task_logs/lhr_events.jsonl`	LNR state-machine store	Append-only run, Stage, ESTRA, resume, and merge events.
`task_logs/resource/resource_events.jsonl`	Unified resource store	Admission, lease, queue, review, advisory, arbiter, strict-gate, timebox, and execution events.
`task_logs/state.json`	ParallelRunner/finalizer	Final status and elapsed/resume-budget accounting.
`workers/w*/logs/evaluator_events.jsonl`	Gate-owned evaluator manager	Artifact SHA, backend/status, metric, validity, readiness, selection eligibility, and policy trace.
`workers/w*/logs/scienceflow_time_trace.csv`	Orchestrator trace callback	LLM timing, model, token, cache, and cost analysis.
`workers/w*/logs/interaction.log`	Agent interaction logger	Human-readable agent and tool activity.
`workers/w*/snapshots/`	SnapshotStore	Validated source and artifact state, with content-addressed storage for large objects when enabled.

2. Evaluator and Gate evidence

The evaluator produces a normalized MetricEvent. The Gate policy then decides whether that evidence is acceptable for Stage creation. The event trace preserves both fact generation and policy decision so later analysis can distinguish evaluator failure, policy rejection, duplicate candidate suppression, and a successful Stage transaction.

Evidence group	Representative fields
Identity	`candidate_id`, `stage_id`, `worker_id`, `artifact_path`, `artifact_sha`, `artifact_kind`
Metric	`metric_name`, `metric_value`, `lower_is_better`, `metric_validity`
Readiness	`validation_ok`, `candidate_ready`, `selection_eligible`
Evaluator	`evaluator_backend`, `evaluator_status`, runtime and output-tail metadata
Gate	Policy name/version, parameters, trigger, accepted/retry/reject result, reason code

Gate-time metric validity and post-commit Stage audit validity are separate fields. A later audit may lower selection confidence without rewriting the historical Gate decision.

3. Score summary and effective Stage facts

solver/lnr/stage/score_summary.py normalizes metric direction and computes raw best, valid best, worker summaries, and selection context for ESTRA, resource review, and finalization. Selection uses effective Stage facts: the append-only ledger preserves the research narrative, while snapshot source events can update validity, eligibility, and reason codes used by current decisions.

A numerically best metric is not automatically the valid best.
Missing direction metadata blocks selection when the evaluator contract requires direction.
Invalid schema, evaluator failure, and a valid zero baseline remain distinct states.
Artifact SHA and workspace-relative paths are preferred over mutable filenames.

4. Global merge outputs

solver/lnr/global_merge/ builds a candidate evidence pack from validated worker stages, exposes those candidates to a merge workspace, asks the live owner agent to construct the required final artifacts, and evaluates every finals/final_* directory through the configured evaluator. Structured fallback can promote ranked valid candidates when the agent does not produce enough finals.

merge/
├── global_merge_workspace/
├── submission_snapshots/
├── finals/
│   ├── final_00/
│   ├── final_01/
│   └── final_02/
└── global_merge_manifest.json

The default configuration requests three finals and allows at most three. The manifest records the required, produced, and valid counts plus fallback sources and evaluator results.

Runs using final_artifact_mode: best_stage instead write merge/best_stage_manifest.json and materialize the selected Stage under merge/finals/final_00/.

5. Monitor surfaces

Surface	Use
`scienceflow monitor --manifest ...`	Live multi-task Rich dashboard assembled from Stage, resource, ESTRA, trace, process, and final-state files.
`scienceflow monitor --log-dir ...`	Single-task compatibility view from a monitor-state path.
`scienceflow monitor-trace -m ... -o ...`	Self-refreshing HTML trend view with optional JSON cache.
`scienceflow resource-summary <root>`	Run-level aggregation of resource events as a table or JSON.

The monitor distinguishes unique Stages from candidate rows. Stage totals are deduplicated by worker and Stage id, while readiness and eligibility counts retain candidate-row semantics. Resource kill_req records a requested/approved review outcome; kill_exec records an actual process termination.

6. Operational interpretation

Use Stage CSV and evaluator events together when determining why a candidate was or was not selectable.
Use stable snapshots and merge finals for artifact comparison; temporary trial directories are not final evidence.
Use task_logs/state.json for completed or manually stopped elapsed time rather than stale process observations.
Keep offline experiment reports outside the canonical scripts/ entry-point set.