FOVEA

Overview

FOVEA is a CLI research framework for assessing whether a video is authentic or AI-generated. Instead of asking one model for a single opaque answer, it separates the process into evidence extraction, adaptive routing, domain-specific inspection, and explicit decision fusion.

Why FOVEA?

• Evidence-first workflow. FFmpeg/ffprobe metadata, sampled frames, hashes, and computer-vision measurements are prepared before model reasoning.
• Adaptive analysis. A planner can select only the analyst domains relevant to the current video.
• Hybrid evidence. Multimodal observations are paired with FFT, optical flow, geometry, ELA, boundary, edge, face, blur, phase, and feature-tracking signals.
• Parallel specialists. Selected style, physics, spatial, temporal, and watermark analysts run through LangGraph.
• Explicit fusion. A configurable judge supports strong-evidence override, confidence-filtered voting, and disagreement-aware weighted fusion.
• Reusable run records. Evidence caches and SQLite records make individual runs inspectable and experiments easier to resume or compare.

[!NOTE] “Training-free” means that FOVEA does not train or fine-tune a detector for each generator. It still relies on pretrained multimodal models at inference time.

Architecture

The diagram below shows the path enabled by the recommended configuration in this README. Human-Eyes screening, planner routing, CV skills, and the deterministic three-stage judge are individually configurable.

flowchart TD
    V[Input video] --> E[Evidence extraction and cache<br/>FFmpeg · ffprobe · SHA-256]
    E --> HE{Human-Eyes enabled?}
    HE -->|Yes| H[Three key frames<br/>spatial · watermark · physics · style]
    HE -->|No| P
    H --> X{High-confidence<br/>fake signal?}
    X -->|Yes| O[Verdict and run record]
    X -->|No| P{Planner routing enabled?}

    P -->|Yes| R[Select a relevant analyst subset]
    P -->|No| C[Use the configured analyst set]

    subgraph A[Selected specialists run in parallel]
        S[Style<br/>FFT]
        PH[Physics<br/>Optical flow · Geometry · NSG-Lite]
        SP[Spatial<br/>ELA · Patch · Boundary · Edge · Face · Blur]
        T[Temporal<br/>Local phase · Feature tracking]
        W[Watermark<br/>Visual text and logo reasoning]
    end

    R --> S
    R --> PH
    R --> SP
    R --> T
    R --> W
    C --> S
    C --> PH
    C --> SP
    C --> T
    C --> W

    S --> J{Judge}
    PH --> J
    SP --> J
    T --> J
    W --> J
    J --> O
    O --> DB[(SQLite persistence)]

If workflow_analysts is empty, the workflow falls back to a direct visual judge.

Analysis domains

After analyst selection, each non-watermark analyst can perform a second, content-aware routing step to choose only the CV sub-skills relevant to the current video. The recommended configuration exposes up to four preview frames to this router.

Each analyst returns a score_fake, three confidence components, a rationale, and structured evidence items. FOVEA uses a conservative weakest-link aggregation:

confidence = min(clarity, model–CV agreement, input quality)

These values are decision signals, not calibrated probabilities.

Decision protocol

With judge_dynamic.enabled: true, the default cascade is:

s_w     = Σ(c_a · s_a) / Σ(c_a)
sigma_w = sqrt(Σ(c_a · (s_a - s_w)^2) / Σ(c_a))
s_final = clip(s_w - lambda · sigma_w, 0, 1)
c_final = mean(c_a) · (1 - sigma_w)

When Human-Eyes screening is enabled, it inspects the first, middle, and last valid frames. An early exit occurs when any of its four dimensions reaches both score >= 0.80 and confidence >= 0.80.

Quick start

1. Prerequisites

• Python 3.10+ (3.11 recommended)
• ffmpeg and ffprobe available on PATH
• A vision-capable model endpoint that supports image inputs and structured output

Install FFmpeg with your platform package manager:

# macOS
brew install ffmpeg

# Ubuntu / Debian
sudo apt-get update && sudo apt-get install -y ffmpeg

# Windows with Scoop
scoop install ffmpeg

Verify the system dependency:

ffmpeg -version
ffprobe -version

2. Clone and install

git clone https://github.com/IatomicreactorI/FOVEA.git
cd FOVEA

python -m venv .venv

Activate the environment and install dependencies:

# macOS / Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

3. Configure environment variables

Copy the environment template:

# macOS / Linux
cp env.example .env

# Windows PowerShell
Copy-Item env.example .env

At minimum, set a writable SQLite path and the API key for the selected provider:

DB_PATH=DB/database.db
OPENAI_API_KEY=your_api_key_here

DB_PATH is required by the current implementation. FOVEA creates the parent directory and initializes the database on first use.

4. Create the runtime configuration

Runtime configurations live under src/config/. This directory is not tracked in the current repository, so create it before running any CLI command:

# macOS / Linux
mkdir -p src/config

# Windows PowerShell
New-Item -ItemType Directory -Force src/config

Create src/config/config.yaml:

experiment_name: fovea_default

human_eyes_enabled: true
planner_mode: true
enable_skills: true
workflow_analysts:
  - style
  - physics
  - spatial
  - temporal
  - watermark

judge_dynamic:
  enabled: true
  tau_ovr: 0.85
  gamma_ovr: 0.60
  gamma: 0.60
  lambda_penalty: 0.15

llm:
  provider: openai
  model: YOUR_VISION_CAPABLE_MODEL
  temperature: 0
  max_retries: 3
  max_images: 40
  skill_router_max_preview_frames: 4

video:
  frame_ext: jpg
  frame_quality: 2
  frame_mime: image/jpeg

concurrency:
  lock_wait_sec: 8.0
  lock_poll_sec: 0.25

batch:
  delay_sec: 0.1

Replace YOUR_VISION_CAPABLE_MODEL with a model available through your chosen provider. The endpoint/model must support both image inputs and structured output (function calling).

5. Add video data

Videos must be stored below DATA_ROOT. If DATA_ROOT is unset, the default is the repository’s data/ directory.

data/
├── Real/
│   └── example_real.mp4
└── Fake/
    ├── Sora/
    │   └── example_sora.mp4
    └── OtherGenerator/
        └── example_fake.mp4

Supported media discovery is recursive for MP4, MOV, AVI, MKV, WebM, FLV, and GIF.

For external datasets, point DATA_ROOT and CACHE_ROOT to writable directories:

DATA_ROOT=/absolute/path/to/videos
CACHE_ROOT=/absolute/path/to/fovea-cache

6. Extract and cache evidence

Conversion is required before analysis:

# Small smoke run
python -X utf8 src/convert.py --label all --limit 5

# Full conversion
python -X utf8 src/convert.py --label all

Conversion is resumable by default and skips videos with a ready evidence cache.

7. Run FOVEA

Use a path relative to DATA_ROOT:

# One video, human-readable summary
python -X utf8 src/main.py \
  --label Real/example_real.mp4 \
  --config config.yaml

# One video, JSON summary
python -X utf8 src/main.py \
  --label Fake/Sora/example_sora.mp4 \
  --config config.yaml \
  --json

# One category
python -X utf8 src/main.py --label Fake --config config.yaml

# One generator directory
python -X utf8 src/main.py --label Fake/Sora --config config.yaml

# Entire data root
python -X utf8 src/main.py --label all --config config.yaml

[!IMPORTANT] Pass only the configuration filename to --config; configuration paths are resolved inside src/config/. For example, use --config config.yaml, not --config src/config/config.yaml.

Output

For a single video, --json prints a compact run summary shaped like this:

{
  "run_id": "abc12345",
  "case_id": "UmVhbC9leGFtcGxlX3JlYWwubXA0",
  "case": {
    "case_id": "UmVhbC9leGFtcGxlX3JlYWwubXA0",
    "video_path": "/path/to/FOVEA/data/Real/example_real.mp4",
    "label": null
  },
  "results": {
    "style": {
      "agent": "style",
      "status": "ok",
      "score_fake": 0.0,
      "confidence": 0.0,
      "evidence_count": 0,
      "error": null
    }
  },
  "verdict": {
    "label": "real",
    "score_fake": 0.0,
    "confidence": 0.0,
    "rationale": "...",
    "evidence_count": 0
  }
}

The numeric values above illustrate the schema only. Detailed agent evidence, configuration, timing, and verdict records are persisted in SQLite. Export a complete run—including evidence arrays—with:

# Latest run to stdout
python -X utf8 src/export_result.py

# Specific run to a file
python -X utf8 src/export_result.py abc12345 --output result.json

Configuration

Behavior-changing keys

Provider adapters

The code contains adapters for the following endpoints. An adapter entry does not guarantee that every model on that endpoint is multimodal or supports structured output.

Environment overrides recognized by the configuration loader:

Evidence, persistence, and experiments

Evidence cache

src/convert.py creates an evidence bundle for each video under CACHE_ROOT/evidence/. The bundle includes sampled frames and probe metadata. Case IDs are derived from the video path relative to DATA_ROOT; metadata also records a SHA-256 digest.

Frames supplied to a multimodal call are uniformly downsampled when they exceed llm.max_images.

SQLite run records

The database stores:

• analysis runs and the full configuration used;
• selected analysts and per-agent scores;
• evidence items and agent errors;
• final verdicts and rationales;
• run and agent timing fields.

Experiment isolation

Always set an explicit, stable experiment_name. Batch analysis skips a video only when the database already contains a verdict under the same experiment name. Change the name for a new configuration or ablation.

experiment_name: fovea_no_cv_ablation
enable_skills: false

Summarize stored results with:

python -X utf8 src/statistics.py --list-experiments
python -X utf8 src/statistics.py --config config.yaml

Reported statistics include a confusion matrix, accuracy, per-class precision/recall/F1, confidence summaries, misclassified cases, and timing summaries when timing data exists.

CLI reference

Evidence conversion

python -X utf8 src/convert.py --label LABEL [options]

Analysis

python -X utf8 src/main.py --label LABEL [--generator NAME] [--delay SECONDS] [--config FILE] [--json]

--json affects single-video output. Batch runs print progress and aggregate counts.

Result inspection

python -X utf8 src/statistics.py [--config FILE] [--list-experiments]
python -X utf8 src/export_result.py [RUN_ID] [--output FILE]

Repository layout

FOVEA/
├── src/
│   ├── main.py                    # Analysis CLI
│   ├── convert.py                 # Evidence extraction and caching
│   ├── statistics.py              # Experiment metrics
│   ├── export_result.py           # Full JSON export
│   ├── agents/
│   │   ├── human_eyes.py          # Early-exit screener
│   │   ├── planner.py             # Content-aware routing
│   │   ├── judge.py               # Decision fusion
│   │   ├── analysts/              # Five analysis domains
│   │   └── routing/               # CV sub-skill routers
│   ├── skill/                     # Computer-vision measurements
│   ├── graph/                     # LangGraph workflow and state
│   ├── pipeline/                  # Analysis and evidence entry points
│   ├── llm/                       # Provider adapters and structured inference
│   ├── database/                  # SQLite schema and persistence
│   ├── apis/video_io/             # FFmpeg/ffprobe integration
│   ├── util/                      # Paths, config, logging, sampling
│   └── config/                    # Local runtime YAML files (create locally)
├── env.example
├── requirements.txt
├── QUICKSTART.md
└── OUTPUT_FORMAT.md

Troubleshooting

Project status

FOVEA is research code. Its heuristic scores depend on the configured multimodal model, decision thresholds, and input quality and should not be interpreted as calibrated probabilities. Video frames are sent to the configured model provider; review that provider’s data-handling terms before processing sensitive material.

The repository currently provides a CLI workflow. It does not include a web dashboard, packaged benchmark results, or a root license declaration.

Contributing

Issues and focused pull requests are welcome. For reproducible bug reports, include:

1. operating system and Python version;
2. the exact command;
3. a redacted configuration showing behavior-changing keys;
4. the relevant log excerpt and stack trace;
5. a minimal non-sensitive video sample or a precise reproduction description.

Never include API keys, private videos, or unredacted database files in an issue.

Citation

Canonical paper and citation metadata will be added when publicly available.