CHIMERA

A physical-modeling voice synthesizer in the browser. Every sound comes from a simulated airway (lungs, vocal folds or syrinx, a branched vocal tract, lips, nose), not from samples. The engine is Rust, compiled to WebAssembly for an AudioWorklet; the same code runs natively for tests and the chimera CLI.

Vision & scope

1. One instrument: a physically modeled voice (lungs, vocal folds, throat, mouth). No samples, no neural audio.
2. The web app is the primary surface: anyone should be able to make songs in it without an LLM's help. Mobile is the likely next surface. The plugin (CLAP/VST3/AU) still ships, but it's secondary.
3. More realistic without losing what it is: the robotic, primitive sound people like today stays available, and every realism change is checked by ear against it.
4. Languages enter through one shared front-end design, not one-off hacks. Each needs a native speaker's check and must leave English unchanged.
5. The design language (the vector display) is settled; the look within it can still evolve.
6. Nothing copyrighted in git; MIT; a deterministic, real-time-safe engine.
7. Not now: a DAW or sequencer, a light theme, MPE, multi-voice dialogue.

What it does

• Play: drag the tongue, lips and jaw in a live cutaway of the head; vowel pad, encoders, computer keyboard and Web MIDI.
• Speak: English text (CMUdict with a letter-to-sound fallback) or IPA, turned into gestures and prosody, with the gestural score and a consonant chart on screen.
• Sing: a pitch roll with lyrics per note; consonants land before the beat. Lyrics can be English or Korean (Hangul: one block per note, with Korean sound rules); each language is a front end (docs/language-front-ends.md).
• Beatbox: pads and a step sequencer, every drum an articulatory gesture.
• Bestiary: songbird (two-voice syrinx), seal, alligator and elephant.
• Duet (?show=<id>): two 3D heads replay a pre-rendered song.

Male, female and child bodies blend continuously. Everything on screen animates from live engine state: pressure in the airway, airflow, the vocal folds, spectra and formants.

Run it

Needs Rust with the wasm32-unknown-unknown target, and Bun.

bun install
bun run dev        # builds the wasm, then Vite on 127.0.0.1:5317
bun run test       # wasm build, cargo test --workspace, bun test tests

The AudioWorklet needs a secure context, so open it on localhost or over HTTPS (for example tailscale serve). Vercel builds it with vercel.json.

Offline renders and measurement:

cargo run --release -p chimera-cli -- render speech|text|song|notes|beatbox|creature|vowels …
cargo run --release -p chimera-cli -- diagnose       # nasal leakage, formants, bandwidths, pitch
cargo run --release -p chimera-cli -- compare csd    # against a real singer (docs/csd-comparison.md)
cargo run --release -p chimera-cli -- compare tsp    # against 23 real speakers (docs/speech-comparison.md)
bun tools/spectrogram.ts in.wav out.png
bun tools/musicxml-to-notes.ts score.mxl P1 --out p1.json   # a score's part → `render notes` melody

Docs

• docs/architecture.md: the acoustic model (FDTD airway, sources, anatomy, gestures), and its references and credits.
• docs/rust-engine.md: the engine's real-time and determinism contract, and the browser boundary.
• docs/ui-redesign.md and docs/mockups/: the vector display look.
• docs/csd-comparison.md: scoring the voice against the Children's Song Dataset.
• docs/speech-comparison.md: scoring the speech against real speakers, phone by phone (the TSP speech database).
• docs/hill-climb/: the rules and log for tuning the voice one measured step at a time.

Agents: see AGENTS.md.

License

MIT. Bundled third-party files keep their own licenses: the CMU Pronouncing Dictionary (public/data/CMUDICT-LICENSE.txt) and the fonts (public/fonts/).