Clark Labs · field report
The operating system
for every voice.
A plain-English record of how the lab moved from one hard question to a working research system. Every step shows the scale, the parallel work, and what the autonomous scientists actually decided.
Read the chronology Verified work · July—August 2026
101.1Mstarter model parameters
254governed data sources
5,323clean-speech benchmark files
1Mlatent persona coverage audit
HOW TO READ THIS
One question. Many workers. One evidence trail.
We do not call a plan a result. The next step starts only after the last step leaves a receipt.
01Ask
The Scientist turns an idea into a testable question.
02Build
The Engineer writes the smallest useful experiment.
03Run in parallel
Workers test data, models, rooms, and failure cases at once.
04Try to break it
Evaluators compare the result and can stop promotion.
NOTEBOOK ENTRY 00 · METHOD
Notes from the working laboratory.
The pages below are arranged like a research journal: observation first, method second, evidence third, and the limit of the claim last. The language is simple on purpose.
“
No result without a trace. No trace without a question.
FIG. 01 · EVENT FIELD
Nodes are speech events. Lines are evidence links. The field keeps uncertainty visible.
THE CORE IDEA
Speech is not text with timestamps. It is a continuous, social, physical signal.
01Waveform
02Activity
03Language
04Speakers
05Identity
06Provenance
THE RESEARCH LOG
What happened, in order.
10 completed steps · the final step is the next experiment
Hear the whole world — not just the words.
OmniVoiceGraph began with a harder question: can one system understand speech, speakers, overlap, identity uncertainty, language, timing and provenance together?
Conventional ASR collapses a living scene into text. It loses who spoke, when they spoke, which voices overlapped, whether a voice is known, and whether the signal was real, replayed or generated.
Represent every recording as an event graph: continuous speech activity, exact timing, multilingual text, local speaker slots, open-world identity states, quality and signal origin — all with calibrated uncertainty.
EVIDENCE LEFT BEHINDspeech → events → people → provenance
A formal event-graph target instead of a transcript-only target
Unknown, ambiguous and insufficient evidence became valid outputs
Identity was separated from acoustic similarity and direct voice hashing
A transcript alone cannot say who spoke, whether turns overlap, or whether the sound is genuine.
Define an event graph G = (V, E, τ, s, p, u), where each event has time, speaker state, provenance, and uncertainty.
The target schema names speech activity, word timing, speaker slots, identity state, quality, and source origin as separate fields.
The graph is a measurement target, not evidence that a model can already recover every field.
WORKING NOTATION
G = (V, E, τ, s, p, u)First, build a system that can say “not yet.”
The foundation was rights-aware, reproducible and deliberately difficult to fool.
A universal speech model is only valuable if every dataset, label, split and claim can survive scrutiny. Public media is not blanket permission, and a good demo is not scientific evidence.
Build canonical manifests, immutable lineage, source-scoped speaker pseudonyms, protected evaluation splits and fail-closed acquisition gates around a raw-waveform multitask model.
EVIDENCE LEFT BEHIND254 sources / 101.1M parameters / zero silent rights bypasses
254 dataset records mapped across ASR, meetings, identity, age, spoofing and noise
91/91 baseline tests passed with 96 immutable workspace files verified
A 101,103,771-parameter shared-loop starter ran finite BF16 training on one GPU
Universal claims fail when data rights, labels, or evaluation splits are implicit.
Freeze manifests, source-scoped pseudonyms, protected splits, and an audio-to-label contract before training.
254 dataset records were mapped; 91/91 baseline tests passed; the shared-loop starter contained 101,103,771 parameters and completed finite BF16 steps.
A passing build proves reproducibility, not useful speech accuracy.
WORKING NOTATION
Nθ = 101,103,77136,023 examples. 125 hours. Invalid.
The first real-data treatment looked like progress — until the label geometry proved it was not.
Long audio was randomly cropped while whole-utterance transcripts were kept. Loss went down, but the model was being asked to align words that no longer existed inside the audio window.
Invalidate every affected checkpoint, preserve the failure as evidence, prohibit model selection from it, then rebuild a duration-safe corpus with matching audio and transcripts.
EVIDENCE LEFT BEHINDnegative evidence became infrastructure
23,894 misaligned rows were identified instead of normalized away
A corrected freeze produced 12,129 train rows / 25.82 hours and 5,063 validation rows
The project adopted a rule: semantic validity outranks a prettier curve
Random crops were paired with whole-utterance transcripts. The loss curve was therefore semantically misleading.
Compare crop duration with transcript duration, quarantine affected checkpoints, and rebuild rows with duration-safe alignment.
23,894 rows were misaligned; 12,129 corrected train rows covered 25.82 hours, with a separate validation freeze.
The corrected freeze fixes label geometry; it does not establish generalization to real rooms or unseen speakers.
WORKING NOTATION
audio(c) ↔ text(c), not text(full)Model the room continuously.
Speaker activity became the center of the architecture — not an annotation attached after transcription.
Turns are not clean blocks. People interrupt, overlap, whisper, trail off, return after silence and move through changing acoustics.
Create a multi-rate Speaker Activity Field: a 5 ms activity clock, 10 ms speaker clock and 20 ms linguistic clock, with VAD, overlap and speaker count as algebraic views of one latent field.
EVIDENCE LEFT BEHIND5 ms activity / 10 ms speaker / 20 ms language
A 54,266,538-parameter field model constructed successfully
143 tests passed; eleven coupled losses remained finite in the synthetic doctor
The receipt explicitly blocked VAD, diarization and quality claims until real platinum labels exist
Activity, overlap, speaker count, and quality are views of the same changing scene.
Use a 5 ms activity clock, a 10 ms speaker clock, and a 20 ms linguistic clock over one shared latent field.
The 54,266,538-parameter field model constructed successfully; 143 tests passed and eleven coupled losses stayed finite in the synthetic doctor.
Synthetic finite losses do not support VAD or diarization claims without platinum real labels.
WORKING NOTATION
Δt = {5, 10, 20} msMake the scientist auditable, too.
The experiment loop itself became a product: Clark Scientist proposes; Clark Engineer builds; deterministic workers execute; evaluators try to break the result.
Long-horizon research fails when prompts, actions, costs and exceptions disappear into chat. Autonomous science needs lineage as much as datasets do.
Route every model call through Clark Code headless, preregister plans and stop rules, record append-only trajectories, keep stable request IDs, and return only verified receipts to the next specialist.
EVIDENCE LEFT BEHINDintent → hypothesis → code → experiment → falsification
A typed Scientist → Engineer → GPU → evaluator → graph chain ran end to end
Provider calls were isolated from numeric executors and tagged clarkchat
Raw trajectories and artifacts remained remote while compact evidence synced locally
Long-horizon research becomes untraceable when prompts, actions, costs, and exceptions live only in chat.
Preregister the question and stop rule; route calls through Clark Code; append visible reasoning summaries, actions, usage, and terminal decisions as JSONL.
A typed Scientist → Engineer → GPU → evaluator → graph chain ran end to end with stable request IDs and remote artifact residency.
An auditable loop can still run a bad experiment; auditability is a control, not a scientific result.
WORKING NOTATION
receipt(n+1) only if receipt(n) is validScale personas before scaling audio.
The program explored a million-persona latent universe without pretending that procedural variation equals a million real people.
Cold-start training needs dense labels for timing, overlap, speakers, rooms, events and provenance. Real data rarely provides all of them together at pretraining scale.
Separate persona, renderer, source and acoustic evidence; generate conversations and scene recipes; enforce generator-disjoint splits; treat demographic and impairment-like controls as renderer factors, never observed truths.
EVIDENCE LEFT BEHINDmillions of worlds, zero invented identities
One-million latent-persona coverage and materialization controls passed
Million-persona factory logic was exercised under explicit storage and rights caps
Neural rendering remained gated until identity diversity and label integrity could be verified
Cold-start training needs labels for overlap, rooms, timing, and source origin that real corpora rarely provide together.
Factor persona, renderer, language, scene, and acoustic evidence; enforce generator-disjoint splits and keep rights provenance attached.
One-million latent-persona coverage and materialization controls passed under explicit storage and rights caps.
A latent persona is a controllable renderer condition, never proof of a real person or a global biometric identity.
WORKING NOTATION
scene = persona × renderer × room × eventBenchmark the best before training the next.
The system moved beyond Whisper as the default and built a pinned, timestamp-complete teacher benchmark on rights-cleared real speech.
A universal student cannot improve without knowing what today’s strongest open models can already do — and where they fail under exactly the same frozen inputs.
Run Qwen3-ASR 0.6B and 1.7B, Parakeet TDT 0.6B and Omnilingual candidates through Clark-gated, manifest-locked comparisons with full timing receipts.
EVIDENCE LEFT BEHINDsame audio / same split / pinned revisions / comparable evidence
5,323 LibriSpeech clean examples completed with 100% timestamp coverage
Qwen3-ASR 1.7B reached 2.14% normalized WER on test-clean
Qwen3-ASR 0.6B reached 2.66%; Parakeet reached 2.56% on the same split
A student cannot be judged without a pinned view of what strong open teachers do on exactly the same audio.
Run Qwen3-ASR 0.6B/1.7B, Parakeet TDT 0.6B, and other candidates over a manifest-locked test-clean split with complete timing receipts.
5,323 files completed with 100% timestamp coverage; normalized WER was 2.14% for Qwen3-ASR 1.7B, 2.66% for 0.6B, and 2.56% for Parakeet.
Clean speech is a calibration point, not a proxy for duplex rooms, codecs, or identity stress.
WORKING NOTATION
WER = (S + D + I) / NRooms bend both words and identity.
Hundreds of controlled scenes turned reverberation, noise and competing speakers into measurable failure surfaces.
A voiceprint that works in a clean clip can collapse in a meeting. A transcript that works near-field can fail when the room, codec or second speaker changes.
Render room sweeps, far-field conditions, environmental noise, neural-clone overlap and real-speech interference while preserving source, scene and speaker provenance.
EVIDENCE LEFT BEHINDoverlap is not noise — it is another person
384 room variants traced WER from 1.0% near-field to 15.3% in the extreme condition
Speaker nearest-reference accuracy fell from 99% near-field to 75% extreme
Across 392 scenes, overlapping real speech cut identity accuracy to 68.8% at 0 dB while non-speech noise held 96.9%
Distance, reverberation, noise, and a second speaker change both words and identity evidence.
Render room and duplex sweeps while holding source, speaker, scene, and label provenance fixed.
Across 384 room variants, WER moved from 1.0% near-field to 15.3% extreme; nearest-reference identity moved from 99% to 75%. Overlap at 0 dB reached 68.8% identity accuracy while non-speech noise held 96.9%.
Rendered stress reveals failure surfaces; it does not replace measurements from real microphones and rooms.
WORKING NOTATION
SNR = 20 log₁₀(rms(signal) / rms(noise))OBSERVED IDENTITY ACCURACY · ROOM SEVERITY
NEAR
99%
MID
88.5%
FAR
85.4%
EXTREME
75%
The clever policy lost to one fixed model.
A learned two-model router improved over Qwen in overlap — and still failed the only comparison that mattered.
Model disagreement looked like a useful routing signal on easier data. Overlap, rooms, Norwegian speech and G.722 exposed a distribution shift.
Freeze 92 source-disjoint duplex scenes, forbid language, room and identity leakage from router features, then compare Qwen, Parakeet, the router and an oracle with bootstrap uncertainty.
EVIDENCE LEFT BEHINDthe evaluator’s job is to kill the attractive story
Router WER 6.20 versus Parakeet 1.53 on held-out scenes
Large-room Norwegian Qwen/router behavior was catastrophic and clearly isolated
Promotion was rejected; the failure became a protected benchmark
A learned router can exploit easy distribution cues and collapse on a held-out environment.
Freeze 92 source-disjoint duplex scenes; forbid language, room, and identity leakage; compare Qwen, Parakeet, router, and oracle with bootstrap intervals.
Router WER was 6.20 versus Parakeet 1.53 on the held-out scenes; the large-room Norwegian/G.722 slice isolated the catastrophic shift.
The negative result speaks to this router and split, not to every future mixture-of-experts policy.
WORKING NOTATION
ΔWER(router, best) = +4.67PROMOTION REJECTED
Good science keeps the experiment that broke the story.
Faster after load. Still not truly incremental.
Native Qwen vLLM streaming beat cumulative-prefix polling on latency, then instrumentation showed why the next claim still had to wait.
A streaming API can appear incremental while repeatedly feeding accumulated audio. Fast output is not proof of efficient internal work.
Pair native and proxy paths on the same held-out rows, bootstrap latency and WER deltas, then trace every decoder call’s audio prefix and token metadata.
EVIDENCE LEFT BEHINDmeasure the mechanism, not the method name
48 paired rows: native post-load scene work 0.70 s versus 5.65 s for polling
WER difference was statistically indistinguishable; no accuracy claim was made
109 native calls exposed 11,861 receipt-level tokens — but not FLOPs or KV-cache truth
A streaming endpoint can look incremental while repeatedly processing the full audio prefix.
Pair native and cumulative-prefix paths on identical rows; reconcile audio prefixes, token deltas, timestamps, and call receipts.
For 48 paired rows, native post-load scene work was 0.70 s versus 5.65 s polling; WER was statistically indistinguishable; 109 native calls exposed 11,861 receipt-level tokens.
The receipts still do not expose FLOPs or prove KV-cache reuse, so mechanism promotion remains blocked.
WORKING NOTATION
latency = load + decode + transportA system designed to keep discovering.
OmniVoiceGraph is not a finished universal model. It is now a working research system that knows how to build evidence, reject weak claims and choose the next experiment.
The opportunity is larger than transcription: a trustworthy layer for understanding every human and synthetic voice inside real environments.
Scale rights-cleared real speech and high-fidelity synthetic scenes; train unified models; stress them adversarially; attach every result to provenance, uncertainty and a reversible identity graph.
EVIDENCE LEFT BEHINDthe next experiment begins where the last claim stopped
A data and model foundation spanning text, timing, activity, speakers and provenance
A remote autonomous lab with reproducible experiments and explicit stop rules
A client-ready path from raw audio to searchable, attributable, uncertainty-aware event graphs
The research system is useful when it can say exactly what remains unknown and choose a bounded next test.
Scale rights-cleared speech and difficult scenes, train unified models, attack them with protected evaluations, and attach every result to provenance and uncertainty.
The current foundation joins data, timing, activity, speakers, transcription, stress, trajectory, and graph receipts in one reversible program.
OmniVoiceGraph is not yet a universal model; the next claim must be earned by the next controlled experiment.
WORKING NOTATION
next = argmax(expected information gain)WHY IT MATTERS
Audio becomes infrastructure.
The end product is not a flashy demo. It is a careful layer that lets other systems understand sound.
Meetings
Who said what, when — including overlap and return speakers.
Media
Search across languages, voices, events, and provenance.
Agents
Give machines a structured, real-time view of human scenes.
Safety
Separate a real voice, a replay, a clone, and uncertainty.