The gate loaded and worked on workpc and took mavwaked from 68% of one core
to 335%. onnxruntime sizes its intra-op pool to every core and spins between
runs, which an always-on gate scoring three graphs twelve times a second
provokes for the whole day. One thread per session brings it to 81%, so the
keyword costs about 13% of a core, and each graph still finishes well inside
its 80ms.
The unit now passes the three -wake- flags and the models sit beside
silero_vad.onnx in ~/.local/share/maven/models. The threshold is left at the
binary's default so there is one place to change it.
Over 65.1 minutes of held-out Common Voice the built binary woke three times
at 0.99 and once at 0.999. The recall difference was one render out of 126.
One render is worth two thirds of the false wakes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
keywordGate is an interface so the decision that ships an utterance can be
exercised with a fake that fires on demand. A gate that can only be tested
with a model file is a gate nobody tests.
The three that carry the fixed-when criterion: keywordless speech never
reaches STT, the keyword does, and barge-in still cuts her off mid-sentence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
Until now every utterance near the microphone became a turn. SurfaceVoice caps
acts at L0, which made that safe rather than expensive, but L0 does not cap
reading: the room could still hear his facts read back.
The gate sits at dispatch, not at the VAD. The keyword opens a window, the VAD
closes the utterance when he stops, and dispatch asks whether the window was
open. That ordering is what lets him say "Мэйвен" and then a sentence: the
window has to outlive the word by the length of what follows it.
One keyword buys one turn. A window that renewed itself on every reply would
leave the microphone open for as long as he kept talking, which is the state
this exists to end.
Her own voice cannot wake her. Every path above the gate returns while the
player is running, so no frame of her reply is ever scored, and the streaming
state is cleared when playback ends.
Nil is a working value. Without -wake-model the gate is open and this is
yesterday's mavwaked, which is what an operator with a missing file should get
rather than a daemon that refuses to listen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
melContext is 480 because melspectrogram.onnx returns N/160-3 frames and frame
i covers [i*160, i*160+400). With 480 samples of history the buffer is 8
frames and the oldest continues exactly one hop after the previous call's
newest. Less history leaves a gap.
Feed reports the threshold CROSSING, not the state. A keyword held above the
threshold for a second is one wake, and firing on every chunk of it would make
the gate look open when it is merely slow to fall.
Nil is the CLOSED gate rather than the open one. A nil that answers "yes,
keyword" reads as a working wake word in every log line it produces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
The two feature models are frozen and pretrained; only the 100KB head was
trained here. The shapes were measured rather than assumed: 2.0s of 16kHz
audio gives 197 mel frames, and 76-frame windows at stride 8 give exactly the
16 embeddings the head was fitted on.
This file knows tensors and nothing about the 80ms cadence, which is why the
scaling openWakeWord applies between the two feature models lives here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
It reaches the player, but not from the push goroutine. An unusable push
is dropped and does not wedge the next one. It waits for a reply to
finish. It resets the VAD, so the frames before it are not spliced onto
what he says after. And a second nudge replaces an unspoken first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
It wired no PushHandler, and SendRequest discards a push frame when there
is none. That was not a missing feature but a silent one. mavend routes a
nudge to the voice session that spoke most recently, so once mavwaked had
spoken once it WAS that session. PushToMostRecent succeeded, the
dispatcher counted the nudge delivered and stopped rerouting to the away
channels, and mavwaked threw the audio away. He heard nothing, anywhere.
It now connects at startup rather than at the first utterance, because
the dispatcher has to tell "he is not at the machine" from "he is, and
she has nothing to say". The receiver redials on its own clock, since
mavend restarts on every deploy.
A nudge is queued, not played where it arrives. The capture loop picks it
up on the next frame, so the half-duplex gate and barge-in cover it the
way they cover a reply. It resets the VAD first: playback is about to
suppress every frame, and a half-heard sentence would otherwise splice
onto whatever he says next. A nudge arriving while one still waits
replaces it, which is the contract internal/voice states for PushHandler.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
Speech is the four piper fixtures mavsttd already scores against, so nothing
of the owner's voice is committed. Non-speech is white noise at the same RMS
as the clip beside it.
Silero calls 0 noise frames speech where the energy threshold calls 68 to 99,
and hears all four spoken clips. 509us per 30ms frame, 1.7% of one core on
the slower machine.
White noise is a floor and not a proof. It says nothing about a television,
which is speech, or a fan, which is narrowband.
silero-vad replaces the energy threshold when -vad-model points at it.
Everything after the speech decision is the same state machine: the speech
hold, the silence hold, the length cap and the utterance buffer.
The model window is 512 samples and the capture frame is 480, so silero.go
re-chunks across frames. main.go claimed the two matched, which was true of
silero v4.
Stage two, the wake word, is not here. It needs a Russian keyword model that
does not exist yet.
Nothing reads the microphone while Send is in flight, so the audio piles
up in arecord's pipe and arrives in a burst the moment dispatch returns.
A round-trip is p50 2.7s through the LLM router, which is about 90
frames of room, of him finishing his sentence, of the television.
The old code reset the VAD on the reply path only, and for a reason that
was not true: the comment said the VAD had been accumulating during the
round-trip, when its state is exactly what Feed left it as. The two
paths with no reset are the ones that mattered, because neither starts
playback and so neither is covered by the half-duplex gate. A text-only
turn fed the whole backlog into the VAD, and a Send error did the same
on every failed turn, so a dead socket drove a retry loop off backlog
alone.
The backlog was scored for barge-in too. Five frames delivered in
microseconds cut her off with audio recorded before she started
speaking, which is the opposite of what the five-frame guard is for.
Both are fixed by the same mechanism: measure the wall time the
round-trip took, convert it to frames, and discard that many before
anything looks at them.
Barge-in also threw away the 150ms that proved he was talking. The VAD
started from the next frame, so the first word of a short interruption
was clipped before whisper saw it. Those frames are kept in a small ring
and replayed after the reset.
A stuck aplay was worse than before this feature existed. Playing()
gates all capture, so a wedged child made her deaf rather than silent,
for the full 30s ceiling inherited from the fire-and-forget version. The
mute window is bounded by the reply's own duration plus a margin now.
Three smaller ones. "-barge-in -barge-in-rms 0" logged "barge-in on" and
then did nothing. The sent counter incremented before the error check,
so failed round-trips counted as shipped. And the threshold the operator
has to guess is now reported: mavwaked logs the mean energy of the
frames it suppressed while speaking, so he can set it from data.
Found in review of #76.
Playback was `go playAudio(reply)` — fire and forget, nobody holding the
process handle. Two audible consequences fell out of that.
She answered herself. The capture loop kept feeding the VAD while the
speaker was running, so her own reply came back in through the mic,
tripped the VAD, and was shipped to the daemon as a fresh command. There
is no acoustic echo canceller in this pipeline, so the fix is
half-duplex: while she is speaking, the capture side is muted. That part
is unconditional — it repairs a defect, it is not a new capability.
And talking over her did nothing, because there was no handle to cancel.
-barge-in now cuts playback when sustained energy clears a room-tuned
threshold (-barge-in-rms, default 0.12 normalised, over -barge-in-frames
consecutive frames, default 5). It is off by default: without an echo
canceller the only way to tell "he is talking over her" from "the mic is
hearing her" is that he is much louder, and how much louder depends on
where the mic sits.
The frame decision moved out of main.go into session.feed, behind a
player and an utteranceSender interface, so all of it is testable with
no mic, no speaker and no daemon. Nine tests cover the self-hearing
case, the off-by-default case, the consecutive-frame requirement,
speaker-leak-level audio not triggering, capturing the interrupting
utterance after a cut, and failed round-trips not starting playback.
The other seven items on #287 (partial STT, per-segment retry, mic
profiles, noise-floor calibration, short-response-while-speaking) are
untouched and stay on the task.
New cmd/mavwaked — always-on voice listening client that:
- Captures PCM from arecord subprocess (16kHz mono int16)
- Runs energy-based VAD in 30ms windows (RMS threshold, adaptive floor)
- Buffers utterances (300ms min speech, 800ms silence end, 10s max)
- Sends complete utterances as PushToTalk with Surface=SurfaceVoice (L0)
- Plays reply audio through aplay subprocess
- No new CGo/onnxruntime deps — pure Go
- 10 VAD tests with -race (speech detect, silence, max duration, reset, adaptive floor)
- Makefile build-waked target + Dockerfile integration + alsa-utils runtime dep
claude
kami