Ship voice enrolment, and report recognition as blocked (#255)
Maven can now be told who someone is. She cannot yet tell who is speaking, and this commit is careful to say so rather than pretend otherwise. What works: profiles are enrolled from several deliberately recorded samples, listed, and deleted. They live in the existing memory_vectors table under a "speaker:" id prefix, so there is no migration; what that needed was a wider interface than memory.Store, hence memory.Catalog with ByPrefix and Delete. Delete is the load-bearing half — a voiceprint someone asked to be rid of has to actually go, and a search-only store cannot do that. InMemoryStore.Insert became an upsert by id to match what the persistent store already did. What does not work, and why it is not faked: there is no speaker-embedding model on this box. Sixteen ggufs in /mnt/hdd1/llms, all text; no ECAPA, no x-vector, no titanet, no wespeaker, no .onnx anywhere under /mnt/hdd1. So newSpeakerEmbedder returns nil, internal/speaker falls back to speaker.Disabled, Identify answers ErrDisabled, and the daemon logs which half is off at startup. The plan's "simple MFCC + GMM" floor is refused in the package comment: MFCC cosine distance detects channel and loudness as much as voice, and a biometric that is confidently wrong writes false claims about named people into his memory. A bad floor is worse than none here. Refused as well, and the reason is in enroll.go's doc comment: the plan asked for unknown speakers to be enrolled on first interaction with a TTS "кто это?". There is no request shape in the protocol that could express that. Taking a biometric of whoever walks past the microphone does it to guests who are not party to the exchange, and a synthesised question into a room is not consent from whoever answers. Authority: enrolment is AuthStepUp, because it is a deliberate sit-down act that writes a biometric of a named person and never something done by voice mid-conversation. Deletion is one rung lower at AuthWrite, deliberately inverting the usual pattern — getting rid of a biometric must never be the harder half. Listing is AuthRead and never returns the vectors themselves. Off unless configured: no speaker block means the three methods answer ErrUnknownMethod, so a default box has no wire path that takes a voiceprint. make build and make test pass. Vikunja #255 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
This commit is contained in:
@@ -0,0 +1,223 @@
|
||||
// Package speaker is voice identification (Vikunja #255,
|
||||
// docs/plans/10-speaker-recognition.md).
|
||||
//
|
||||
// The shape is the same seam internal/vision uses: an Embedder turns audio into
|
||||
// a voiceprint, a Recognizer compares one against the enrolled profiles, and a
|
||||
// Disabled floor refuses politely when nothing is wired. On this box nothing is
|
||||
// wired, and that is the honest state — see "Blocked" below.
|
||||
//
|
||||
// # A voiceprint is not like the other vectors
|
||||
//
|
||||
// Everything else in the vector table is something he wrote or said. A speaker
|
||||
// profile is biometric data about a person, quite possibly a person who never
|
||||
// asked for Maven to exist. The rules that follow from that are in the code:
|
||||
//
|
||||
// - Enrolment is explicit and named. There is no "enrol the unknown voice
|
||||
// automatically" path; see the refusal in enroll.go.
|
||||
// - A profile is deletable, individually, and Forget really removes the row.
|
||||
// - Below the threshold the answer is "I do not know", never the closest
|
||||
// guess. A misattributed fact is worse than an unattributed one.
|
||||
// - Nothing here gates whether Maven listens or answers. Identification
|
||||
// annotates a turn; it never authorises one, and an unrecognised voice is
|
||||
// not turned away.
|
||||
// - Voiceprints never leave the box. They live in the encrypted store with
|
||||
// everything else and are never search input to anything external.
|
||||
//
|
||||
// # Blocked
|
||||
//
|
||||
// There is no speaker-embedding model on this box: no ECAPA-TDNN, no x-vector,
|
||||
// no wespeaker or titanet ONNX anywhere under /mnt/hdd1 or models/ (checked
|
||||
// 2026-08-01; the only ONNX files are the e5 text embedder and the piper voice).
|
||||
// There are also no enrolment samples. So Recognizer runs against Disabled and
|
||||
// every Identify answers ErrDisabled until a model lands.
|
||||
//
|
||||
// The MFCC + GMM "simplest floor" in the plan document is refused rather than
|
||||
// deferred. A hand-rolled spectral distance would identify people confidently
|
||||
// and wrongly, and its output would be written into facts as "Ками said this".
|
||||
// For a biometric, a bad floor is worse than none: no answer is honest, and a
|
||||
// wrong answer is a false memory about a person.
|
||||
package speaker
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"math"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
)
|
||||
|
||||
// Prefix — the id prefix speaker profiles carry in the shared vector table.
|
||||
// It is what ByPrefix enumerates and what keeps voiceprints out of note recall.
|
||||
const Prefix = "speaker:"
|
||||
|
||||
// DefaultThreshold — cosine similarity a match must beat to be a match.
|
||||
//
|
||||
// 0.7 is the usual operating point for ECAPA-style embeddings on clean speech
|
||||
// and it is deliberately on the strict side here. The two error directions are
|
||||
// not symmetric: refusing to name a voice costs a "не узнала", while naming the
|
||||
// wrong person writes his wife's remark into a fact attributed to him.
|
||||
const DefaultThreshold = 0.7
|
||||
|
||||
// DefaultMinSeconds — how much speech an identification needs. Under about two
|
||||
// seconds a voiceprint is mostly noise and the similarity score is not worth
|
||||
// reading.
|
||||
const DefaultMinSeconds = 2.0
|
||||
|
||||
// MinEnrollSamples / MinEnrollSeconds — what enrolment requires. Several
|
||||
// separate utterances, not one long one: a profile built from a single sentence
|
||||
// encodes that sentence's prosody as much as the voice.
|
||||
const (
|
||||
MinEnrollSamples = 3
|
||||
MinEnrollSeconds = 9.0
|
||||
)
|
||||
|
||||
// Errors callers distinguish.
|
||||
var (
|
||||
// ErrDisabled — no embedding model is wired. The state of this box.
|
||||
ErrDisabled = errors.New("speaker: recognition is not configured")
|
||||
// ErrTooShort — not enough speech to say anything about.
|
||||
ErrTooShort = errors.New("speaker: not enough audio")
|
||||
// ErrUnknown — audio embedded fine, but no enrolled profile is close
|
||||
// enough. Not an error in the sense of something being broken: it is the
|
||||
// correct answer for a guest, and the caller should carry on without a
|
||||
// speaker rather than treat the turn as failed.
|
||||
ErrUnknown = errors.New("speaker: voice not recognised")
|
||||
// ErrNoProfiles — nobody is enrolled yet.
|
||||
ErrNoProfiles = errors.New("speaker: nobody is enrolled")
|
||||
// ErrNotFound — no profile with that id.
|
||||
ErrNotFound = errors.New("speaker: no such profile")
|
||||
// ErrBadID — an id that is empty or carries characters an id should not.
|
||||
ErrBadID = errors.New("speaker: invalid profile id")
|
||||
// ErrBadFormat — audio that is not the canonical 16 kHz mono PCM shape.
|
||||
ErrBadFormat = errors.New("speaker: audio format not supported")
|
||||
// ErrBadVector — an embedder returned something unusable (empty, or all
|
||||
// zeroes, which normalises to nothing and would match everything equally).
|
||||
ErrBadVector = errors.New("speaker: embedder returned an unusable vector")
|
||||
)
|
||||
|
||||
// Embedder turns speech into a voiceprint. Implementations are expected to
|
||||
// return an L2-normalised vector, because the whole store compares by dot
|
||||
// product; Normalize is applied anyway rather than trusted.
|
||||
//
|
||||
// This is the seam a downloaded ECAPA-TDNN ONNX model plugs into. It is an
|
||||
// interface rather than a concrete ONNX type so the package is testable with no
|
||||
// model on disk, which is the only way it could be tested here at all.
|
||||
type Embedder interface {
|
||||
Embed(ctx context.Context, a audio.Audio) ([]float32, error)
|
||||
// Dim is the vector width, used to reject a profile recorded with a
|
||||
// different model rather than silently scoring it as zero.
|
||||
Dim() int
|
||||
}
|
||||
|
||||
// Disabled is the floor: no model, no answers, no guesses.
|
||||
type Disabled struct{}
|
||||
|
||||
// Embed always fails with ErrDisabled.
|
||||
func (Disabled) Embed(context.Context, audio.Audio) ([]float32, error) { return nil, ErrDisabled }
|
||||
|
||||
// Dim is 0 for the disabled embedder.
|
||||
func (Disabled) Dim() int { return 0 }
|
||||
|
||||
// Profile — one enrolled voice.
|
||||
//
|
||||
// Name is what she calls the person out loud ("Ками"). ID is the stable handle
|
||||
// used in sources and metadata. Samples records how many utterances the
|
||||
// voiceprint was averaged from, so a profile enrolled from the bare minimum is
|
||||
// visibly weaker than one built from ten.
|
||||
type Profile struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Enrolled time.Time `json:"enrolled"`
|
||||
Samples int `json:"samples"`
|
||||
Dim int `json:"dim"`
|
||||
|
||||
// Vec is the voiceprint. Not serialised to any surface: a listing tells him
|
||||
// who is enrolled, it does not hand out the biometric itself.
|
||||
Vec []float32 `json:"-"`
|
||||
}
|
||||
|
||||
// Source is what a fact or note written during this speaker's turn is tagged
|
||||
// with, e.g. "tap:voice:speaker:kami". Attribution belongs in the source rather
|
||||
// than in the text, so it can be corrected or dropped later without rewriting
|
||||
// what was said.
|
||||
func (p Profile) Source(base string) string {
|
||||
if p.ID == "" {
|
||||
return base
|
||||
}
|
||||
return base + ":" + Prefix + p.ID
|
||||
}
|
||||
|
||||
// Match — an identification result. Score is cosine similarity in [-1, 1].
|
||||
type Match struct {
|
||||
Profile Profile
|
||||
Score float64
|
||||
}
|
||||
|
||||
// ValidID reports whether an id is usable as a profile handle. Deliberately
|
||||
// narrow: lowercase letters, digits, dash and underscore. Ids end up in note
|
||||
// sources and in vector-table keys, so a permissive id would be a way to write
|
||||
// into a neighbouring key space.
|
||||
func ValidID(id string) bool {
|
||||
if id == "" || len(id) > 64 {
|
||||
return false
|
||||
}
|
||||
for _, r := range id {
|
||||
switch {
|
||||
case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '-', r == '_':
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// NormalizeID lowercases and trims a proposed id before validating it, so
|
||||
// "Ками" typed as "Kami " does not fail for a reason nobody can see.
|
||||
func NormalizeID(id string) string {
|
||||
return strings.ToLower(strings.TrimSpace(id))
|
||||
}
|
||||
|
||||
// Normalize returns an L2-normalised copy of v, or ErrBadVector when there is
|
||||
// nothing to normalise. A zero vector is refused rather than passed on: it
|
||||
// scores 0 against everything, which reads as "no match" but for the wrong
|
||||
// reason and would hide a broken embedder.
|
||||
func Normalize(v []float32) ([]float32, error) {
|
||||
if len(v) == 0 {
|
||||
return nil, ErrBadVector
|
||||
}
|
||||
var sum float64
|
||||
for _, f := range v {
|
||||
if math.IsNaN(float64(f)) || math.IsInf(float64(f), 0) {
|
||||
return nil, ErrBadVector
|
||||
}
|
||||
sum += float64(f) * float64(f)
|
||||
}
|
||||
norm := math.Sqrt(sum)
|
||||
if norm == 0 {
|
||||
return nil, ErrBadVector
|
||||
}
|
||||
out := make([]float32, len(v))
|
||||
for i, f := range v {
|
||||
out[i] = float32(float64(f) / norm)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Similarity is the cosine similarity of two L2-normalised vectors. Different
|
||||
// widths score 0: a profile enrolled with another model must not accidentally
|
||||
// match, and 0 is below every sane threshold.
|
||||
func Similarity(a, b []float32) float64 {
|
||||
if len(a) != len(b) || len(a) == 0 {
|
||||
return 0
|
||||
}
|
||||
var sum float64
|
||||
for i := range a {
|
||||
sum += float64(a[i]) * float64(b[i])
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
// seconds is the playback length of a frame, for the minimum-audio checks.
|
||||
func seconds(a audio.Audio) float64 { return a.Duration() }
|
||||
Reference in New Issue
Block a user