7c7bd8ceeb
Maven can now be told who someone is. She cannot yet tell who is speaking, and this commit is careful to say so rather than pretend otherwise. What works: profiles are enrolled from several deliberately recorded samples, listed, and deleted. They live in the existing memory_vectors table under a "speaker:" id prefix, so there is no migration; what that needed was a wider interface than memory.Store, hence memory.Catalog with ByPrefix and Delete. Delete is the load-bearing half — a voiceprint someone asked to be rid of has to actually go, and a search-only store cannot do that. InMemoryStore.Insert became an upsert by id to match what the persistent store already did. What does not work, and why it is not faked: there is no speaker-embedding model on this box. Sixteen ggufs in /mnt/hdd1/llms, all text; no ECAPA, no x-vector, no titanet, no wespeaker, no .onnx anywhere under /mnt/hdd1. So newSpeakerEmbedder returns nil, internal/speaker falls back to speaker.Disabled, Identify answers ErrDisabled, and the daemon logs which half is off at startup. The plan's "simple MFCC + GMM" floor is refused in the package comment: MFCC cosine distance detects channel and loudness as much as voice, and a biometric that is confidently wrong writes false claims about named people into his memory. A bad floor is worse than none here. Refused as well, and the reason is in enroll.go's doc comment: the plan asked for unknown speakers to be enrolled on first interaction with a TTS "кто это?". There is no request shape in the protocol that could express that. Taking a biometric of whoever walks past the microphone does it to guests who are not party to the exchange, and a synthesised question into a room is not consent from whoever answers. Authority: enrolment is AuthStepUp, because it is a deliberate sit-down act that writes a biometric of a named person and never something done by voice mid-conversation. Deletion is one rung lower at AuthWrite, deliberately inverting the usual pattern — getting rid of a biometric must never be the harder half. Listing is AuthRead and never returns the vectors themselves. Off unless configured: no speaker block means the three methods answer ErrUnknownMethod, so a default box has no wire path that takes a voiceprint. make build and make test pass. Vikunja #255 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
147 lines
5.2 KiB
Go
147 lines
5.2 KiB
Go
// mavend/speaker.go — core's half of voice identification (Vikunja #255,
|
|
// docs/plans/10-speaker-recognition.md).
|
|
//
|
|
// # What is actually wired here, and what is not
|
|
//
|
|
// The enrolment plumbing is real: profiles are stored, listed and deleted, and
|
|
// the wire methods exist as soon as a speaker block is configured. The
|
|
// recognising half is NOT, and cannot be on this box, because there is no
|
|
// speaker-embedding model on disk — no ECAPA, no x-vector, no titanet, no
|
|
// wespeaker, nothing in /mnt/hdd1/llms but text ggufs. Until one is downloaded,
|
|
// newSpeakerEmbedder returns nil, internal/speaker falls back to
|
|
// speaker.Disabled, and every Identify answers ErrDisabled. The daemon logs
|
|
// which half is off at startup rather than pretending.
|
|
//
|
|
// This is deliberately not papered over with a hand-rolled MFCC floor. A
|
|
// biometric that is confidently wrong writes false claims about named people
|
|
// into his memory, and that is worse than a capability that is honestly absent.
|
|
//
|
|
// # Off unless configured
|
|
//
|
|
// No speaker block, or one without enabled, ⇒ the three methods do not exist and
|
|
// answer ErrUnknownMethod. On an unconfigured box there is no wire path that
|
|
// takes a voiceprint at all.
|
|
//
|
|
// # The refused design step
|
|
//
|
|
// The plan asks for unknown speakers to be enrolled on first interaction. That
|
|
// is refused in internal/speaker/enroll.go and there is no handler for it here:
|
|
// no request shape in the protocol enrols whoever just spoke. Taking a biometric
|
|
// of a guest who walked past the microphone is not something this daemon does.
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"log"
|
|
|
|
"github.com/kami/maven/internal/config"
|
|
"github.com/kami/maven/internal/ipc"
|
|
"github.com/kami/maven/internal/speaker"
|
|
"github.com/kami/maven/internal/store"
|
|
)
|
|
|
|
// speakerWiring holds the recognizer behind the three IPC handlers.
|
|
type speakerWiring struct {
|
|
rec *speaker.Recognizer
|
|
}
|
|
|
|
// newSpeakerEmbedder loads the speaker-embedding model named by the config.
|
|
//
|
|
// It always returns nil today. The seam exists so that wiring a real model is a
|
|
// change to this one function and nothing else: give it a loader, and Identify
|
|
// starts working with no change to the store, the protocol, the auth table or
|
|
// the handlers. See the plan document for what to download.
|
|
func newSpeakerEmbedder(cfg *config.SpeakerConfig) speaker.Embedder {
|
|
if cfg == nil || cfg.ModelPath == "" {
|
|
return nil
|
|
}
|
|
log.Printf("speaker: model_path %q is configured but no embedding backend is built yet; "+
|
|
"enrolment and deletion work, recognition does not (Vikunja #255)", cfg.ModelPath)
|
|
return nil
|
|
}
|
|
|
|
// newSpeakerWiring builds the recognizer, or nil when the capability is off.
|
|
func newSpeakerWiring(st *store.Store, cfg *config.Config) *speakerWiring {
|
|
if cfg == nil || cfg.Speaker == nil || !cfg.Speaker.Enabled {
|
|
return nil
|
|
}
|
|
if st == nil {
|
|
log.Print("speaker: enabled but there is no store to keep profiles in; staying off")
|
|
return nil
|
|
}
|
|
rec, err := speaker.New(newSpeakerEmbedder(cfg.Speaker), st.VectorMemory(), speaker.Config{
|
|
Threshold: cfg.Speaker.Threshold,
|
|
MinSeconds: cfg.Speaker.MinSeconds,
|
|
})
|
|
if err != nil {
|
|
log.Printf("speaker: %v; staying off", err)
|
|
return nil
|
|
}
|
|
if rec.Enabled() {
|
|
log.Printf("speaker: recognition on, threshold %.2f", rec.Threshold())
|
|
} else {
|
|
log.Print("speaker: enrolment on, recognition BLOCKED — no speaker-embedding model " +
|
|
"on this box (see docs/plans/10-speaker-recognition.md)")
|
|
}
|
|
return &speakerWiring{rec: rec}
|
|
}
|
|
|
|
func (w *speakerWiring) enroll(ctx context.Context, req ipc.EnrollSpeakerReq) (ipc.EnrollSpeakerResp, error) {
|
|
p, err := w.rec.Enroll(ctx, req.ID, req.Name, req.Samples)
|
|
if err != nil {
|
|
return ipc.EnrollSpeakerResp{}, speakerErr(err)
|
|
}
|
|
return ipc.EnrollSpeakerResp{Speaker: toWireSpeaker(p)}, nil
|
|
}
|
|
|
|
func (w *speakerWiring) list(ctx context.Context) (ipc.ListSpeakersResp, error) {
|
|
ps, err := w.rec.List(ctx)
|
|
if err != nil {
|
|
return ipc.ListSpeakersResp{}, speakerErr(err)
|
|
}
|
|
out := make([]ipc.Speaker, 0, len(ps))
|
|
for _, p := range ps {
|
|
out = append(out, toWireSpeaker(p))
|
|
}
|
|
return ipc.ListSpeakersResp{Speakers: out, Enabled: w.rec.Enabled()}, nil
|
|
}
|
|
|
|
func (w *speakerWiring) forget(ctx context.Context, req ipc.ForgetSpeakerReq) error {
|
|
return speakerErr(w.rec.Forget(ctx, req.ID))
|
|
}
|
|
|
|
// toWireSpeaker drops the voiceprint. A listing says who is enrolled; it does
|
|
// not hand the biometric back out over the socket.
|
|
func toWireSpeaker(p speaker.Profile) ipc.Speaker {
|
|
return ipc.Speaker{ID: p.ID, Name: p.Name, Enrolled: p.Enrolled, Samples: p.Samples}
|
|
}
|
|
|
|
// speakerErr maps the package sentinels onto the wire vocabulary so a surface
|
|
// can tell "you asked wrong" from "core broke".
|
|
func speakerErr(err error) error {
|
|
switch {
|
|
case err == nil:
|
|
return nil
|
|
case errors.Is(err, speaker.ErrNotFound):
|
|
return ipc.ErrNoFact
|
|
case errors.Is(err, speaker.ErrBadID),
|
|
errors.Is(err, speaker.ErrBadFormat),
|
|
errors.Is(err, speaker.ErrTooShort):
|
|
return errors.Join(ipc.ErrBadParams, err)
|
|
default:
|
|
return err
|
|
}
|
|
}
|
|
|
|
// wireSpeaker attaches the three handlers when the capability is configured.
|
|
func wireSpeaker(srv *ipc.Server, st *store.Store, cfg *config.Config) {
|
|
w := newSpeakerWiring(st, cfg)
|
|
if w == nil {
|
|
return
|
|
}
|
|
srv.EnrollSpeakerFn = w.enroll
|
|
srv.ListSpeakersFn = w.list
|
|
srv.ForgetSpeakerFn = w.forget
|
|
}
|