// Package media is the intake for everything Maven sees or hears that is not // text: a photo he sends her, a meeting she was asked to record, a voice sample // used to enrol a speaker. All three senses (vision, hearing, speaker // recognition) share one problem — a blob arrives, it has to be stored, and // something has to describe it — so the storing half lives here once instead of // three times. // // # What this package is // // A content-addressed blob store on the local filesystem. Put returns a Blob // keyed by the sha256 of its bytes, so the same photo sent twice is one file. // Each blob gets a sidecar `.json` with its kind, mime, size, source and // creation time; the sidecar is the whole index, because at personal scale a // directory walk is cheaper than another sqlite table and the store has to be // readable with `ls` when something goes wrong. // // Blobs are NOT in the sqlite database. The database is small, encrypted, and // read on every tick; a 40 MB meeting recording has no business in it. What // goes in the database is the *text* a blob produced — a transcript, a // description — written as an ordinary note, which is the durable artefact and // the only part worth recalling later. // // # Invariants (these are the point of the package, not decoration) // // - Nothing is captured that was not asked for. This package never records; // it stores what a caller hands it, and every caller is an explicit act // with a start and a stop. There is no ambient path in, and none may be // added: see the refusal recorded in docs/plans/08-hearing.md. // - A blob never leaves the box. No provider in this repo may upload one, and // the vision provider refuses a non-private endpoint for exactly that // reason (internal/vision). // - A blob is never search input and never embedded. His photos and the audio // of his meetings are not corpus. Only text derived from them, once he can // see it as a note, participates in recall. // - Storage is bounded. Retention is a config knob with a default, Prune // enforces it, and an unpruned store is a bug: audio of people accumulating // forever on disk is the failure mode this capability has to avoid. // // # Layout // // ///. the bytes // ///.json the sidecar metadata // // `aa` is the first two hex chars of the digest — one fan-out level, enough to // keep a directory listing usable after a few thousand blobs. package media import ( "errors" "fmt" "time" ) // Kind — what a blob is. Two values today; the kind is a directory name and a // retention bucket, so adding a third is additive. type Kind string const ( // KindImage — a still image (png / jpeg / gif / webp bytes as received). KindImage Kind = "image" // KindAudio — raw PCM in the canonical internal/audio format, or a WAV // container. Meeting captures and enrolment samples both land here. KindAudio Kind = "audio" ) // Valid reports whether k is a kind this package will store. An unknown kind is // refused at Put rather than creating a stray directory. func (k Kind) Valid() bool { return k == KindImage || k == KindAudio } // Errors callers distinguish. ErrNotFound is the only one a caller usually // handles; the rest mean the call was wrong. var ( // ErrNotFound — no blob with that id in this store. ErrNotFound = errors.New("media: not found") // ErrEmpty — Put was handed zero bytes. Storing an empty capture would // leave a sidecar claiming a recording exists when it does not. ErrEmpty = errors.New("media: empty payload") // ErrTooLarge — the payload is over the store's cap. The cap exists so a // runaway capture cannot fill the disk that mavend's database lives on. ErrTooLarge = errors.New("media: payload too large") // ErrBadKind — unknown Kind. ErrBadKind = errors.New("media: unknown kind") // ErrBadID — the id is not a 64-char lowercase hex digest, so it cannot // have come from this store and must not be turned into a path. ErrBadID = errors.New("media: malformed id") ) // Blob — one stored item. ID is the sha256 of the bytes in lowercase hex, which // makes it both the primary key and the dedupe mechanism. Path is absolute and // local; it is a debugging affordance and the argument a subprocess (whisper, // llama-server) is pointed at, never something handed to a network client. type Blob struct { ID string `json:"id"` Kind Kind `json:"kind"` MIME string `json:"mime"` Size int64 `json:"size"` Source string `json:"source"` // provenance: "telegram", "web:upload", "capture:meeting", "enroll" Created time.Time `json:"created"` // UTC Path string `json:"-"` // filled by the store; not part of the sidecar } // Age is how long ago the blob was stored, measured against now. Prune uses it; // it is exported because the /media surface will want to show it. func (b Blob) Age(now time.Time) time.Duration { return now.Sub(b.Created) } // String is a one-line summary for logs. Deliberately does not include Path: // a log line is not the place to spell out where his meeting audio lives. func (b Blob) String() string { return fmt.Sprintf("%s %s %dB from %s", b.Kind, shortID(b.ID), b.Size, b.Source) } // shortID trims a digest to something readable in a log line. Twelve hex chars // is unambiguous at personal scale and short enough to fit next to the rest. func shortID(id string) string { if len(id) <= 12 { return id } return id[:12] }