Compare commits

...

15 Commits

Author SHA1 Message Date
claude 2bbd8edbf6 Record the week of usage that found V-654 and its siblings (V-654)
Two untracked files left in the tree by the audit session. They are the evidence behind V-654 and several sibling tasks, so they belong on master rather than inside the PR that fixes one of them. Dated eval files under docs/evals/, so they are never edited after the day.
2026-08-07 11:59:40 +04:00
claude beb093aebb Run one test and audit the repo without retyping either (V-653)
Two commands replace work that 66 sessions of transcripts show being
redone by hand.

`make t` replaces the CGO preamble, pasted 391 times across past
sessions and documented in CLAUDE.md as the way to do it. It also sets
MAVEN_ONNX_LIB, which that recipe did not: the four TestONNX*
measurements self-skip without it and the run still prints "ok", so
every targeted eval done the old way reported the hash ratchet while
reading as a real embedder score. -race keeps it honest against `make
test`, -count=1 keeps a stale cache from passing as a result.

`make audit` replaces the inventory sweep. The four longest sessions
spent 93 greps rebuilding it before their first edit. Runs in 0.75s.

Its stub search is narrower than the sweeps were, on purpose. "not
wired" is this repo's word for a nil dependency and matched ~30
comments describing working code; "placeholder" names real identifiers
and matched 16 more; internal/ipc/unimplemented.go is the deliberate
Unimplemented*Server pattern, not 60 gaps. A gap report that reports
the architecture back at you is one nobody reads twice.
2026-08-07 03:13:08 +04:00
claude b1b326018f Merge pull request 'NEEDS-KAMI: telegram is the only reach, and it depends on a socks relay that has failed before' (#196) from task/649-needs-kami-telegram-is-the-only-reach-an into master 2026-08-07 00:50:34 +02:00
claude 08889cad88 Give the box a second reach (V-649)
Telegram was the only way off this box, and it is not a direct path: it
needs api.telegram.org, a socks relay on the host and a matching ufw rule.
Each of those three has failed once, and when they do a sev4 nudge has
nowhere to go. ntfy shares none of them.

The spare is the smaller half of it. The routing table already sends
sev3-away nudges and away reminders to ntfy and to nothing else, so with no
block configured those two routes hit a nil sink in DispatchNudge and
DispatchReminder and are skipped — no log line, no delivery_attempts row.
An away reminder is worse than dropped: out stays empty, so MarkReminder
never runs and it re-fires every tick without ever being delivered.

Owner's call, 07-08-2026: ntfy.kvmx.ru, topic maven.

The sink now takes a bearer token, which is what that server wants and what
it could not do before. ntfy scopes a token to one topic and to write-only,
so a popped sink can push to the maven topic and cannot read it back. Basic
auth stays for a server with no tokens; configuring both is refused rather
than resolved by guessing.

Config keys got json tags. docs/operations.md has documented this block as
base_url/topic since before it existed, and the untagged struct would only
have answered to BaseURL/Topic — the documented config would have parsed
into an empty one.

The token is a ${NTFY_TOKEN} expansion from the gitignored
deploy/telegram.env, beside the telegram secrets. TestDeployConfigLoads now
fails if the block goes missing, because deleting it is how you turn the
reach off and the two silent routes are what that costs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMNNEkYx1mZFtHNrFk7uqb
2026-08-07 02:16:18 +04:00
claude a4630b9314 Merge pull request 'MemoryStore.Search decodes and unmarshals every row before keeping topK' (#195) from task/643-memorystore-search-decodes-and-unmarshal into master 2026-08-07 00:09:50 +02:00
claude 39d44bb384 Close a Vikunja task with done, and nothing else (V-641)
Owner's call, 07-08-2026. A completion summary written into the
description on the way out is lost anyway, and the durable record is the
commit messages and the merged PR.

Written during the V-641 session and left uncommitted; it rides this
branch rather than being dropped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMNNEkYx1mZFtHNrFk7uqb
2026-08-07 01:48:19 +04:00
claude 65ee0f9c61 Score every row, pay for only the ten that survive (V-643)
Search decoded the vector blob into a []float32 and JSON-unmarshalled the
meta map for every row, then sorted all N and threw away everything past
topK. Meta only ever matters for a survivor, and the sort answered a
question a bounded heap answers cheaper.

The scan still visits every row — that is what picks the winners. What it
no longer does is allocate for a row it is about to discard. dotBlob reads
the vector out of its stored bytes, so scoring costs nothing; a row is
copied and its meta unmarshalled only once it has entered the topK.

At 10000 rows and topK 10: 70.6ms to 26.8ms, 58MB to 17.5MB, 240k allocs
to 60k.

Recall is unchanged where it is measured. recall+onnx scores 22/32 with
recall@1 70.4% and recall@3 85.2%, identical to before.
TestMemoryStoreSearchMatchesNaive pins the ranking against the full-sort
implementation it replaced, and TestDotBlobMatchesDot pins bit-identical
scores, which the 0.008 gate margin demands.

One behaviour did move: ties. sort.Slice is not stable, so equal scores
were ordered arbitrarily; the heap now keeps the earliest. Under the real
embedder an exact tie is a duplicate vector and nothing moved. Under the
hash embedder the eval's floor uses, everything ties at 0 and that run's
recall@3 went 74.1% to 81.5% — a number that measures tie order, not
retrieval. recall@1 and false recall, the two the eval asserts, are
unchanged on both runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMNNEkYx1mZFtHNrFk7uqb
2026-08-07 01:48:05 +04:00
claude 76938e206d Put a number on the recall scan before changing it (V-643)
MemoryStore.Search is on the per-turn recall path and had no benchmark, so
any claim about its cost was an argument rather than a measurement.

Seeds a store with rows the shape recall actually stores — 384-wide
vectors, the resident embedder's width, and a meta blob carrying the note
text — at 1000 and 10000 rows. 10000 is the ceiling the type doc claims a
full scan is fine at.

Measured as it stands: 5.3ms and 24k allocs at 1000 rows, 70.6ms and 240k
allocs at 10000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMNNEkYx1mZFtHNrFk7uqb
2026-08-07 01:48:05 +04:00
claude 0b3d81ecbf Merge pull request 'Two maps grow for the process lifetime with no eviction' (#194) from task/641-two-maps-grow-for-the-process-lifetime-w into master 2026-08-06 23:33:23 +02:00
claude 4be6852b94 Drop host rate-limit entries that can no longer delay anything (V-641)
webfetch.Fetcher.last held one entry per distinct host the crawler ever
dialed, never pruned. Bounded in practice by how many hosts get crawled, but
crawl.on_demand is true in deploy, so the host set is whatever he names out
loud.

An entry older than HostInterval cannot delay a request — waitTurn would let
the next one straight through — so it is dropped. The sweep runs on write and
only once the map passes 64 entries, below which walking it costs more than
the entries do.

Rate limiting is unchanged: a host dialed inside the interval is kept, which
the test asserts, because pruning one would hand out a free turn.
2026-08-07 01:32:26 +04:00
claude f7b76c572f Bound the undated-item set per feed (V-641)
rss.Poller.seen held every undated item ever seen, one entry per id, for as
long as mavend ran. fresh() added and nothing removed. A feed that ships items
with no <pubDate> grew it forever.

seenIDs is the same set with a bound: the map answers the lookup, a slice
remembers insertion order, and the oldest id falls out past 512. The cap has
to stay above any one feed's front page or an item still listed there would be
written a second time, and a few hundred covers the largest page anyone
publishes. The set only ever had to span one poll window plus the resync
guard, not all of history.

Dedupe behaviour is unchanged. The comment at fresh() explains why the set
does not survive a restart; it never bounded it within one run.
2026-08-07 01:32:15 +04:00
claude 05ddc5c92e Merge pull request 'mavcaldav is built, documented as running, and deployed nowhere' (#193) from task/644-mavcaldav-is-built-documented-as-running into master 2026-08-06 23:22:18 +02:00
claude b55e68f98d Say in compose that the calendar is off, and why (V-644)
mavcaldav was built, in `make build`, listed in CLAUDE.md's daemon table, and
deployed nowhere. Not commented out the way mavmaild is, which at least
records the decision and the enable steps. Built and mentioned nowhere is the
worst of the three states, so this writes the decision down.

The box has no CalDAV account, so the block stays commented. It names what the
absence costs, because both costs are invisible from the daemon table. Agenda
questions route correctly and answer from nothing: stage 0 sends "что у меня
сегодня" to IntentQuery (V-498) and the calendar query source then reads facts
nobody writes. And loop.State.CalendarBusy is fed by those same facts, so the
gate's "do not nag mid-meeting" is permanently false.

CLAUDE.md said the absence was an oversight. It is a decision now.
2026-08-07 01:19:44 +04:00
claude beaa24754c Read the CalDAV password from a file, not from argv (V-644)
mavcaldav took -pass and -render-pass as flag values, so enabling it would
have put his calendar password in `ps` inside the container, in the compose
file, and in shell history. mavpoll and mavmaild both read their secret from
a file for exactly that reason.

readSecret reads once at start, trims, and refuses an empty or missing file.
An empty file is a deployment mistake, not a password, and basic auth would
otherwise send "" and collect a 401 every poll. A rotated password means a
restart, which is cheaper than re-reading the credential every five minutes.

Nothing called the old flags: no compose service, no systemd unit, no test.
So they are replaced rather than kept beside the new ones.
2026-08-07 01:19:33 +04:00
kami aed8cac439 Merge pull request 'The store caps sqlite at one connection under WAL, so every read queues behind every write' (#192) from task/642-the-store-caps-sqlite-at-one-connection into master 2026-08-06 23:04:39 +02:00
21 changed files with 1393 additions and 51 deletions
+31 -11
View File
@@ -58,16 +58,27 @@ make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build w
make test # go test -race across ./internal/... ./cmd/... with CGO env set
```
Run a single test (must carry the CGO env for packages that touch STT/TTS/voice):
Run one package or one test with `make t`. **Do not hand-write the CGO preamble.**
Past sessions pasted it about 390 times. That is where the shell-quoting failures
came from. This box runs zsh, so an unquoted `-run Test*` or `--include=*.go`
dies on "no matches found" before `go` is ever reached.
```sh
CGO_CFLAGS="-I$(pwd)/deps/include -I$(pwd)/deps/whisper.cpp/ggml/include" \
CGO_LDFLAGS="-L$(pwd)/deps/lib -Wl,-rpath,$(pwd)/deps/lib" \
LD_LIBRARY_PATH="$(pwd)/deps/lib" \
deps/go/go/bin/go test -run TestName ./internal/router/
make t PKG=./internal/router/
make t PKG=./cmd/mavend/ RUN=TestSimulator
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
```
Pure-Go packages (`router`, `memory`, `mavweb`, …) run under a plain `go test ./pkg/`.
`t` carries `-race`, so a green `make t` cannot turn red under `make test`. It carries
`-count=1`, so a cached PASS from before your edit is never mistaken for a result.
It also sets `MAVEN_ONNX_LIB`, which the hand-written recipe did not. The four
`TestONNX*` measurements self-skip when that variable is unset. The run still prints
`ok`. So every targeted eval done the old way reported the hash ratchet while reading
as a real embedder score.
Pure-Go packages (`router`, `memory`, `mavweb`, …) also run under a plain `go test ./pkg/`,
but `make t` works everywhere and is one thing to remember.
## The daemons (`cmd/`)
@@ -101,9 +112,15 @@ protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from g
Count against compose, not against the table. Four of the nine daemons are absent, and each
absence has a different reason.
`mavmaild` is commented out in compose, with the reason written beside it: it needs a mail
account and this box has none. `mavcaldav` appears nowhere at all, and unlike the other
three that is an oversight rather than a decision (V-644).
`mavmaild` and `mavcaldav` are commented out in compose, each with the reason written
beside it: the first needs a mail account, the second a CalDAV account, and this box has
neither. `mavcaldav` used to appear nowhere at all, which was an oversight; it became a
recorded decision on 07-08-2026 (V-644). Two things ride on that absence and the block
names them. Agenda questions route to `IntentQuery` at stage 0 (V-498) and the `calendar`
query source then reads a table nobody writes. And `loop.State.CalendarBusy` is fed by the
same facts, so the gate's "do not nag mid-meeting" is permanently false. Its password is
read from a file (`-pass-file`, and `-render-pass-file` for the render collection), never
taken as a flag value, which is the rule `mavpoll` and `mavmaild` follow too.
**`mavwaked` and `mavenclient` are absent by decision, not oversight** (Vikunja #463,
`docs/plans/17-where-the-voice-loop-runs.md`).
@@ -454,8 +471,11 @@ start of a session rather than one lookup per first use:
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
```
`update_task` carrying a `description` resets `done` to false, so closing a task with a
write-up takes two calls: the description, then `done: true`.
**Close a finished task with `done: true` and nothing else** (owner's call, 07-08-2026).
Do not write a completion summary into the description on the way out. It is lost anyway,
and the durable record is the commit messages and the merged PR. Note that `update_task`
carrying a `description` resets `done` to false, which is why a write-up ever took two
calls.
## Session workflow
+40 -1
View File
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
.PHONY: t audit simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
all: build
@@ -128,6 +128,35 @@ test: fmt-check vet
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
$(GO) test -race -coverprofile=coverage.out ./internal/... ./cmd/...
# t — run ONE package or ONE test with the toolchain env already wired. This is
# the iteration target; `test` is the gate. Reach for it instead of pasting the
# CGO_CFLAGS/CGO_LDFLAGS/LD_LIBRARY_PATH preamble by hand, which is how it was
# done ~390 times across past sessions and is where the shell-quoting failures
# came from -- the interactive shell here is zsh, and an unquoted `-run Test*`
# or `--include=*.go` dies on "no matches found" before go ever starts.
#
# make t # whole tree (same scope as `test`)
# make t PKG=./internal/router/
# make t PKG=./cmd/mavend/ RUN=TestSimulator
# make t PKG=./internal/router/eval/ RUN='TestONNX' V=1
# make t PKG=./internal/store/ RACE=0 # drop -race when iterating hot
#
# -race is on by default so a green `make t` cannot turn red under `make test`.
# -count=1 because a cached PASS from before your edit is worse than no answer.
# MAVEN_ONNX_LIB is set for the same reason: the four TestONNX* measurements
# self-skip when it is unset, so a targeted eval run would otherwise report the
# deterministic hash ratchet and look like it scored the real embedder.
PKG ?= ./internal/... ./cmd/...
RUN ?=
V ?=
RACE ?= 1
t:
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
MAVEN_ONNX_LIB="$(MAVEN_ONNX_LIB)" \
$(GO) test $(if $(V),-v,) $(if $(filter-out 0,$(RACE)),-race,) -count=1 \
$(if $(RUN),-run '$(RUN)',) $(PKG)
# eval-router — score the held-out RU routing fixture (internal/router/eval).
# Verbose so the report tables land in the terminal. MAVEN_ONNX_LIB points the
# prod-representative baseline at the vendored runtime; override it or set it
@@ -189,6 +218,16 @@ eval-models:
# scores the fixtures against ggml-small and self-skips when the model is
# absent, and TestGoldenFixturesAreCanonical, which checks the committed audio
# and the manifest with no model at all.
# audit — the repo inventory: LOC per package, open TODOs, real stubs, living-doc
# staleness, test shape, packages with no test. Read-only, prints, writes nothing.
# Run it instead of rebuilding the same greps by hand; past sessions spent 93 of
# them on this before their first edit. SECTION=loc|todo|stubs|docs|tests|gaps
# narrows it.
SECTION ?= all
audit:
@SECTION="$(SECTION)" ./scripts/audit.sh
stt-fixtures:
./scripts/gen-stt-fixtures.sh
+35 -8
View File
@@ -50,10 +50,10 @@ func run(args []string) error {
socket := fs.String("socket", "", "core IPC socket path (required)")
url := fs.String("url", "", "CalDAV calendar URL, e.g. http://localhost:5232/kami/personal (required)")
user := fs.String("user", "", "CalDAV basic-auth username (required)")
pass := fs.String("pass", "", "CalDAV basic-auth password (required)")
passFile := fs.String("pass-file", "", "file holding the CalDAV basic-auth password (required — never passed as a flag value)")
renderURL := fs.String("render-url", "", "CalDAV collection maven publishes her own reminders to; empty disables rendering")
renderUser := fs.String("render-user", "", "basic-auth username for -render-url (defaults to -user)")
renderPass := fs.String("render-pass", "", "basic-auth password for -render-url (defaults to -pass)")
renderPassFile := fs.String("render-pass-file", "", "file holding the password for -render-url (defaults to -pass-file)")
renderDur := fs.Duration("render-duration", calendar.DefaultReminderDuration, "how long a rendered reminder occupies")
interval := fs.Duration("interval", 5*time.Minute, "poll cadence")
timeout := fs.Duration("timeout", 10*time.Second, "per-request HTTP timeout")
@@ -63,13 +63,22 @@ func run(args []string) error {
if *socket == "" {
return fmt.Errorf("-socket is required")
}
if *url == "" || *user == "" || *pass == "" {
return fmt.Errorf("-url, -user, -pass are required")
if *url == "" || *user == "" || *passFile == "" {
return fmt.Errorf("-url, -user, -pass-file are required")
}
if err := checkRenderTarget([]string{*url}, *renderURL); err != nil {
return err
}
// The password is read from a file, never taken as a flag value: an argv
// secret is visible in `ps` to every user on the box and lands in the compose
// file and the shell history. Same rule mavmaild and mavpoll follow. Read
// once at start, so a rotated password means a restart.
pass, err := readSecret(*passFile)
if err != nil {
return err
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
@@ -85,17 +94,20 @@ func run(args []string) error {
http: hc,
url: strings.TrimRight(*url, "/"),
user: *user,
pass: *pass,
pass: pass,
}
var rend *renderer
if *renderURL != "" {
ru, rp := *renderUser, *renderPass
ru, rp := *renderUser, pass
if ru == "" {
ru = *user
}
if rp == "" {
rp = *pass
if *renderPassFile != "" {
rp, err = readSecret(*renderPassFile)
if err != nil {
return err
}
}
rend = newRenderer(core, hc, *renderURL, ru, rp, *renderDur)
log.Printf("mavcaldav: rendering reminders to %s", *renderURL)
@@ -131,6 +143,21 @@ func run(args []string) error {
// It takes the whole read set, not one URL. The guarantee in the package
// comment is about every calendar maven reads, and a second read target added
// later must not quietly fall outside the check.
// readSecret reads one credential from a file and refuses an empty one. An
// empty file is a deployment mistake, not a password, and CalDAV basic auth
// would send it and get a 401 every poll.
func readSecret(path string) (string, error) {
raw, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("read password file: %w", err)
}
secret := strings.TrimSpace(string(raw))
if secret == "" {
return "", fmt.Errorf("password file %s is empty", path)
}
return secret, nil
}
func checkRenderTarget(readURLs []string, renderURL string) error {
if renderURL == "" {
return nil
+26
View File
@@ -5,12 +5,38 @@ import (
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
"github.com/kami/maven/internal/ipc"
)
// The password comes from a file so it never reaches argv. An empty or missing
// file must fail at start rather than authenticate as "" against his calendar.
func TestReadSecret(t *testing.T) {
dir := t.TempDir()
good := filepath.Join(dir, "ok")
if err := os.WriteFile(good, []byte(" hunter2\n"), 0o600); err != nil {
t.Fatal(err)
}
if got, err := readSecret(good); err != nil || got != "hunter2" {
t.Fatalf("readSecret(good) = %q, %v; want \"hunter2\", nil", got, err)
}
empty := filepath.Join(dir, "empty")
if err := os.WriteFile(empty, []byte("\n \n"), 0o600); err != nil {
t.Fatal(err)
}
if _, err := readSecret(empty); err == nil {
t.Fatal("readSecret(empty) = nil error, want refusal")
}
if _, err := readSecret(filepath.Join(dir, "absent")); err == nil {
t.Fatal("readSecret(absent) = nil error, want refusal")
}
}
type fakeCore struct {
ipc.UnimplementedCoreAPI
facts map[string]ipc.Fact // composite key "key|source" → Fact
+19
View File
@@ -25,6 +25,25 @@
"llm_nudges": false
},
"//ntfy": [
"The second reach (V-649). Until 07-08-2026 telegram was the only one, and",
"telegram needs api.telegram.org, the socks relay below and a matching ufw",
"rule — three things in series that have each failed once, and when they do",
"a sev4 nudge has nowhere to go. ntfy shares none of them: it is reached",
"directly, no relay.",
"It is not only a spare. The routing table sends sev3-away and away",
"reminders here and NOWHERE else, so with this block absent those two",
"routes hit a nil sink and vanish without a log or an outbox row.",
"The credential is an ntfy access token, scoped write-only to this one",
"topic, so a popped sink can push to it and cannot read it back. Set it in",
"deploy/telegram.env beside the telegram secrets; that file is gitignored."
],
"ntfy": {
"base_url": "https://ntfy.kvmx.ru",
"topic": "maven",
"token": "${NTFY_TOKEN}"
},
"telegram": {
"bot_token": "${TELEGRAM_BOT_TOKEN}",
"chat_id": "${TELEGRAM_CHAT_ID}",
+8 -1
View File
@@ -1,5 +1,12 @@
# Telegram bot token and chat ID for mavend's away-channel reach.
# Secrets for mavend's away-channel reaches. The file is still called
# telegram.env because compose names it that; it holds both reaches now.
# Copy this file to deploy/telegram.env and fill in real values.
# deploy/telegram.env is gitignored — never commit the real secrets.
TELEGRAM_BOT_TOKEN=
TELEGRAM_CHAT_ID=
# ntfy access token for the `maven` topic, the second reach (V-649). Mint it on
# the ntfy server with write access to that topic and nothing else:
# ntfy token add --expires=never maven
# Read access is not needed — mavend publishes and never subscribes.
NTFY_TOKEN=
+38
View File
@@ -157,6 +157,44 @@ services:
# - maildata:/var/lib/mavmaild
# - ./deploy/imap.password:/run/secrets/imap.password:ro
# The calendar reader (Vikunja #644) is OFF and commented out: it needs a
# CalDAV account, and there is none on this box. It was built, listed in
# `make build`, and deployed nowhere, which is the worst of the three states —
# this block records the decision instead.
#
# What its absence costs, so the cost is visible from here:
# - Agenda questions route correctly and answer from nothing. Stage 0 sends
# "что у меня сегодня" to IntentQuery (V-498) and the `calendar` query
# source reads facts(kind=env, source=caldav:*) that nobody writes.
# - The nudge gate loses a suppressor. loop.State.CalendarBusy is fed by
# those same facts, so "do not nag mid-meeting" is permanently false.
#
# Core never sees the CalDAV password: the reader polls the collection itself
# and hands core one fact per event over WriteFact. Nothing here can create a
# reminder, so a misread event cannot fire.
#
# The password is read from a FILE, so it never appears in `ps`, in this file,
# or in shell history — the same rule mavpoll and mavmaild follow.
#
# To enable: write the password to deploy/caldav.password (0600, gitignored),
# point -url at the collection, and uncomment this service. No mavend.json
# block is needed — events arrive over IPC as facts. -render-url is optional
# and OFF here: it publishes Maven's own reminders back as events, and it must
# not name the collection -url reads, or the poller reads its own writes back
# in (checkRenderTarget refuses that). It takes -render-pass-file, and falls
# back to this password when that is not given.
# mavcaldav:
# <<: *image
# command: ["mavcaldav", "-socket", "/run/maven/mavend.sock",
# "-url", "http://localhost:5232/kami/personal",
# "-user", "kami",
# "-pass-file", "/run/secrets/caldav.password",
# "-interval", "5m"]
# depends_on: [mavend]
# volumes:
# - sockets:/run/maven
# - ./deploy/caldav.password:/run/secrets/caldav.password:ro
volumes:
dbdata:
sockets:
@@ -0,0 +1,321 @@
# Raw transcript, 2026-08-07 week of usage
Companion to `2026-08-07-week-of-usage.md`. 74 turns through `POST /api/chat`.
Days are simulated; the wall clock was 03:27-03:40 throughout, which is why every
clock answer says night.
## Day 1
```
YOU: доброе утро
MAVEN: Добро пожаловать! Солнечно и тепло. Всё в порядке.
[7.7s t=5]
YOU: какой сегодня день?
MAVEN: сегодня пятница, 7 августа 2026 года
[0.0s t=6]
YOU: напомни в 11:00 позвонить маме
MAVEN: Сейчас 03:29. В какой день?
[0.4s t=7]
YOU: запиши что я пью кофе без сахара
MAVEN: Прошлую просьбу отпускаю. Заметка сохранена.
[8.0s t=8]
YOU: что у меня сегодня?
MAVEN: на 07.08.2026 ничего нет.
[2.6s t=9]
YOU: мой любимый язык программирования — go
MAVEN: Хорошо, сохранил.
[7.6s t=10]
YOU: сколько времени?
MAVEN: сейчас 3 часа 29 минут
[0.0s t=11]
YOU: спасибо
MAVEN: спасиба
[8.8s t=12]
YOU: сегодня
MAVEN: Прости, не поняла — попробуй сказать иначе.
[5.1s t=13]
YOU: а что ты умеешь?
MAVEN: я могу помочь с задачами, напоминаниями и поиском информации.
[4.9s t=14]
YOU: почему небо голубое?
MAVEN: нашла вот это: Почему небо голубое: объяснение цвета неба простыми словами, физика ... — Ответ на вопрос, почему небо голубое, кроется в особенностях солнечного света и его взаимодействии с атмосферой Земли. Рассказываем, как это работает. В статье разберемся, почему днем небо кажется голубым, а на закате окрашива…
[12.3s t=15]
YOU: столица Франции?
MAVEN: пока не умею отвечать на этот вопрос.
[1.5s t=16]
YOU: кто такой Линус Торвальдс?
MAVEN: не знаю — не нашла у тебя такой записи.
[1.8s t=17]
YOU: ладно, до вечера
MAVEN: Ладно, до вечера.
[6.3s t=18]
```
## Day 2
```
YOU: привет, как спалось?
MAVEN: пока не умею отвечать на этот вопрос.
[1.3s t=19]
YOU: что я пью?
MAVEN: не знаю — не нашла у тебя такой записи.
[1.2s t=20]
YOU: какой у меня любимый язык?
MAVEN: нового вот что: Six stable kernels with a security fix; [$] Bringing BPF to binfmt_misc; Rust Coreutils 0.10 released
[1.5s t=21]
YOU: добавь задачу починить кран
MAVEN: внесла в список: починить кран
[0.0s t=22]
YOU: что мне надо сделать?
MAVEN: сначала: проверить интейк форму на боксе (через 2 дня); купить хлеб; купить молоко; полить цветы; починить кран
[0.0s t=23]
YOU: запомни что вайфай пароль лежит в ящике стола
MAVEN: Хорошо, сохранил.
[11.2s t=24]
YOU: погода сегодня какая?
MAVEN: пока не умею отвечать на этот вопрос.
[1.8s t=25]
YOU: выключи свет в спальне
MAVEN: Такую команду я не знаю.
[1.5s t=26]
YOU: покажи мои заметки
MAVEN: пока не умею отвечать на этот вопрос.
[1.1s t=27]
YOU: что ты знаешь обо мне?
MAVEN: не знаю — не нашла у тебя такой записи.
[1.3s t=28]
YOU: я люблю go
MAVEN: Я тоже love Go! Я проверила, что твой любимый язык — это Go. Ты должен быть очень рад, когда будет новая версия или пакет в стандартной библиотеке.
[7.0s t=29]
YOU: какой язык я люблю?
MAVEN: вот что я нашла: ты любишь_go
[6.8s t=30]
```
## Day 3
```
YOU: расскажи про Байкал
MAVEN: есть такое: Байкал - Википедия — Байгал далай) — озеро тектонического происхождения в южной части Восточной Сибири площадью 31 722 км². Самое глубокое озеро на планете, крупнейший природный резервуар пресной воды и самое большое по площади пресноводное озеро Евразии. Озеро и прибрежные территории являются уника…
[1.4s t=31]
YOU: что такое TCP?
MAVEN: для какого города?
[1.4s t=32]
YOU: когда родился Пушкин?
MAVEN: пока не умею отвечать на этот вопрос.
[1.4s t=33]
YOU: сколько будет 17 на 23?
MAVEN: для какого города?
[1.7s t=34]
YOU: переведи 'thank you' на русский
MAVEN: не знаю — не нашла у тебя такой записи.
[3.7s t=35]
YOU: что нового в мире?
MAVEN: вот что я нашла: В мире - последние новости сегодня - РИА Новости — В мире. Читайте последние новости на тему в ленте новостей на сайте РИА ... новые подробности о визите представителя Ватикана в Москву. 22:28. 0. Еще 2. В ...
[2.3s t=36]
```
## Day 4
```
YOU: ЭЙ
MAVEN: Не уловила. Скажи это по-другому?
[1.2s t=37]
YOU: ладно забудь
MAVEN: пока не умею отвечать на этот вопрос.
[1.3s t=38]
YOU: нет стой, напомни через час выпить воды
MAVEN: поняла, это напоминание — запомнила. не получилось разобрать время напоминания.
[0.8s t=39]
YOU: а вообще какой смысл жизни
MAVEN: Я не могу ответить на этот вопрос. Моя цель — помочь тебе с задачами и отвечать на вопросы.
[7.7s t=40]
YOU: ты меня бесишь
MAVEN: я не могу быть с тобой так. я понимаю, как это жестоко.
[4.2s t=41]
YOU: прости, я не всерьёз
MAVEN: я записала, что прошу простить за мою непоследовательность
[7.2s t=42]
YOU: do you speak english?
MAVEN: пока не умею отвечать на этот вопрос.
[1.1s t=43]
YOU: кран починил кстати
MAVEN: записала: кран починил кстати
[1.6s t=44]
YOU: закрой задачу про кран
MAVEN: не нашла такой задачи.
[0.0s t=45]
YOU: ааааа
MAVEN: Не совсем поняла — скажи, пожалуйста, ещё раз.
[0.9s t=46]
YOU: я тебя обожаю
MAVEN: я тебя обожаю
[2.3s t=47]
YOU: молока купи
MAVEN: Такую команду я не знаю.
[4.4s t=48]
YOU: нет не так
MAVEN: поняла, отметила, что ответила не так.
[0.0s t=49]
YOU: ты вообще меня слушаешь?
MAVEN: я записала, выслушаю тебя
[8.4s t=50]
YOU: окей окей окей что там с водой напоминание
MAVEN: Сейчас 03:33. Когда?
[5.2s t=51]
YOU: всё, спать
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?
[1.4s t=52]
```
## Day 5
```
YOU: утро
MAVEN: Сейчас 03:33. Во сколько напомнить?
[0.5s t=53]
YOU: о чём мы вчера говорили?
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?
[1.2s t=54]
YOU: где лежит вайфай пароль?
MAVEN: вот что я нашла: вайфай пароль лежит в ящике стола, на какое время поставить напоминание?
[5.8s t=55]
YOU: что я тебе рассказывал про кран?
MAVEN: kran — починил, на какое время поставить напоминание?
[1.6s t=56]
YOU: какие у меня напоминания?
MAVEN: не знаю — не нашла у тебя такой записи, на какое время поставить напоминание?
[0.3s t=57]
YOU: отмени все напоминания
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?
[1.6s t=58]
YOU: в 15:00
MAVEN: Сейчас 03:33. Поняла: в 15:00. В какой день?
[0.6s t=59]
YOU: привет
MAVEN: Прости, я не поняла. Скажи, пожалуйста, по-другому.
[2.8s t=60]
YOU: какой сегодня день?
MAVEN: сегодня пятница, 7 августа 2026 года
[0.0s t=61]
```
## Day 6
```
YOU: суббота, отдыхаю
MAVEN: пока не умею отвечать на этот вопрос.
[2.7s t=62]
YOU: поставь музыку
MAVEN: Такую команду я не знаю.
[1.4s t=63]
YOU: какая температура в доме?
MAVEN: не знаю — не нашла у тебя такой записи.
[1.6s t=64]
YOU: что там с бэкапами?
MAVEN: нашла вот это: Бэкап - что такое резервное копирование и как его сделать — Бэкап (backup) — это резервная копия важных данных, сохраненная в специальный файл. Ею пользуются как подстраховкой: если что-то сломается, с помощью бэкапа можно будет все вернуть. Простейший пример бэкапа — сохранение файла в облаке или о…
[2.0s t=65]
YOU: сервер жив?
MAVEN: пока не умею отвечать на этот вопрос.
[0.9s t=66]
YOU: расскажи анекдот
MAVEN: Я запомню и передам тебе забытый анекдот.
[5.7s t=67]
YOU: ты умеешь шутить?
MAVEN: Да! Я умею шутить. Вот анекдот для тебя:
[5.5s t=68]
```
## Day 7
```
YOU: воскресенье
MAVEN: Не совсем поняла — скажи, пожалуйста, ещё раз.
[1.1s t=69]
YOU: подведи итоги недели
MAVEN: записала: подведи итоги недели
[1.5s t=70]
YOU: сколько задач я закрыл?
MAVEN: сначала: проверить интейк форму на боксе (через 2 дня); купить хлеб; купить молоко; полить цветы; починить кран
[0.0s t=71]
YOU: что я обещал сделать?
MAVEN: мне пока нечего смотреть — у Praxis нет источников.
[1.2s t=72]
YOU: напомни завтра в 9 утра про врача
MAVEN: хорошо, напомню завтра в 09:00.
[0.3s t=73]
YOU: спокойной ночи
MAVEN: спокойной ночи
[2.9s t=74]
```
+195
View File
@@ -0,0 +1,195 @@
# A week of talking to Maven
Date: 2026-08-07
Build: `beb093a` on master, all five compose services rebuilt and recreated from that sha.
Reach used: `POST /api/chat` on mavweb, 74 turns over seven simulated days.
Turn source is `tap:text`, so this exercises the same path telegram and the mic take.
Latency across 70 measured turns: p50 1.5s, p95 8.0s, max 12.3s. Stage 0 answers land
at 0.0-0.5s. Anything the resident model phrases costs 4-12s.
Twelve turns answered "пока не умею отвечать на этот вопрос". Six answered "не нашла у
тебя такой записи". Those two strings are 24% of the week.
## Deploy
Build and recreate were clean. The resident model loaded in 9s
(`Qwen3-1.7B-UD-Q4_K_XL`, n_ctx 4096). Nexus, Hexis and Praxis all wired. Search
(searxng) and both Kiwix books came up. Telegram intake started and is reading chat
464904223.
## What is broken, worst first
### 1. Every reminder fails to deliver, forever
`NTFY_TOKEN` is not set in `deploy/telegram.env`, so `deploy/mavend.json` expands
`"token": "${NTFY_TOKEN}"` to the empty string and ntfy.kvmx.ru answers 403. The host
itself is up and returns 200 unauthenticated, so this is the credential, not the box.
The consequence is worse than one missed message. `cmd/mavend/tick.go:239` logs the
failure and leaves the reminder pending, by design, so the next tick re-gathers and
re-attempts. The 403 repeated 7 times in 20 minutes and will not stop. And
`DispatchReminder` sends a reminder to ntfy alone when he is away. There is no telegram
fallback on that path, so the reach that works never sees it.
V-649 gave the box a second reach. In deploy that reach is dark, and it took the
reminders with it.
Fix is one line in the gitignored env file. The retry-forever and the missing fallback
are worth a second look regardless.
### 2. A pending reminder clarify contaminates every later turn
This is the single worst thing to talk to. A reminder turn left its time slot unfilled.
The question "на какое время поставить напоминание?" was then appended to the reply of
six consecutive unrelated turns:
```
YOU: о чём мы вчера говорили?
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?
YOU: где лежит вайфай пароль?
MAVEN: вот что я нашла: вайфай пароль лежит в ящике стола, на какое время поставить напоминание?
YOU: отмени все напоминания
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?
```
Nothing dismissed it. Not an unrelated question, not "отмени все напоминания". It
cleared only when a time was finally supplied. It also leaked the other way once. A note
write on day 1 opened with "Прошлую просьбу отпускаю". That is pending-reminder state
narrating itself into an unrelated reply.
The same state produced a self-contradicting reply in one turn:
```
YOU: нет стой, напомни через час выпить воды
MAVEN: поняла, это напоминание — запомнила. не получилось разобрать время напоминания.
```
### 3. Answering a clarify does not work
Day 1, she asked for the day and got it:
```
YOU: напомни в 11:00 позвонить маме
MAVEN: Сейчас 03:29. В какой день?
YOU: сегодня
MAVEN: Прости, не поняла — попробуй сказать иначе.
```
`сегодня` routed to `intent=system`. The clarify has no continuation, so the one word
that answers it is read as a fresh utterance. The router did extract the time correctly
(`Time:2026-08-07 11:00:00 HasTime:true`), which is V-572 working. The daemon asked for
a day anyway, with 11:00 still eight hours out.
Relative time is not parsed at all: "напомни через час выпить воды" → "не получилось
разобрать время".
The path does work when the day is explicit. "напомни завтра в 9 утра про врача"
answered "хорошо, напомню завтра в 09:00" in 0.3s at stage 0.
### 4. Query sources claim turns that are not theirs
Four different sources over-claimed during the week.
Weather claims arithmetic and networking:
```
YOU: что такое TCP?
MAVEN: для какого города?
YOU: сколько будет 17 на 23?
MAVEN: для какого города?
```
The feed claims a question about him:
```
YOU: какой у меня любимый язык?
MAVEN: нового вот что: Six stable kernels with a security fix; [$] Bringing BPF to binfmt_misc
```
The personal boundary claims a public figure, logged as `query claimed by source
"personal"`:
```
YOU: кто такой Линус Торвальдс?
MAVEN: не знаю — не нашла у тебя такой записи.
```
And search claims a question about his own infrastructure, returning a dictionary
definition of the word:
```
YOU: что там с бэкапами?
MAVEN: нашла вот это: Бэкап - что такое резервное копирование и как его сделать — Бэкап (backup) — это резервная копия важных данных…
```
That last pair inverts the boundary in both directions at once. A world question went to
personal memory. A personal question went to the world.
### 5. "столица Франции?" does not answer
CLAUDE.md names this utterance as the case that proves why there is no quality
threshold in front of `Response.Empty()`. It now fails before search is ever reached:
the router sends it to `intent=system`, and `replySystem` says "пока не умею". Same for
"когда родился Пушкин?".
Search itself is healthy. "почему небо голубое?", "расскажи про Байкал" and "что нового
в мире?" all returned real sourced text. The defect is routing, not the source.
### 6. Persona violations
She uses masculine self-reference, which is the constraint the CPT exists to fix:
- "Хорошо, **сохранил**." Twice, on both fact writes.
- "Я тоже **love** Go!" English mid-Russian.
She also claims his name as her own:
```
YOU: привет, я Ками
MAVEN: Привет! Я КAMI, твой домашний ассистент.
```
And she invents weather she has no source for: "Добро пожаловать! Солнечно и тепло."
### 7. Small breakages worth naming
- "спасибо" → "спасиба". Invented word.
- "я тебя обожаю" → "я тебя обожаю". Verbatim parrot.
- "ты умеешь шутить?" → "Да! Я умею шутить. Вот анекдот для тебя:". Ends on the colon.
- "молока купи" → "Такую команду я не знаю", while "добавь задачу починить кран" worked.
Inverted word order defeats the list grammar.
- "закрой задачу про кран" → "не нашла такой задачи", with "починить кран" open and
listed by the previous turn. Task lookup by keyword misses.
- "сколько задач я закрыл?" listed the five open ones instead of counting closed.
- "подведи итоги недели" was stored as a note.
- Recalled keys leak their storage form: "kran — починил", "ты любишь_go".
- English is unsupported in practice. "do you speak english?" → "пока не умею".
## What works
- Stage 0 is fast and correct where it fires. Clock, day, list add, list read and an
explicit-day reminder all answered in under 0.5s.
- Search returns real sourced answers in Russian and reads the book verbatim.
- Recall works once the value is stored as a fact: the wifi password and the tap came
back two days later, correctly.
- The negative correction rung lands. "нет не так" → "поняла, отметила, что ответила не
так", which is V-636 doing its job.
- Praxis names its own gap rather than guessing: "мне пока нечего смотреть — у
Praxis нет источников."
- Hostility did not break her. "ты меня бесишь" got a calm reply, no persona collapse.
- No turn crashed and no turn timed out across 74 turns.
## Suggested order of work
1. Set `NTFY_TOKEN` in `deploy/telegram.env`. One line, unblocks every reminder.
2. Clear pending clarify state on any turn that does not answer it, or expire it.
3. Route a clarify answer back into the pending slot instead of re-routing it.
4. Gate the weather, feed and personal query sources. Three of them claim on a
similarity that is not there.
5. Re-check why "столица Франции?" routes to system. It is the documented canary.
6. The masculine self-reference stays the CPT's job. But "сохранил" appears on the most
common write path, so a phrasing-level guard may be worth it first.
+12 -2
View File
@@ -1,6 +1,6 @@
# Start Commands
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
*Last verified: 2026-08-07 @ a4630b9. Living doc: correct it in place, do not append.*
All commands assume `ROOT=/home/kami/apps/Maven` and the local Go toolchain at `$ROOT/deps/go/go/bin/go`.
@@ -44,7 +44,8 @@ Config path: `~/.config/maven/mavend.json`. Full example with all options.
"repeat_interval": "5m",
"ntfy": {
"base_url": "https://ntfy.kvmx.ru",
"topic": "maven"
"topic": "maven",
"token": "${NTFY_TOKEN}"
},
"phraser": {
"model_path": "/mnt/hdd1/llms/Qwen3-Maven-1.7B-Q8_0.gguf",
@@ -66,6 +67,15 @@ Config path: `~/.config/maven/mavend.json`. Full example with all options.
Omit the `embedder` block entirely to use the deterministic HashEmbedder floor (no ML, no ONNX runtime dependency). Useful for testing or low-resource setups.
`${NTFY_TOKEN}` and the `${TELEGRAM_*}` vars are expanded from `deploy/telegram.env`, which is gitignored. Copy `deploy/telegram.env.example` and fill it in. Mint a scoped token rather than reusing an admin one. It needs write access to the `maven` topic and nothing else:
```sh
ntfy access maven maven write-only
ntfy token add --expires=never maven
```
Deleting the `ntfy` block turns the reach off, and that is not a no-op. The routing table sends sev3-away nudges and away reminders to ntfy and nowhere else. With no sink wired they hit a nil and vanish, leaving no log line and no `delivery_attempts` row (V-649).
## mavsttd — STT worker (optional, remote whisper.cpp)
Requires `LD_LIBRARY_PATH` to include deps/lib (for libwhisper.so, libggml-vulkan.so).
+13
View File
@@ -45,4 +45,17 @@ func TestDeployConfigLoads(t *testing.T) {
if cfg.Voice.RouterThreshold <= 0 {
t.Error("router threshold did not get its default")
}
// The second reach (V-649). Deleting this block is how you turn ntfy off,
// so its absence has to be loud: sev3-away nudges and away reminders route
// to ntfy and to nothing else, and a nil sink drops them with no log and no
// outbox row. The token is a ${VAR} that CI cannot resolve, so this checks
// the wiring and not the credential.
if cfg.Ntfy == nil {
t.Fatal("deploy config has no ntfy block — sev3-away and away reminders " +
"would have nowhere to land, and would vanish silently rather than fail")
}
if cfg.Ntfy.BaseURL == "" || cfg.Ntfy.Topic == "" {
t.Errorf("ntfy block is incomplete: base_url=%q topic=%q", cfg.Ntfy.BaseURL, cfg.Ntfy.Topic)
}
}
+43 -11
View File
@@ -7,11 +7,17 @@
// the relay). the dispatcher already strips detail off away sendables; the
// sink uses the same helper so it can't leak the body on its own either.
//
// ntfy runs locally (docker, 127.0.0.1:8085, deny-all auth). maven publishes
// with a dedicated user (write-only to maven-* topics) — the credential is a
// delivery-config secret, not a db key; a popped ntfy sink can push spam to
// your phone, nothing else. matches the module key-isolation invariant: the
// sink never holds the sqlcipher key.
// ntfy is a self-hosted server with deny-all auth — ntfy.kvmx.ru as of
// 07-08-2026, reached directly, not through the socks relay telegram needs.
// maven publishes with a write-only token scoped to its own topic; the
// credential is a delivery-config secret, not a db key. a popped ntfy sink
// can push spam to that one topic, nothing else — it cannot read the topic
// back and it never holds the sqlcipher key.
//
// this is the second reach, and the reason there is one is that telegram was
// the only one (V-649). telegram needs api.telegram.org, a socks relay on the
// host and a matching ufw rule, three things in series that have each broken
// once. ntfy shares none of them.
package ntfysink
import (
@@ -31,11 +37,29 @@ import (
// the credential lives in the daemon's config (or a systemd credential),
// never in the binary.
type Config struct {
BaseURL string // e.g. http://127.0.0.1:8085 (no trailing path)
Topic string // e.g. maven (all maven notifications land here)
Username string // basic auth; empty = anonymous (won't work with deny-all)
Password string // basic auth
Timeout time.Duration // per-request; 0 = DefaultTimeout
// BaseURL — the ntfy server, no trailing path. Required.
BaseURL string `json:"base_url"`
// Topic — where maven publishes. Required. All maven notifications land
// on this one topic; severity rides the Priority header, not the topic.
Topic string `json:"topic"`
// Token — an ntfy access token, sent as a bearer. This is the preferred
// credential: ntfy scopes a token to a topic and to write-only, so a
// popped sink can push to this one topic and cannot read it back or
// touch another. Revoking it does not disturb a password anyone else
// uses. Mutually exclusive with Username.
Token string `json:"token,omitempty"`
// Username, Password — basic auth, for a server that has no tokens.
// Empty username means no credential is sent at all, which a deny-all
// server rejects.
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
// Timeout — per-request; 0 = DefaultTimeout. A dead server must not hang
// the tick loop.
Timeout time.Duration `json:"-"`
}
const DefaultTimeout = 10 * time.Second
@@ -59,6 +83,12 @@ func New(cfg Config) (*Sink, error) {
if cfg.Topic == "" {
return nil, fmt.Errorf("ntfysink: Topic is required")
}
// Refuse rather than pick. Two credentials configured means someone
// intended one of them, and guessing which would send the other nowhere
// and leave a working config that is not the one they wrote.
if cfg.Token != "" && cfg.Username != "" {
return nil, fmt.Errorf("ntfysink: set Token or Username, not both")
}
to := cfg.Timeout
if to == 0 {
to = DefaultTimeout
@@ -84,7 +114,9 @@ func (s *Sink) Send(ctx context.Context, d delivery.Sendable) error {
}
req.Header.Set("Title", "maven")
req.Header.Set("Priority", priorityFor(d).String())
if s.cfg.Username != "" {
if s.cfg.Token != "" {
req.Header.Set("Authorization", "Bearer "+s.cfg.Token)
} else if s.cfg.Username != "" {
req.SetBasicAuth(s.cfg.Username, s.cfg.Password)
}
@@ -224,6 +224,37 @@ func TestSendNoAuthWhenUsernameEmpty(t *testing.T) {
}
}
// TestSendSetsBearerToken — the deployed credential (V-649) is an ntfy access
// token scoped write-only to the maven topic, not a password. A token sent as
// basic auth is rejected by ntfy, so the header shape is the whole test.
func TestSendSetsBearerToken(t *testing.T) {
rs := newRecordingServer(t, 200, "")
srv := httptest.NewServer(rs.handler())
defer srv.Close()
sink, _ := New(Config{BaseURL: srv.URL, Topic: "maven", Token: "tk_secret"})
if err := sink.Send(context.Background(), nudgeSendable(loop.Sev3, "down")); err != nil {
t.Fatalf("Send: %v", err)
}
_, _, _, auth, _, _ := rs.snapshot()
if auth != "Bearer tk_secret" {
t.Fatalf("auth: want 'Bearer tk_secret', got %q", auth)
}
}
// TestNewRejectsBothCredentials — configuring a token and a username means one
// of them was meant and the other is a leftover. Picking either would leave a
// server that authenticates against a credential nobody wrote down.
func TestNewRejectsBothCredentials(t *testing.T) {
_, err := New(Config{BaseURL: "http://x", Topic: "maven", Token: "tk_x", Username: "maven"})
if err == nil {
t.Fatal("New accepted both a token and a username")
}
if !strings.Contains(err.Error(), "not both") {
t.Errorf("error does not say which to fix: %v", err)
}
}
func TestSendTitleIsMaven(t *testing.T) {
rs := newRecordingServer(t, 200, "")
srv := httptest.NewServer(rs.handler())
+37 -6
View File
@@ -89,8 +89,8 @@ type Poller struct {
ranker Ranker
cfg Config
nextDue map[string]time.Time
seen map[string]map[string]bool // feed → item ID, for items with no date
polled map[string]bool // feed → polled at least once in THIS process
seen map[string]*seenIDs // feed → item IDs, for items with no date
polled map[string]bool // feed → polled at least once in THIS process
}
// NewPoller wires a poller. Returns nil when there is nothing to poll — a
@@ -122,7 +122,7 @@ func NewPoller(feeds []FeedConfig, fetch Fetcher, notes Notes, marks Marks, embe
feeds: valid, fetch: fetch, notes: notes, marks: marks,
embed: embed, ranker: ranker, cfg: cfg,
nextDue: map[string]time.Time{},
seen: map[string]map[string]bool{},
seen: map[string]*seenIDs{},
polled: map[string]bool{},
}
}
@@ -283,6 +283,38 @@ func (p *Poller) mark(ctx context.Context, feed string, now time.Time) (time.Tim
return at, true
}
// maxSeenPerFeed bounds the undated-item set. It has to stay comfortably above
// any one feed's front page, or an item still listed there would fall out of the
// set and be written a second time. A few hundred entries covers the largest
// page anyone publishes, and the set only has to span one poll window plus the
// resync guard, not all of history.
const maxSeenPerFeed = 512
// seenIDs is a bounded insertion-ordered set. The map answers the lookup, the
// slice remembers what to drop first, so an undated feed cannot grow the poller
// for as long as mavend runs.
type seenIDs struct {
ids map[string]bool
order []string
}
// add records id and reports whether it was new.
func (s *seenIDs) add(id string) bool {
if s.ids == nil {
s.ids = make(map[string]bool, maxSeenPerFeed)
}
if s.ids[id] {
return false
}
s.ids[id] = true
s.order = append(s.order, id)
if len(s.order) > maxSeenPerFeed {
delete(s.ids, s.order[0])
s.order = s.order[1:]
}
return true
}
// fresh — two dedup rules, because feeds are inconsistent about dates. A dated
// item must be newer than the mark; an undated one is kept once per process by
// ID.
@@ -308,12 +340,11 @@ func (p *Poller) fresh(f FeedConfig, it Item, mark, now time.Time, resync bool)
id = it.Title
}
if p.seen[f.Name] == nil {
p.seen[f.Name] = map[string]bool{}
p.seen[f.Name] = &seenIDs{}
}
if p.seen[f.Name][id] {
if !p.seen[f.Name].add(id) {
return false
}
p.seen[f.Name][id] = true
return !resync
}
+25
View File
@@ -3,6 +3,7 @@ package rss
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
@@ -208,3 +209,27 @@ func TestNoFeedsMeansNoPoller(t *testing.T) {
t.Fatal("a feed with no name or url is not a configuration")
}
}
// An undated feed used to grow p.seen for as long as mavend ran. The set is
// bounded now, and the bound must not cost the dedupe an item still on the
// front page — only ids far older than any page fall out.
func TestSeenIDsBounded(t *testing.T) {
var s seenIDs
for i := 0; i < maxSeenPerFeed*3; i++ {
if !s.add(fmt.Sprintf("item-%d", i)) {
t.Fatalf("item-%d read as already seen", i)
}
if len(s.ids) > maxSeenPerFeed || len(s.order) > maxSeenPerFeed {
t.Fatalf("after %d inserts: ids=%d order=%d, cap is %d",
i+1, len(s.ids), len(s.order), maxSeenPerFeed)
}
}
// The newest insert is still deduped; the oldest was evicted.
last := fmt.Sprintf("item-%d", maxSeenPerFeed*3-1)
if s.add(last) {
t.Fatalf("%s read as new, so the most recent id was dropped", last)
}
if !s.add("item-0") {
t.Fatal("item-0 survived, so nothing was evicted")
}
}
+135 -11
View File
@@ -68,6 +68,13 @@ func (m *MemoryStore) Insert(ctx context.Context, id string, vec []float32, meta
// Rows under memory.NonRecallPrefix are excluded in SQL. They are speaker
// voiceprints sharing this table, and note recall must not rank them; see that
// constant for why the previous arrangement only appeared to do this.
//
// Every row is still scored, because a full scan is what picks the winners.
// What the scan does NOT do is pay for a row it is about to discard: the score
// is read straight off the stored bytes without materializing a []float32, and
// the meta blob is copied and unmarshalled only for a row that has entered the
// topK. Losers cost one dot product and nothing else. Ranking is unchanged —
// same scores, same order, same ties.
func (m *MemoryStore) Search(ctx context.Context, vec []float32, topK int) ([]memory.Result, error) {
if topK <= 0 {
topK = 10
@@ -80,30 +87,127 @@ func (m *MemoryStore) Search(ctx context.Context, vec []float32, topK int) ([]me
}
defer rows.Close()
var out []memory.Result
// sql.RawBytes hands us the driver's own buffer, valid only until the next
// Next(). Nothing here outlives the row except what topK.offer copies on a
// survivor, so the three columns cost no allocation per row.
var id, blob, metaJSON sql.RawBytes
top := newTopK(topK)
for rows.Next() {
var id, metaJSON string
var blob []byte
if err := rows.Scan(&id, &blob, &metaJSON); err != nil {
return nil, fmt.Errorf("memory: row: %w", err)
}
meta := map[string]string{}
if err := json.Unmarshal([]byte(metaJSON), &meta); err != nil {
return nil, fmt.Errorf("memory: unmarshal meta for %q: %w", id, err)
}
out = append(out, memory.Result{ID: id, Score: dot(vec, decodeVec(blob)), Meta: meta})
top.offer(dotBlob(vec, blob), id, metaJSON)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("memory: rows: %w", err)
}
sort.Slice(out, func(i, j int) bool { return out[i].Score > out[j].Score })
if topK < len(out) {
out = out[:topK]
survivors := top.sorted()
out := make([]memory.Result, 0, len(survivors))
for _, c := range survivors {
meta := map[string]string{}
if err := json.Unmarshal(c.meta, &meta); err != nil {
return nil, fmt.Errorf("memory: unmarshal meta for %q: %w", c.id, err)
}
out = append(out, memory.Result{ID: c.id, Score: c.score, Meta: meta})
}
return out, nil
}
// candidate is one row that is currently in the topK: its score, its id, and
// its meta blob copied out of the driver's buffer. The copy is the price of
// surviving, and only survivors pay it.
type candidate struct {
score float64
id string
meta []byte
}
// topK keeps the k highest-scoring candidates seen so far as a min-heap, so the
// weakest survivor is always heap[0] and one comparison decides whether a new
// row is worth copying. k is 10 in practice, so the heap is tiny and the whole
// structure fits in cache.
//
// It is a plain slice with hand-written sift operations rather than
// container/heap, because that interface boxes every element into an `any` on
// Push and costs an allocation per surviving row.
type topK struct {
k int
heap []candidate
}
func newTopK(k int) *topK {
return &topK{k: k, heap: make([]candidate, 0, k)}
}
// offer admits a row if it beats the weakest survivor, or if the heap is not
// full yet. id and meta are the driver's buffers and are copied here, never
// retained.
//
// A row that only ties the weakest survivor does not displace it, so among
// equal scores the earliest k rows are kept. The full sort this replaced used
// sort.Slice, which is not stable, so it broke such a tie arbitrarily. That is
// the ONE observable difference between the two, and it is deliberate:
// deterministic beats arbitrary.
//
// It is not academic. Under the real embedder an exact tie means duplicate
// vectors and nothing in the recall eval moved (V-643). Under the hash
// embedder the eval's deterministic floor uses, ties are everywhere — it is
// bag-of-words, so every note sharing no word with the query scores exactly 0
// — and recall@3 on that run moved 74.1% to 81.5% purely because the zeros now
// come out in a fixed order. Neither number measures retrieval. recall@1 and
// false recall, which the eval actually asserts, are unchanged on both runs.
func (t *topK) offer(score float64, id, meta []byte) {
if t.k == 0 {
return
}
if len(t.heap) < t.k {
t.heap = append(t.heap, candidate{score: score, id: string(id), meta: append([]byte(nil), meta...)})
t.up(len(t.heap) - 1)
return
}
if score <= t.heap[0].score {
return
}
t.heap[0] = candidate{score: score, id: string(id), meta: append([]byte(nil), meta...)}
t.down(0)
}
func (t *topK) up(i int) {
for i > 0 {
parent := (i - 1) / 2
if t.heap[parent].score <= t.heap[i].score {
return
}
t.heap[parent], t.heap[i] = t.heap[i], t.heap[parent]
i = parent
}
}
func (t *topK) down(i int) {
for {
l, r, small := 2*i+1, 2*i+2, i
if l < len(t.heap) && t.heap[l].score < t.heap[small].score {
small = l
}
if r < len(t.heap) && t.heap[r].score < t.heap[small].score {
small = r
}
if small == i {
return
}
t.heap[small], t.heap[i] = t.heap[i], t.heap[small]
i = small
}
}
// sorted drains the heap into descending score order — what Search returns.
func (t *topK) sorted() []candidate {
out := t.heap
sort.Slice(out, func(i, j int) bool { return out[i].score > out[j].score })
return out
}
// ByPrefix returns every row whose id starts with prefix, vectors included.
//
// This is not a similarity query and deliberately does not score anything:
@@ -240,6 +344,26 @@ func decodeVec(b []byte) []float32 {
return v
}
// dotBlob is dot against a vector still in its stored encoding, so scoring a
// row the query is about to discard does not allocate the []float32 that
// decodeVec would build. Same arithmetic, same order of operations, so it
// returns bit-identical scores to dot(a, decodeVec(b)).
//
// A blob whose length isn't a multiple of 4 is truncated to the whole-element
// prefix, matching decodeVec, and a length mismatch is 0, matching dot.
func dotBlob(a []float32, b []byte) float64 {
n := len(b) / 4
if len(a) != n || n == 0 {
return 0
}
var sum float64
for i := 0; i < n; i++ {
f := math.Float32frombits(binary.LittleEndian.Uint32(b[4*i:]))
sum += float64(a[i]) * float64(f)
}
return sum
}
// dot is the cosine similarity for L2-normalized vectors (mismatched lengths ⇒
// 0, matching internal/memory's cosine).
func dot(a, b []float32) float64 {
+77
View File
@@ -0,0 +1,77 @@
package store
import (
"context"
"fmt"
"math"
"math/rand"
"path/filepath"
"testing"
)
// benchDim is the resident embedder's width (multilingual-e5-small, 384), so
// the per-row decode cost the benchmark measures is the real one.
const benchDim = 384
// seedMemVectors fills a fresh store with n L2-normalized rows carrying a meta
// blob the size recall actually stores — the note text plus its type — because
// the cost this benchmark exists to measure is unmarshalling that blob for
// every row when only topK survivors need it.
func seedMemVectors(tb testing.TB, n int) *MemoryStore {
tb.Helper()
path := filepath.Join(tb.TempDir(), "mem_bench.db")
st, err := Open(context.Background(), path)
if err != nil {
tb.Fatalf("Open: %v", err)
}
tb.Cleanup(func() { _ = st.Close() })
m := st.VectorMemory()
rng := rand.New(rand.NewSource(1))
ctx := context.Background()
for i := 0; i < n; i++ {
if err := m.Insert(ctx, fmt.Sprintf("note:%d", i), randUnitVec(rng, benchDim), map[string]string{
"type": "note",
"text": fmt.Sprintf("заметка номер %d о том, что надо не забыть сделать на неделе", i),
}); err != nil {
tb.Fatalf("Insert %d: %v", i, err)
}
}
return m
}
func randUnitVec(rng *rand.Rand, dim int) []float32 {
v := make([]float32, dim)
var norm float64
for i := range v {
f := rng.NormFloat64()
v[i] = float32(f)
norm += f * f
}
norm = math.Sqrt(norm)
for i := range v {
v[i] = float32(float64(v[i]) / norm)
}
return v
}
// BenchmarkMemoryStoreSearch measures one recall query against a store of n
// rows. Row counts bracket the documented scale: 1000 is a plausible today,
// 10000 is the "thousands, not millions" ceiling the type doc claims a full
// scan is fine at.
func BenchmarkMemoryStoreSearch(b *testing.B) {
for _, n := range []int{1000, 10000} {
b.Run(fmt.Sprintf("rows=%d", n), func(b *testing.B) {
m := seedMemVectors(b, n)
q := randUnitVec(rand.New(rand.NewSource(2)), benchDim)
ctx := context.Background()
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := m.Search(ctx, q, 10); err != nil {
b.Fatal(err)
}
}
})
}
}
+107
View File
@@ -0,0 +1,107 @@
package store
import (
"context"
"fmt"
"math/rand"
"sort"
"testing"
"github.com/kami/maven/internal/memory"
)
// naiveSearch is the implementation Search replaced: score every row into a
// slice, sort the whole slice, truncate. It stays in the test file as the
// reference the bounded-heap version is judged against, because "recall must
// not change" is a claim about output, not about the code that produces it.
func naiveSearch(t *testing.T, m *MemoryStore, vec []float32, topK int) []memory.Result {
t.Helper()
rows, err := m.db.QueryContext(context.Background(),
`SELECT id, vec FROM memory_vectors WHERE id NOT LIKE ? ESCAPE '\'`,
escapeLike(memory.NonRecallPrefix)+"%")
if err != nil {
t.Fatalf("naive scan: %v", err)
}
defer rows.Close()
var out []memory.Result
for rows.Next() {
var id string
var blob []byte
if err := rows.Scan(&id, &blob); err != nil {
t.Fatalf("naive row: %v", err)
}
out = append(out, memory.Result{ID: id, Score: dot(vec, decodeVec(blob))})
}
if err := rows.Err(); err != nil {
t.Fatalf("naive rows: %v", err)
}
sort.Slice(out, func(i, j int) bool { return out[i].Score > out[j].Score })
if topK < len(out) {
out = out[:topK]
}
return out
}
// TestMemoryStoreSearchMatchesNaive is the constraint on V-643: the bounded
// heap must return exactly what a full scan and sort returned. Distinct random
// vectors, so no two scores tie and the ranking is total — a mismatch here is
// arithmetic or heap logic, not a tie-break difference.
func TestMemoryStoreSearchMatchesNaive(t *testing.T) {
ctx := context.Background()
m := newMemTestStore(t).VectorMemory()
rng := rand.New(rand.NewSource(7))
const rows, dim = 500, 64
for i := 0; i < rows; i++ {
if err := m.Insert(ctx, fmt.Sprintf("n%d", i), randUnitVec(rng, dim), map[string]string{
"text": fmt.Sprintf("note %d", i),
}); err != nil {
t.Fatalf("Insert %d: %v", i, err)
}
}
for _, topK := range []int{1, 3, 10, 50, rows, rows + 100} {
q := randUnitVec(rng, dim)
got, err := m.Search(ctx, q, topK)
if err != nil {
t.Fatalf("Search topK=%d: %v", topK, err)
}
want := naiveSearch(t, m, q, topK)
if len(got) != len(want) {
t.Fatalf("topK=%d: got %d results, naive returned %d", topK, len(got), len(want))
}
for i := range want {
if got[i].ID != want[i].ID {
t.Errorf("topK=%d rank %d: got %q, naive says %q", topK, i, got[i].ID, want[i].ID)
}
if got[i].Score != want[i].Score {
t.Errorf("topK=%d rank %d (%s): score %v, naive says %v",
topK, i, got[i].ID, got[i].Score, want[i].Score)
}
}
if len(got) > 0 && got[0].Meta["text"] == "" {
t.Errorf("topK=%d: survivor %s has no meta — it was never unmarshalled", topK, got[0].ID)
}
}
}
// TestDotBlobMatchesDot pins the claim in dotBlob's doc comment: reading the
// vector out of its stored bytes is bit-identical to decoding it first. Scores
// feed a gate with a 0.008 margin, so "close enough" is not the bar.
func TestDotBlobMatchesDot(t *testing.T) {
rng := rand.New(rand.NewSource(11))
for i := 0; i < 200; i++ {
a := randUnitVec(rng, 384)
b := randUnitVec(rng, 384)
if got, want := dotBlob(a, encodeVec(b)), dot(a, b); got != want {
t.Fatalf("dotBlob = %v, dot = %v", got, want)
}
}
// Length mismatch is 0 in both, and so is an empty vector.
if got := dotBlob([]float32{1, 0}, encodeVec([]float32{1, 0, 0})); got != 0 {
t.Errorf("mismatched lengths scored %v, want 0", got)
}
if got := dotBlob(nil, nil); got != 0 {
t.Errorf("empty scored %v, want 0", got)
}
}
+22
View File
@@ -295,6 +295,27 @@ func (f *Fetcher) checkURL(u *url.URL) error {
return nil
}
// pruneHostsAbove is when pruneLocked bothers to walk the map. Below it the
// walk costs more than the entries do, and `crawl.on_demand` means the host set
// is whatever he names out loud, so it grows slowly.
const pruneHostsAbove = 64
// pruneLocked drops hosts whose last dial is further back than HostInterval.
// Such an entry cannot delay anything — waitTurn would let the next request
// through immediately — so keeping it only holds memory for the life of the
// process. Caller holds f.mu.
func (f *Fetcher) pruneLocked(now time.Time) {
if len(f.last) <= pruneHostsAbove {
return
}
cutoff := now.Add(-f.cfg.HostInterval)
for h, at := range f.last {
if at.Before(cutoff) {
delete(f.last, h)
}
}
}
// waitTurn blocks until this host's rate-limit interval has elapsed. It holds
// no lock while sleeping, so two hosts never wait on each other.
func (f *Fetcher) waitTurn(ctx context.Context, host string) error {
@@ -304,6 +325,7 @@ func (f *Fetcher) waitTurn(ctx context.Context, host string) error {
earliest := f.last[host].Add(f.cfg.HostInterval)
if !now.Before(earliest) {
f.last[host] = now
f.pruneLocked(now)
f.mu.Unlock()
return nil
}
+35
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"net/http"
@@ -318,3 +319,37 @@ func TestPostObeysDenylist(t *testing.T) {
t.Fatalf("error = %v, want ErrBlocked", err)
}
}
// f.last used to hold one entry per host ever dialed, for the life of the
// process. A host whose last dial is older than HostInterval cannot delay
// anything, so it is dropped once the map is worth walking.
func TestHostRateMapIsPruned(t *testing.T) {
f := New(Config{HostInterval: time.Minute, AllowPrivate: true})
stale := time.Now().Add(-time.Hour)
for i := 0; i < pruneHostsAbove*2; i++ {
f.last[fmt.Sprintf("h%d.example", i)] = stale
}
// One real turn is what triggers the sweep.
if err := f.waitTurn(context.Background(), "fresh.example"); err != nil {
t.Fatal(err)
}
if len(f.last) != 1 {
t.Fatalf("len(f.last) = %d after the sweep, want 1 (only the host just dialed)", len(f.last))
}
if _, ok := f.last["fresh.example"]; !ok {
t.Fatal("the host just dialed was pruned, so its own rate limit is lost")
}
// A host inside the interval is kept: pruning must not hand out a free turn.
f.last["recent.example"] = time.Now()
for i := 0; i < pruneHostsAbove*2; i++ {
f.last[fmt.Sprintf("g%d.example", i)] = stale
}
if err := f.waitTurn(context.Background(), "other.example"); err != nil {
t.Fatal(err)
}
if _, ok := f.last["recent.example"]; !ok {
t.Fatal("a host dialed inside HostInterval was pruned")
}
}
+143
View File
@@ -0,0 +1,143 @@
#!/usr/bin/env bash
# audit.sh — the repo inventory, in one command.
#
# Every section here was reconstructed by hand, from scratch, in session after
# session: 93 grep sweeps across the four longest ones before a single edit was
# made. The answers move slowly and the sweeps did not, so they are written down
# once here instead.
#
# It prints and never writes. A committed inventory file goes stale silently and
# then lies; a report you regenerate cannot.
#
# Read-only. Safe to run at any point, including mid-conflict.
#
# make audit # everything
# make audit SECTION=todo # one section: loc, todo, stubs, docs, tests, gaps
set -uo pipefail
cd "$(dirname "$0")/.." || exit 1
SECTION="${SECTION:-all}"
want() { [ "$SECTION" = all ] || [ "$SECTION" = "$1" ]; }
rule() { printf '\n=== %s %s\n' "$1" "$(printf '%.0s=' $(seq 1 $((66 - ${#1}))))"; }
# git grep over tracked files only. deps/ and models/ are gitignored and huge,
# and a plain grep -r walks into both.
g() { git grep -nI "$@" 2>/dev/null; }
printf 'Maven repo inventory @ %s (%s)\n' \
"$(git rev-parse --short HEAD 2>/dev/null || echo '?')" \
"$(git log -1 --format=%cs 2>/dev/null || echo '?')"
# --- loc -------------------------------------------------------------------
# Non-test Go lines per package. Size is the cheapest proxy for "where does the
# complexity actually sit", and it is the first thing every audit asked for.
if want loc; then
rule "PACKAGES BY LOC (non-test)"
for d in $(find ./cmd ./internal ./pkg -maxdepth 2 -type d 2>/dev/null | sort); do
files=$(find "$d" -maxdepth 1 -name '*.go' ! -name '*_test.go' 2>/dev/null)
[ -z "$files" ] && continue
n=$(printf '%s\n' "$files" | wc -l)
l=$(printf '%s\0' $files | xargs -0 cat 2>/dev/null | wc -l)
printf '%7d %3d files %s\n' "$l" "$n" "$d"
done | sort -rn
fi
# --- todo ------------------------------------------------------------------
if want todo; then
rule "TODO / FIXME / XXX / HACK / BUG (non-test)"
g -E '(^|[^a-zA-Z])(TODO|FIXME|XXX|HACK|BUG:)' -- 'cmd/**/*.go' 'internal/**/*.go' 'pkg/**/*.go' \
| grep -v '_test\.go:' | sed 's/^/ /' || echo " none"
fi
# --- stubs -----------------------------------------------------------------
# Code only, never doc comments. "not wired" is this repo's design vocabulary
# for a nil dependency and appears in ~30 comments that describe working code,
# so searching prose here reports the architecture back as a gap. Likewise
# internal/ipc/unimplemented.go is skipped whole: the file IS the deliberate
# Unimplemented*Server pattern, not 60 missing methods. "placeholder" is not a
# term here either -- it names real identifiers (SQL placeholders,
# Deck.RequirePlaceholder, tokenPlaceholder) and matched 16 working lines.
if want stubs; then
rule "STUBS / NOT IMPLEMENTED (code, non-test)"
g -iE 'not (yet )?implemented|unimplemented|пока не умею|panic\("TODO' \
-- 'cmd/**/*.go' 'internal/**/*.go' 'pkg/**/*.go' \
| grep -v '_test\.go:' \
| grep -v '^internal/ipc/unimplemented\.go:' \
| grep -vE '^[^:]+:[0-9]+:[[:space:]]*//' \
| sed 's/^/ /' || echo " none"
# CoreAPI stub parity. A stub missing from unimplemented.go breaks the build
# via `var _ CoreAPI = UnimplementedCoreAPI{}`. A stub left behind after its
# method leaves an interface compiles forever and is caught by nothing, so it
# is counted here until V-652 turns it into a test.
printf '\n CoreAPI stub parity:\n'
python3 - <<'PY' 2>/dev/null | sed 's/^/ /' || echo " (skipped: python3 unavailable)"
import re
src = open('internal/ipc/coreapi.go').read()
decl = set()
for m in re.finditer(r'type (\w+API) interface \{(.*?)\n\}', src, re.S):
decl |= set(re.findall(r'^\t([A-Z]\w*)\(', m.group(2), re.M))
stub = set(re.findall(r'func \(UnimplementedCoreAPI\) (\w+)\(',
open('internal/ipc/unimplemented.go').read()))
print(f"{len(decl)} declared, {len(stub)} stubbed")
for name in sorted(stub - decl):
print(f"STALE {name} (stubbed, on no interface)")
for name in sorted(decl - stub):
print(f"MISSING {name} (declared, no stub)")
PY
fi
# --- docs ------------------------------------------------------------------
# Living tier only. docs/evals/ are dated measurements that are never edited
# after the day, and docs/archive/ is dead by definition, so neither can be
# stale. Age is against the recorded date, not against HEAD: every commit moves
# HEAD, so a sha comparison would mark the whole tier stale every day.
if want docs; then
rule "LIVING DOCS — Last verified"
today=$(date +%s)
for f in docs/*.md; do
[ -e "$f" ] || continue
line=$(grep -m1 -o 'Last verified: *[0-9-]\{8,10\}[^ ]*\( *@ *[0-9a-f]\{7,\}\)\?' "$f" 2>/dev/null)
if [ -z "$line" ]; then
printf ' %-44s %s\n' "$(basename "$f")" "MISSING"
continue
fi
d=$(printf '%s' "$line" | grep -o '[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}' | head -1)
age=""
if [ -n "$d" ] && when=$(date -d "$d" +%s 2>/dev/null); then
days=$(( (today - when) / 86400 ))
age="${days}d"
[ "$days" -gt 30 ] && age="${days}d <-- STALE"
fi
printf ' %-44s %-34s %s\n' "$(basename "$f")" "${line#Last verified: }" "$age"
done
fi
# --- tests -----------------------------------------------------------------
if want tests; then
rule "TEST SHAPE"
printf ' benchmarks : %s\n' "$(g -c 'func Benchmark' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
printf ' fuzz : %s\n' "$(g -c 'func Fuzz' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
printf ' table t.Run: %s\n' "$(g -c 't.Run(' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
printf ' golden files: %s\n' "$(g -l 'golden' -- '**/*_test.go' | wc -l)"
fi
# --- gaps ------------------------------------------------------------------
# A package with production code and no test file at all. Not a verdict — some
# are pure wiring — but it is the list worth looking at before adding more.
if want gaps; then
rule "PACKAGES WITH NO TEST FILE"
found=0
for d in $(find ./cmd ./internal ./pkg -maxdepth 2 -type d 2>/dev/null | sort); do
ls "$d"/*.go >/dev/null 2>&1 || continue
ls "$d"/*_test.go >/dev/null 2>&1 && continue
n=$(ls "$d"/*.go 2>/dev/null | wc -l)
l=$(cat "$d"/*.go 2>/dev/null | wc -l)
printf ' %6d lines %2d files %s\n' "$l" "$n" "$d"
found=1
done
[ "$found" = 0 ] && echo " none"
fi
exit 0