Run one test and audit the repo without retyping either (V-653)
Two commands replace work that 66 sessions of transcripts show being redone by hand. `make t` replaces the CGO preamble, pasted 391 times across past sessions and documented in CLAUDE.md as the way to do it. It also sets MAVEN_ONNX_LIB, which that recipe did not: the four TestONNX* measurements self-skip without it and the run still prints "ok", so every targeted eval done the old way reported the hash ratchet while reading as a real embedder score. -race keeps it honest against `make test`, -count=1 keeps a stale cache from passing as a result. `make audit` replaces the inventory sweep. The four longest sessions spent 93 greps rebuilding it before their first edit. Runs in 0.75s. Its stub search is narrower than the sweeps were, on purpose. "not wired" is this repo's word for a nil dependency and matched ~30 comments describing working code; "placeholder" names real identifiers and matched 16 more; internal/ipc/unimplemented.go is the deliberate Unimplemented*Server pattern, not 60 gaps. A gap report that reports the architecture back at you is one nobody reads twice.
This commit is contained in:
@@ -58,16 +58,27 @@ make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build w
|
|||||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||||
```
|
```
|
||||||
|
|
||||||
Run a single test (must carry the CGO env for packages that touch STT/TTS/voice):
|
Run one package or one test with `make t`. **Do not hand-write the CGO preamble.**
|
||||||
|
Past sessions pasted it about 390 times. That is where the shell-quoting failures
|
||||||
|
came from. This box runs zsh, so an unquoted `-run Test*` or `--include=*.go`
|
||||||
|
dies on "no matches found" before `go` is ever reached.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
CGO_CFLAGS="-I$(pwd)/deps/include -I$(pwd)/deps/whisper.cpp/ggml/include" \
|
make t PKG=./internal/router/
|
||||||
CGO_LDFLAGS="-L$(pwd)/deps/lib -Wl,-rpath,$(pwd)/deps/lib" \
|
make t PKG=./cmd/mavend/ RUN=TestSimulator
|
||||||
LD_LIBRARY_PATH="$(pwd)/deps/lib" \
|
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
|
||||||
deps/go/go/bin/go test -run TestName ./internal/router/
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Pure-Go packages (`router`, `memory`, `mavweb`, …) run under a plain `go test ./pkg/`.
|
`t` carries `-race`, so a green `make t` cannot turn red under `make test`. It carries
|
||||||
|
`-count=1`, so a cached PASS from before your edit is never mistaken for a result.
|
||||||
|
|
||||||
|
It also sets `MAVEN_ONNX_LIB`, which the hand-written recipe did not. The four
|
||||||
|
`TestONNX*` measurements self-skip when that variable is unset. The run still prints
|
||||||
|
`ok`. So every targeted eval done the old way reported the hash ratchet while reading
|
||||||
|
as a real embedder score.
|
||||||
|
|
||||||
|
Pure-Go packages (`router`, `memory`, `mavweb`, …) also run under a plain `go test ./pkg/`,
|
||||||
|
but `make t` works everywhere and is one thing to remember.
|
||||||
|
|
||||||
## The daemons (`cmd/`)
|
## The daemons (`cmd/`)
|
||||||
|
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
|||||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||||
|
|
||||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
|
.PHONY: t audit simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
|
||||||
|
|
||||||
all: build
|
all: build
|
||||||
|
|
||||||
@@ -128,6 +128,35 @@ test: fmt-check vet
|
|||||||
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||||
$(GO) test -race -coverprofile=coverage.out ./internal/... ./cmd/...
|
$(GO) test -race -coverprofile=coverage.out ./internal/... ./cmd/...
|
||||||
|
|
||||||
|
# t — run ONE package or ONE test with the toolchain env already wired. This is
|
||||||
|
# the iteration target; `test` is the gate. Reach for it instead of pasting the
|
||||||
|
# CGO_CFLAGS/CGO_LDFLAGS/LD_LIBRARY_PATH preamble by hand, which is how it was
|
||||||
|
# done ~390 times across past sessions and is where the shell-quoting failures
|
||||||
|
# came from -- the interactive shell here is zsh, and an unquoted `-run Test*`
|
||||||
|
# or `--include=*.go` dies on "no matches found" before go ever starts.
|
||||||
|
#
|
||||||
|
# make t # whole tree (same scope as `test`)
|
||||||
|
# make t PKG=./internal/router/
|
||||||
|
# make t PKG=./cmd/mavend/ RUN=TestSimulator
|
||||||
|
# make t PKG=./internal/router/eval/ RUN='TestONNX' V=1
|
||||||
|
# make t PKG=./internal/store/ RACE=0 # drop -race when iterating hot
|
||||||
|
#
|
||||||
|
# -race is on by default so a green `make t` cannot turn red under `make test`.
|
||||||
|
# -count=1 because a cached PASS from before your edit is worse than no answer.
|
||||||
|
# MAVEN_ONNX_LIB is set for the same reason: the four TestONNX* measurements
|
||||||
|
# self-skip when it is unset, so a targeted eval run would otherwise report the
|
||||||
|
# deterministic hash ratchet and look like it scored the real embedder.
|
||||||
|
PKG ?= ./internal/... ./cmd/...
|
||||||
|
RUN ?=
|
||||||
|
V ?=
|
||||||
|
RACE ?= 1
|
||||||
|
|
||||||
|
t:
|
||||||
|
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||||
|
MAVEN_ONNX_LIB="$(MAVEN_ONNX_LIB)" \
|
||||||
|
$(GO) test $(if $(V),-v,) $(if $(filter-out 0,$(RACE)),-race,) -count=1 \
|
||||||
|
$(if $(RUN),-run '$(RUN)',) $(PKG)
|
||||||
|
|
||||||
# eval-router — score the held-out RU routing fixture (internal/router/eval).
|
# eval-router — score the held-out RU routing fixture (internal/router/eval).
|
||||||
# Verbose so the report tables land in the terminal. MAVEN_ONNX_LIB points the
|
# Verbose so the report tables land in the terminal. MAVEN_ONNX_LIB points the
|
||||||
# prod-representative baseline at the vendored runtime; override it or set it
|
# prod-representative baseline at the vendored runtime; override it or set it
|
||||||
@@ -189,6 +218,16 @@ eval-models:
|
|||||||
# scores the fixtures against ggml-small and self-skips when the model is
|
# scores the fixtures against ggml-small and self-skips when the model is
|
||||||
# absent, and TestGoldenFixturesAreCanonical, which checks the committed audio
|
# absent, and TestGoldenFixturesAreCanonical, which checks the committed audio
|
||||||
# and the manifest with no model at all.
|
# and the manifest with no model at all.
|
||||||
|
# audit — the repo inventory: LOC per package, open TODOs, real stubs, living-doc
|
||||||
|
# staleness, test shape, packages with no test. Read-only, prints, writes nothing.
|
||||||
|
# Run it instead of rebuilding the same greps by hand; past sessions spent 93 of
|
||||||
|
# them on this before their first edit. SECTION=loc|todo|stubs|docs|tests|gaps
|
||||||
|
# narrows it.
|
||||||
|
SECTION ?= all
|
||||||
|
|
||||||
|
audit:
|
||||||
|
@SECTION="$(SECTION)" ./scripts/audit.sh
|
||||||
|
|
||||||
stt-fixtures:
|
stt-fixtures:
|
||||||
./scripts/gen-stt-fixtures.sh
|
./scripts/gen-stt-fixtures.sh
|
||||||
|
|
||||||
|
|||||||
Executable
+143
@@ -0,0 +1,143 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# audit.sh — the repo inventory, in one command.
|
||||||
|
#
|
||||||
|
# Every section here was reconstructed by hand, from scratch, in session after
|
||||||
|
# session: 93 grep sweeps across the four longest ones before a single edit was
|
||||||
|
# made. The answers move slowly and the sweeps did not, so they are written down
|
||||||
|
# once here instead.
|
||||||
|
#
|
||||||
|
# It prints and never writes. A committed inventory file goes stale silently and
|
||||||
|
# then lies; a report you regenerate cannot.
|
||||||
|
#
|
||||||
|
# Read-only. Safe to run at any point, including mid-conflict.
|
||||||
|
#
|
||||||
|
# make audit # everything
|
||||||
|
# make audit SECTION=todo # one section: loc, todo, stubs, docs, tests, gaps
|
||||||
|
|
||||||
|
set -uo pipefail
|
||||||
|
cd "$(dirname "$0")/.." || exit 1
|
||||||
|
|
||||||
|
SECTION="${SECTION:-all}"
|
||||||
|
want() { [ "$SECTION" = all ] || [ "$SECTION" = "$1" ]; }
|
||||||
|
rule() { printf '\n=== %s %s\n' "$1" "$(printf '%.0s=' $(seq 1 $((66 - ${#1}))))"; }
|
||||||
|
|
||||||
|
# git grep over tracked files only. deps/ and models/ are gitignored and huge,
|
||||||
|
# and a plain grep -r walks into both.
|
||||||
|
g() { git grep -nI "$@" 2>/dev/null; }
|
||||||
|
|
||||||
|
printf 'Maven repo inventory @ %s (%s)\n' \
|
||||||
|
"$(git rev-parse --short HEAD 2>/dev/null || echo '?')" \
|
||||||
|
"$(git log -1 --format=%cs 2>/dev/null || echo '?')"
|
||||||
|
|
||||||
|
# --- loc -------------------------------------------------------------------
|
||||||
|
# Non-test Go lines per package. Size is the cheapest proxy for "where does the
|
||||||
|
# complexity actually sit", and it is the first thing every audit asked for.
|
||||||
|
if want loc; then
|
||||||
|
rule "PACKAGES BY LOC (non-test)"
|
||||||
|
for d in $(find ./cmd ./internal ./pkg -maxdepth 2 -type d 2>/dev/null | sort); do
|
||||||
|
files=$(find "$d" -maxdepth 1 -name '*.go' ! -name '*_test.go' 2>/dev/null)
|
||||||
|
[ -z "$files" ] && continue
|
||||||
|
n=$(printf '%s\n' "$files" | wc -l)
|
||||||
|
l=$(printf '%s\0' $files | xargs -0 cat 2>/dev/null | wc -l)
|
||||||
|
printf '%7d %3d files %s\n' "$l" "$n" "$d"
|
||||||
|
done | sort -rn
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- todo ------------------------------------------------------------------
|
||||||
|
if want todo; then
|
||||||
|
rule "TODO / FIXME / XXX / HACK / BUG (non-test)"
|
||||||
|
g -E '(^|[^a-zA-Z])(TODO|FIXME|XXX|HACK|BUG:)' -- 'cmd/**/*.go' 'internal/**/*.go' 'pkg/**/*.go' \
|
||||||
|
| grep -v '_test\.go:' | sed 's/^/ /' || echo " none"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- stubs -----------------------------------------------------------------
|
||||||
|
# Code only, never doc comments. "not wired" is this repo's design vocabulary
|
||||||
|
# for a nil dependency and appears in ~30 comments that describe working code,
|
||||||
|
# so searching prose here reports the architecture back as a gap. Likewise
|
||||||
|
# internal/ipc/unimplemented.go is skipped whole: the file IS the deliberate
|
||||||
|
# Unimplemented*Server pattern, not 60 missing methods. "placeholder" is not a
|
||||||
|
# term here either -- it names real identifiers (SQL placeholders,
|
||||||
|
# Deck.RequirePlaceholder, tokenPlaceholder) and matched 16 working lines.
|
||||||
|
if want stubs; then
|
||||||
|
rule "STUBS / NOT IMPLEMENTED (code, non-test)"
|
||||||
|
g -iE 'not (yet )?implemented|unimplemented|пока не умею|panic\("TODO' \
|
||||||
|
-- 'cmd/**/*.go' 'internal/**/*.go' 'pkg/**/*.go' \
|
||||||
|
| grep -v '_test\.go:' \
|
||||||
|
| grep -v '^internal/ipc/unimplemented\.go:' \
|
||||||
|
| grep -vE '^[^:]+:[0-9]+:[[:space:]]*//' \
|
||||||
|
| sed 's/^/ /' || echo " none"
|
||||||
|
|
||||||
|
# CoreAPI stub parity. A stub missing from unimplemented.go breaks the build
|
||||||
|
# via `var _ CoreAPI = UnimplementedCoreAPI{}`. A stub left behind after its
|
||||||
|
# method leaves an interface compiles forever and is caught by nothing, so it
|
||||||
|
# is counted here until V-652 turns it into a test.
|
||||||
|
printf '\n CoreAPI stub parity:\n'
|
||||||
|
python3 - <<'PY' 2>/dev/null | sed 's/^/ /' || echo " (skipped: python3 unavailable)"
|
||||||
|
import re
|
||||||
|
src = open('internal/ipc/coreapi.go').read()
|
||||||
|
decl = set()
|
||||||
|
for m in re.finditer(r'type (\w+API) interface \{(.*?)\n\}', src, re.S):
|
||||||
|
decl |= set(re.findall(r'^\t([A-Z]\w*)\(', m.group(2), re.M))
|
||||||
|
stub = set(re.findall(r'func \(UnimplementedCoreAPI\) (\w+)\(',
|
||||||
|
open('internal/ipc/unimplemented.go').read()))
|
||||||
|
print(f"{len(decl)} declared, {len(stub)} stubbed")
|
||||||
|
for name in sorted(stub - decl):
|
||||||
|
print(f"STALE {name} (stubbed, on no interface)")
|
||||||
|
for name in sorted(decl - stub):
|
||||||
|
print(f"MISSING {name} (declared, no stub)")
|
||||||
|
PY
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- docs ------------------------------------------------------------------
|
||||||
|
# Living tier only. docs/evals/ are dated measurements that are never edited
|
||||||
|
# after the day, and docs/archive/ is dead by definition, so neither can be
|
||||||
|
# stale. Age is against the recorded date, not against HEAD: every commit moves
|
||||||
|
# HEAD, so a sha comparison would mark the whole tier stale every day.
|
||||||
|
if want docs; then
|
||||||
|
rule "LIVING DOCS — Last verified"
|
||||||
|
today=$(date +%s)
|
||||||
|
for f in docs/*.md; do
|
||||||
|
[ -e "$f" ] || continue
|
||||||
|
line=$(grep -m1 -o 'Last verified: *[0-9-]\{8,10\}[^ ]*\( *@ *[0-9a-f]\{7,\}\)\?' "$f" 2>/dev/null)
|
||||||
|
if [ -z "$line" ]; then
|
||||||
|
printf ' %-44s %s\n' "$(basename "$f")" "MISSING"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
d=$(printf '%s' "$line" | grep -o '[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}' | head -1)
|
||||||
|
age=""
|
||||||
|
if [ -n "$d" ] && when=$(date -d "$d" +%s 2>/dev/null); then
|
||||||
|
days=$(( (today - when) / 86400 ))
|
||||||
|
age="${days}d"
|
||||||
|
[ "$days" -gt 30 ] && age="${days}d <-- STALE"
|
||||||
|
fi
|
||||||
|
printf ' %-44s %-34s %s\n' "$(basename "$f")" "${line#Last verified: }" "$age"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- tests -----------------------------------------------------------------
|
||||||
|
if want tests; then
|
||||||
|
rule "TEST SHAPE"
|
||||||
|
printf ' benchmarks : %s\n' "$(g -c 'func Benchmark' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
|
||||||
|
printf ' fuzz : %s\n' "$(g -c 'func Fuzz' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
|
||||||
|
printf ' table t.Run: %s\n' "$(g -c 't.Run(' -- '**/*_test.go' | awk -F: '{s+=$2} END{print s+0}')"
|
||||||
|
printf ' golden files: %s\n' "$(g -l 'golden' -- '**/*_test.go' | wc -l)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- gaps ------------------------------------------------------------------
|
||||||
|
# A package with production code and no test file at all. Not a verdict — some
|
||||||
|
# are pure wiring — but it is the list worth looking at before adding more.
|
||||||
|
if want gaps; then
|
||||||
|
rule "PACKAGES WITH NO TEST FILE"
|
||||||
|
found=0
|
||||||
|
for d in $(find ./cmd ./internal ./pkg -maxdepth 2 -type d 2>/dev/null | sort); do
|
||||||
|
ls "$d"/*.go >/dev/null 2>&1 || continue
|
||||||
|
ls "$d"/*_test.go >/dev/null 2>&1 && continue
|
||||||
|
n=$(ls "$d"/*.go 2>/dev/null | wc -l)
|
||||||
|
l=$(cat "$d"/*.go 2>/dev/null | wc -l)
|
||||||
|
printf ' %6d lines %2d files %s\n' "$l" "$n" "$d"
|
||||||
|
found=1
|
||||||
|
done
|
||||||
|
[ "$found" = 0 ] && echo " none"
|
||||||
|
fi
|
||||||
|
|
||||||
|
exit 0
|
||||||
Reference in New Issue
Block a user