Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0ca5748699 | |||
| b43bb265b5 |
@@ -69,19 +69,30 @@ protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from g
|
|||||||
|
|
||||||
## Routing — read this before touching the router
|
## Routing — read this before touching the router
|
||||||
|
|
||||||
`internal/router/` has TWO layered engines and the committed default is an **interim
|
`internal/router/` has TWO layered engines. **The LLM router is now the default and it is
|
||||||
stopgap, not the intended design** (see memory `routing-architecture-target`):
|
on in deploy** — this section used to say it was wired `nil`, which stopped being true on
|
||||||
|
2026-07-31.
|
||||||
|
|
||||||
- **Target (REARCH.md):** LLM-as-router. One resident Qwen3-1.7B (`llmrouter.go`) emits
|
- **LLM router (the intended design, REARCH.md):** the resident Qwen3-1.7B (`llmrouter.go`)
|
||||||
GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is demoted
|
emits GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is
|
||||||
from a routing gate to a RAG hint.
|
demoted from a routing gate to a RAG hint. Wired at `voice.go:214` via
|
||||||
- **Current stopgap:** `llmrouter` is wired `nil` (around `voice.go`), so the
|
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`; the flag is `voice.llm_router`
|
||||||
`classifier.go` + `embedder.go` nearest-neighbour cascade actually runs. It routes by
|
(`config.go`), `DefaultLLMRouter` is **on**, and `deploy/mavend.json` sets it `true`.
|
||||||
similarity to frozen seed phrases — the known cause of weak RU query handling.
|
- **Classifier cascade (the failure floor, not dead code):** `classifier.go` +
|
||||||
|
`embedder.go` nearest-neighbour over frozen seed phrases. It runs when the LLM router is
|
||||||
|
off, when there is no llama-server to talk to (`pickLLMRouter` logs that and degrades),
|
||||||
|
and on any per-turn LLM error. Do not delete it — routing by seed similarity is the known
|
||||||
|
cause of weak RU query handling, but a turn must never break on the model.
|
||||||
|
|
||||||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
||||||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
||||||
|
|
||||||
|
Measured on the 77-case RU fixture (`MODEL-BAKEOFF-31-07-2026.md`): the classifier scores
|
||||||
|
36.8% full accuracy at p50 31ms; Qwen3-1.7B scores 67.5% intent-only / 72.7% through the
|
||||||
|
cascade at p50 ≈2.7s. Accuracy roughly doubled, latency is ~90× worse, and that trade was
|
||||||
|
accepted deliberately. Still open: `Confidence: 1.0` is hardcoded in `llmrouter.go`, so the
|
||||||
|
LLM path never asks for clarification (6/6 refusal cases missed) — Vikunja #359.
|
||||||
|
|
||||||
## LLM output contract
|
## LLM output contract
|
||||||
|
|
||||||
All phrasing paths emit `{"response":"...","mood":"..."}` (parsed in `replier_llm.go` and
|
All phrasing paths emit `{"response":"...","mood":"..."}` (parsed in `replier_llm.go` and
|
||||||
@@ -94,7 +105,11 @@ workspace enforces that the Go and relabelling prompts remain identical.
|
|||||||
## Non-goals (hard constraints)
|
## Non-goals (hard constraints)
|
||||||
|
|
||||||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
||||||
self-reference must use feminine forms (the user is male; see memory `maven-persona-gender`).
|
self-reference must use feminine forms — `рада`, not `рад`; `поняла`, not `понял`. The owner
|
||||||
|
is male and is addressed informally: "ты", singular, never "вы"/"ваш" and never "он"/"его"
|
||||||
|
(she talks TO him, not about him). Pet names ("милый", "дорогой") are forbidden; his name
|
||||||
|
("Ками") is not. The eval enforces this: `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||||
|
`internal/phraser/eval/checks.go`, scored by `make eval-phrasing`.
|
||||||
|
|
||||||
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
||||||
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
||||||
|
|||||||
@@ -19,7 +19,10 @@ Settles Vikunja **#278 / #250**.
|
|||||||
- Same fixture and scorer as `ROUTING-EVAL-31-07-2026.md`: `internal/router/eval/`
|
- Same fixture and scorer as `ROUTING-EVAL-31-07-2026.md`: `internal/router/eval/`
|
||||||
(`ru_routing_v1.json`, 76 held-out cases).
|
(`ru_routing_v1.json`, 76 held-out cases).
|
||||||
- Reproduce: `MAVEN_LLM_URL=http://127.0.0.1:<port> make eval-router`
|
- Reproduce: `MAVEN_LLM_URL=http://127.0.0.1:<port> make eval-router`
|
||||||
(`TestLLMRouterBaseline`). Note: there is no `make eval-models` target.
|
(`TestLLMRouterBaseline`). (This line used to say there is no `make eval-models` target.
|
||||||
|
There is one now — start a server with the gguf you want, then
|
||||||
|
`make eval-models MAVEN_LLM_URL=http://127.0.0.1:<port>`. It runs only the LLM test, since
|
||||||
|
the classifier baselines do not depend on the model.)
|
||||||
- All three models served by the same `llama-server` flags — `-c 2048 -ngl 99 -t 6`, only
|
- All three models served by the same `llama-server` flags — `-c 2048 -ngl 99 -t 6`, only
|
||||||
`-m` and `--port` differ. One server at a time on an otherwise idle box, so latencies are
|
`-m` and `--port` differ. One server at a time on an otherwise idle box, so latencies are
|
||||||
real and not contention.
|
real and not contention.
|
||||||
@@ -207,10 +210,14 @@ swapped again when the CPT lands.
|
|||||||
- Routing is one run per model, not three. The gaps between families are far larger
|
- Routing is one run per model, not three. The gaps between families are far larger
|
||||||
than the run-to-run spread seen on the talk fixture, but the 2B-vs-1.7B gap (62.3
|
than the run-to-run spread seen on the talk fixture, but the 2B-vs-1.7B gap (62.3
|
||||||
vs 67.5) is not safe to call on one run.
|
vs 67.5) is not safe to call on one run.
|
||||||
- The routing numbers only reach production once the LLM router is wired on. It is
|
- ~~The routing numbers only reach production once the LLM router is wired on. It is
|
||||||
still `nil`.
|
still `nil`.~~ **Resolved the same evening:** the LLM router is wired at `voice.go:214`
|
||||||
- `/mnt/hdd1/llms/LFM2.5/Qwen3-1.7B-UD-Q4_K_XL.gguf` is a 293 MB truncated download
|
behind `voice.llm_router`, the default is on, and `deploy/mavend.json` sets it `true`.
|
||||||
in the wrong directory. The good 1.13 GB copy is in `qwen3/`. Delete the stray one.
|
These numbers are the production path now, so the p50 ≈2.7s is a real per-turn cost and
|
||||||
|
not a bench artifact.
|
||||||
|
- ~~`/mnt/hdd1/llms/LFM2.5/Qwen3-1.7B-UD-Q4_K_XL.gguf` is a 293 MB truncated download
|
||||||
|
in the wrong directory.~~ **Deleted 2026-07-31.** The good 1.13 GB copy in `qwen3/` is
|
||||||
|
what `deploy/mavend.json` loads.
|
||||||
- Harness: `scratchpad/bakeoff.sh`, one server at a time, health-checked before each
|
- Harness: `scratchpad/bakeoff.sh`, one server at a time, health-checked before each
|
||||||
run, `/v1/models` recorded per run. Never run two LLM consumers at once — see the
|
run, `/v1/models` recorded per run. Never run two LLM consumers at once — see the
|
||||||
contamination note in `TALK-EVAL-31-07-2026.md`.
|
contamination note in `TALK-EVAL-31-07-2026.md`.
|
||||||
|
|||||||
Reference in New Issue
Block a user