feat(router): array route contract + shorter RU router prompt

Route contract is now a JSON array of action objects (one per ask) so
compound utterances route all their intents, not just the first. Grammar
root emits `[{intent...},...]`; parseActions tolerates a bare object.
Cascade still returns one Decision — full N-action dispatch lands with the
engine turn-on (marked in-code).

Router prompt rewritten shorter + decision-ordered (prompt-guy feedback),
fact redefined as "implicit update" not "trackable state", kept in Russian
to match the CPT base + phraser. "интент" → "намерение".

CLAUDE.md: routing-architecture section + refreshed open items.
docs/plans: route-data generation plan.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017GMrVfuYN3nE4L1vEiFYC9
This commit is contained in:
kami
2026-07-11 23:27:44 +04:00
parent 6a5121657a
commit 0c65387a5f
3 changed files with 129 additions and 18 deletions
+30 -2
View File
@@ -56,6 +56,30 @@ preserves backward compat with plain text and the old `{"body","summary"}` forma
Every training sample's assistant turn must produce `{"response":"...","mood":"..."}`.
## Routing architecture: LLM-as-router (REARCH.md, target)
Target arch = **LLM-as-router** (`REARCH.md`, supersedes the classifier-first
model). One resident model — the **CPT'd Qwen3-1.7B** (RU-CPT run; replaces LFM,
"too meh") — fills **both** router and phraser roles, two call-sites / two contracts:
| Prompt | Contract | Source of truth |
|---|---|---|
| route-prompt | `{"intent":<enum>, key?, value?, text?, verb?}` GBNF-constrained | `internal/router/llmrouter.go` (`routeSystem`+`routeGrammar`) |
| phrase-prompt | `{"response","mood"}` | above section |
7 intents: `fact, reminder, note, query, act, chat, system`. Key rule:
«запомни/запиши» = note, «напомни/не забудь» = reminder. Embedder is **demoted**
from router to a tool (RAG hint), not a threshold gate.
**Status: phase 1 NOT done.** Plumbing exists (`LLMRouter`, `Route` cascade calls
it at `router.go:87`) but `voice.go:209` wires it `nil` — engine OFF, classifier
stopgap still active. Flip `nil``NewLLMRouter(qwen)` after CPT finishes. Do NOT
read the current committed code as the intended design — it's the interim stopgap.
Route-training data: `esp32-whisper-fine-tune/llm/gen_route_data.py` relabels real
utterances through the verbatim `routeSystem` into `{intent,...}`. Keep its
`ROUTE_SYSTEM` in sync with the Go const.
## Data defects measured in current corpus
- **Mood collapse:** `neutral 1051, thinking 666, happy 280, confused 206, tired 7`
@@ -115,7 +139,11 @@ python convert_hf_to_gguf.py ./Vikhr-merged --outfile vikhr-maven-f16.gguf --out
## Open items
- [x] **Decide output contract****B: `{response,mood}`**, Go side patched (2026-07-11).
- [ ] Confirm `Vikhrmodels/Vikhr-Qwen-2.5-1.5B-Instruct` repo id on HF.
- [ ] Write `gen_data.py` distiller (topic → canonical prompt → validate → balanced JSONL w/ mood quotas).
- [x] **Base model** — Qwen3-1.7B, RU via **continued pretraining** (not Vikhr). CPT run in progress.
- [x] **Write `gen_data.py` distiller** — ran, produced `persona_train.jsonl` (2045) + eval (107), mood collapse fixed.
- [x] **Write `gen_route_data.py`** — route-schema relabeler (run when router up).
- [ ] **Turn router engine ON** — swap `voice.go:209` `nil``NewLLMRouter(qwen)` after CPT (REARCH phase 1).
- [ ] Run `gen_route_data.py`; train route-LoRA (or fold into persona SFT).
- [ ] Improve `routeSystem` prompt for sub-1B disambiguation (awaiting prompt-guy input).
- [ ] Write Cyrillic-validity + JSON eval (extend `llama-eval-test.py`).
- [ ] Normalize `user-*.jsonl` into `{"messages":[...]}`.
+42
View File
@@ -0,0 +1,42 @@
# Route-training data plan (LLM-as-router)
Goal: training data that teaches the CPT'd Qwen3-1.7B to emit the **route
contract** — `{"intent":<enum>, key?, value?, text?, verb?}`, GBNF-constrained —
matching `internal/router/llmrouter.go` (`routeSystem` + `routeGrammar`) verbatim.
Train=deploy parity: label with the EXACT prompt the daemon sends.
7 intents: `fact, reminder, note, query, act, chat, system`. Hard cases (grammar
can't enforce): note vs reminder («запомни» vs «напомни»), fact vs note (trackable
state vs static memo).
## Steps
1. **Source utterances** — real RU turns, not synthetic. Main: `function_calling.jsonl`
(473); plus `user-*.jsonl` fragments for chat/fact/system coverage. `gen_route_data.py`
dedups across all.
2. **Relabel, don't convert** — old taxonomy (time/weather/timer) ≠ 7 intents. Feed
each utterance through `routeSystem` to a strong router model → take its `{intent,...}`.
3. **Validate** — intent ∈ enum, keys ⊆ {intent,key,value,text,verb}. Drop invalid.
4. **Balance check** — after a run, count intents. `act`/`system`/`fact` likely thin
(function_calling skews query/act). Author extra examples for the holes; re-run.
5. **Better prompt first** — improve `routeSystem` for sub-1B disambiguation before a
big generation run (awaiting prompt-guy input). Re-labeling is cheap; regenerate.
6. **Train** — route-LoRA on top of CPT base, OR fold into the persona SFT as a second
contract (decide once volume known). Eval = route accuracy on a held-out REAL set.
## Blocking
- Router at `inference.kvmx.ru` / `localhost:6446` must be up (currently down).
- CPT must finish before the route-LoRA trains on it.
## Files
- `esp32-whisper-fine-tune/llm/gen_route_data.py` — the relabeler (done, self-checks).
`ROUTE_SYSTEM` const = verbatim copy of Go `routeSystem`; **keep in sync**.
- Output: `llm/data/route_train.jsonl` (resumable append).
## Prompt-guy question (sent 2026-07-11)
How to structure the router system prompt for a sub-1B model doing 7-intent
classification + slot extraction, GBNF-constrained — example ordering/count,
contrastive near-miss pairs (note vs reminder) vs more singles, rule placement.
+57 -16
View File
@@ -21,10 +21,12 @@ type LLMRouter struct{ c Completer }
func NewLLMRouter(c Completer) *LLMRouter { return &LLMRouter{c: c} }
// routeGrammar — GBNF constraining the model to a fixed-shape JSON object with
// an intent enum. Prevents free-form drift from a sub-1B model.
// routeGrammar — GBNF constraining the model to a JSON ARRAY of fixed-shape
// action objects (one per ask; compound utterances → multiple). Enum + key set
// prevent free-form drift from a sub-1B model.
const routeGrammar = `
root ::= "{" ws "\"intent\"" ws ":" ws intent ("," ws field)* ws "}"
root ::= "[" ws action ("," ws action)* ws "]"
action ::= "{" ws "\"intent\"" ws ":" ws intent ("," ws field)* ws "}"
intent ::= "\"fact\"" | "\"reminder\"" | "\"note\"" | "\"query\"" | "\"act\"" | "\"chat\"" | "\"system\""
field ::= key ws ":" ws string
key ::= "\"key\"" | "\"value\"" | "\"text\"" | "\"verb\""
@@ -32,19 +34,36 @@ string ::= "\"" ([^"\\] | "\\" .)* "\""
ws ::= [ \t\n]*
`
const routeSystem = `Ты — маршрутизатор Maven. По реплике верни ОДИН JSON-объект: {"intent": ...}.
const routeSystem = `Классифицируй ровно одно сообщение пользователя. Верни ОДИН JSON-объект.
Интенты:
- fact — состояние/показатель, который надо запомнить и ОТСЛЕЖИВАТЬ во времени. "я выпил воду" → {"intent":"fact","key":"water","value":"выпил"}.
- reminder — просьба напомнить о чём-то В БУДУЩЕМ, есть время или срок ("напомни", "не забудь", "через час", "завтра", "в 9:00"). text = что напомнить. "напомни завтра в 9 позвонить маме" → {"intent":"reminder","text":"позвонить маме"}.
- note — заметка «на память» БЕЗ отслеживания и БЕЗ будущего времени ("запомни", "запиши", "заметь"). text = суть. "запомни что кофе закончился" → {"intent":"note","text":"кофе закончился"}.
- query — вопрос, требующий ответа. text = вопрос.
- act — команда выполнить действие на сервере. verb = глагол.
- chat — свободный разговор. text.
- system — вопрос о текущем времени/дате/дне недели.
Ровно одно намерение: fact, reminder, note, query, act, chat, system.
ВАЖНО: «запомни/запиши» = note (просто сохранить), «напомни/не забудь» = reminder (напомнить позже). Нет будущего времени и это не «напомни» → note, НЕ reminder.
Только JSON, без пояснений.`
Классифицируй по цели пользователя. Порядок решения:
1. Хочет напоминание в будущем → reminder
2. Явно просит сохранить информацию → note
3. Сообщает или обновляет текущее состояние/событие → fact
4. Хочет получить информацию → query
5. Просит выполнить работу → act
6. Про ассистента, настройки или память → system
7. Иначе → chat
Различия:
- note — сохранить информацию, без напоминания. text = суть.
- reminder — уведомить позже. text = что напомнить.
- fact — неявное обновление: пользователь сообщает, что что-то в мире изменилось (текущее/изменённое состояние, случившееся событие). key/value.
Примеры:
"запиши пароль" → {"intent":"note","text":"пароль"}
"напомни купить молоко" → {"intent":"reminder","text":"купить молоко"}
"запиши купить молоко" → {"intent":"note","text":"купить молоко"}
"я выпил воду" → {"intent":"fact","key":"water","value":"выпил"}
"мой любимый фильм — Интерстеллар" → {"intent":"note","text":"любимый фильм — Интерстеллар"}
"что такое docker?" → {"intent":"query","text":"что такое docker"}
"напиши письмо" → {"intent":"act","verb":"написать письмо"}
"очисти память" → {"intent":"system"}
"привет" → {"intent":"chat","text":"привет"}
Ответ — JSON-массив: по одному объекту на каждую просьбу. Обычно один. Если в реплике несколько просьб — по объекту на каждую. "напомни купить молоко, и запиши что кофе кончился" → [{"intent":"reminder","text":"купить молоко"},{"intent":"note","text":"кофе кончился"}]. Только JSON, без пояснений.`
type routeAction struct {
Intent string `json:"intent"`
@@ -59,10 +78,18 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
if err != nil {
return Decision{}, false, err
}
var a routeAction
if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &a); err != nil {
acts, err := parseActions(strings.TrimSpace(raw))
if err != nil {
return Decision{}, false, fmt.Errorf("llmrouter: parse %q: %w", raw, err)
}
if len(acts) == 0 {
return Decision{}, false, fmt.Errorf("llmrouter: empty action list %q", raw)
}
// ponytail: contract is an array (compound utterances → N actions), but the
// Router cascade still returns one Decision. Dispatch of all actions lands
// with the engine turn-on (Router.Route → []Decision, both voice.go handlers
// loop). Until then only the first ask is honored.
a := acts[0]
d := Decision{Utterance: utterance, Stage: 1, Confidence: 1.0}
switch Intent(a.Intent) {
case IntentFact:
@@ -90,6 +117,20 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
return d, true, nil
}
// parseActions accepts the array contract (`[{...},...]`) or a bare object
// (`{...}`) for robustness against a model that drops the wrapper.
func parseActions(raw string) ([]routeAction, error) {
if strings.HasPrefix(raw, "[") {
var as []routeAction
return as, json.Unmarshal([]byte(raw), &as)
}
var a routeAction
if err := json.Unmarshal([]byte(raw), &a); err != nil {
return nil, err
}
return []routeAction{a}, nil
}
func firstNonEmpty(a, b string) string {
if strings.TrimSpace(a) != "" {
return a