feat(router): array route contract + shorter RU router prompt
Route contract is now a JSON array of action objects (one per ask) so
compound utterances route all their intents, not just the first. Grammar
root emits `[{intent...},...]`; parseActions tolerates a bare object.
Cascade still returns one Decision — full N-action dispatch lands with the
engine turn-on (marked in-code).
Router prompt rewritten shorter + decision-ordered (prompt-guy feedback),
fact redefined as "implicit update" not "trackable state", kept in Russian
to match the CPT base + phraser. "интент" → "намерение".
CLAUDE.md: routing-architecture section + refreshed open items.
docs/plans: route-data generation plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017GMrVfuYN3nE4L1vEiFYC9
This commit is contained in:
@@ -56,6 +56,30 @@ preserves backward compat with plain text and the old `{"body","summary"}` forma
|
||||
|
||||
Every training sample's assistant turn must produce `{"response":"...","mood":"..."}`.
|
||||
|
||||
## Routing architecture: LLM-as-router (REARCH.md, target)
|
||||
|
||||
Target arch = **LLM-as-router** (`REARCH.md`, supersedes the classifier-first
|
||||
model). One resident model — the **CPT'd Qwen3-1.7B** (RU-CPT run; replaces LFM,
|
||||
"too meh") — fills **both** router and phraser roles, two call-sites / two contracts:
|
||||
|
||||
| Prompt | Contract | Source of truth |
|
||||
|---|---|---|
|
||||
| route-prompt | `{"intent":<enum>, key?, value?, text?, verb?}` GBNF-constrained | `internal/router/llmrouter.go` (`routeSystem`+`routeGrammar`) |
|
||||
| phrase-prompt | `{"response","mood"}` | above section |
|
||||
|
||||
7 intents: `fact, reminder, note, query, act, chat, system`. Key rule:
|
||||
«запомни/запиши» = note, «напомни/не забудь» = reminder. Embedder is **demoted**
|
||||
from router to a tool (RAG hint), not a threshold gate.
|
||||
|
||||
**Status: phase 1 NOT done.** Plumbing exists (`LLMRouter`, `Route` cascade calls
|
||||
it at `router.go:87`) but `voice.go:209` wires it `nil` — engine OFF, classifier
|
||||
stopgap still active. Flip `nil` → `NewLLMRouter(qwen)` after CPT finishes. Do NOT
|
||||
read the current committed code as the intended design — it's the interim stopgap.
|
||||
|
||||
Route-training data: `esp32-whisper-fine-tune/llm/gen_route_data.py` relabels real
|
||||
utterances through the verbatim `routeSystem` into `{intent,...}`. Keep its
|
||||
`ROUTE_SYSTEM` in sync with the Go const.
|
||||
|
||||
## Data defects measured in current corpus
|
||||
|
||||
- **Mood collapse:** `neutral 1051, thinking 666, happy 280, confused 206, tired 7`
|
||||
@@ -115,7 +139,11 @@ python convert_hf_to_gguf.py ./Vikhr-merged --outfile vikhr-maven-f16.gguf --out
|
||||
## Open items
|
||||
|
||||
- [x] **Decide output contract** — **B: `{response,mood}`**, Go side patched (2026-07-11).
|
||||
- [ ] Confirm `Vikhrmodels/Vikhr-Qwen-2.5-1.5B-Instruct` repo id on HF.
|
||||
- [ ] Write `gen_data.py` distiller (topic → canonical prompt → validate → balanced JSONL w/ mood quotas).
|
||||
- [x] **Base model** — Qwen3-1.7B, RU via **continued pretraining** (not Vikhr). CPT run in progress.
|
||||
- [x] **Write `gen_data.py` distiller** — ran, produced `persona_train.jsonl` (2045) + eval (107), mood collapse fixed.
|
||||
- [x] **Write `gen_route_data.py`** — route-schema relabeler (run when router up).
|
||||
- [ ] **Turn router engine ON** — swap `voice.go:209` `nil` → `NewLLMRouter(qwen)` after CPT (REARCH phase 1).
|
||||
- [ ] Run `gen_route_data.py`; train route-LoRA (or fold into persona SFT).
|
||||
- [ ] Improve `routeSystem` prompt for sub-1B disambiguation (awaiting prompt-guy input).
|
||||
- [ ] Write Cyrillic-validity + JSON eval (extend `llama-eval-test.py`).
|
||||
- [ ] Normalize `user-*.jsonl` into `{"messages":[...]}`.
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# Route-training data plan (LLM-as-router)
|
||||
|
||||
Goal: training data that teaches the CPT'd Qwen3-1.7B to emit the **route
|
||||
contract** — `{"intent":<enum>, key?, value?, text?, verb?}`, GBNF-constrained —
|
||||
matching `internal/router/llmrouter.go` (`routeSystem` + `routeGrammar`) verbatim.
|
||||
Train=deploy parity: label with the EXACT prompt the daemon sends.
|
||||
|
||||
7 intents: `fact, reminder, note, query, act, chat, system`. Hard cases (grammar
|
||||
can't enforce): note vs reminder («запомни» vs «напомни»), fact vs note (trackable
|
||||
state vs static memo).
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Source utterances** — real RU turns, not synthetic. Main: `function_calling.jsonl`
|
||||
(473); plus `user-*.jsonl` fragments for chat/fact/system coverage. `gen_route_data.py`
|
||||
dedups across all.
|
||||
2. **Relabel, don't convert** — old taxonomy (time/weather/timer) ≠ 7 intents. Feed
|
||||
each utterance through `routeSystem` to a strong router model → take its `{intent,...}`.
|
||||
3. **Validate** — intent ∈ enum, keys ⊆ {intent,key,value,text,verb}. Drop invalid.
|
||||
4. **Balance check** — after a run, count intents. `act`/`system`/`fact` likely thin
|
||||
(function_calling skews query/act). Author extra examples for the holes; re-run.
|
||||
5. **Better prompt first** — improve `routeSystem` for sub-1B disambiguation before a
|
||||
big generation run (awaiting prompt-guy input). Re-labeling is cheap; regenerate.
|
||||
6. **Train** — route-LoRA on top of CPT base, OR fold into the persona SFT as a second
|
||||
contract (decide once volume known). Eval = route accuracy on a held-out REAL set.
|
||||
|
||||
## Blocking
|
||||
|
||||
- Router at `inference.kvmx.ru` / `localhost:6446` must be up (currently down).
|
||||
- CPT must finish before the route-LoRA trains on it.
|
||||
|
||||
## Files
|
||||
|
||||
- `esp32-whisper-fine-tune/llm/gen_route_data.py` — the relabeler (done, self-checks).
|
||||
`ROUTE_SYSTEM` const = verbatim copy of Go `routeSystem`; **keep in sync**.
|
||||
- Output: `llm/data/route_train.jsonl` (resumable append).
|
||||
|
||||
## Prompt-guy question (sent 2026-07-11)
|
||||
|
||||
How to structure the router system prompt for a sub-1B model doing 7-intent
|
||||
classification + slot extraction, GBNF-constrained — example ordering/count,
|
||||
contrastive near-miss pairs (note vs reminder) vs more singles, rule placement.
|
||||
@@ -21,10 +21,12 @@ type LLMRouter struct{ c Completer }
|
||||
|
||||
func NewLLMRouter(c Completer) *LLMRouter { return &LLMRouter{c: c} }
|
||||
|
||||
// routeGrammar — GBNF constraining the model to a fixed-shape JSON object with
|
||||
// an intent enum. Prevents free-form drift from a sub-1B model.
|
||||
// routeGrammar — GBNF constraining the model to a JSON ARRAY of fixed-shape
|
||||
// action objects (one per ask; compound utterances → multiple). Enum + key set
|
||||
// prevent free-form drift from a sub-1B model.
|
||||
const routeGrammar = `
|
||||
root ::= "{" ws "\"intent\"" ws ":" ws intent ("," ws field)* ws "}"
|
||||
root ::= "[" ws action ("," ws action)* ws "]"
|
||||
action ::= "{" ws "\"intent\"" ws ":" ws intent ("," ws field)* ws "}"
|
||||
intent ::= "\"fact\"" | "\"reminder\"" | "\"note\"" | "\"query\"" | "\"act\"" | "\"chat\"" | "\"system\""
|
||||
field ::= key ws ":" ws string
|
||||
key ::= "\"key\"" | "\"value\"" | "\"text\"" | "\"verb\""
|
||||
@@ -32,19 +34,36 @@ string ::= "\"" ([^"\\] | "\\" .)* "\""
|
||||
ws ::= [ \t\n]*
|
||||
`
|
||||
|
||||
const routeSystem = `Ты — маршрутизатор Maven. По реплике верни ОДИН JSON-объект: {"intent": ...}.
|
||||
const routeSystem = `Классифицируй ровно одно сообщение пользователя. Верни ОДИН JSON-объект.
|
||||
|
||||
Интенты:
|
||||
- fact — состояние/показатель, который надо запомнить и ОТСЛЕЖИВАТЬ во времени. "я выпил воду" → {"intent":"fact","key":"water","value":"выпил"}.
|
||||
- reminder — просьба напомнить о чём-то В БУДУЩЕМ, есть время или срок ("напомни", "не забудь", "через час", "завтра", "в 9:00"). text = что напомнить. "напомни завтра в 9 позвонить маме" → {"intent":"reminder","text":"позвонить маме"}.
|
||||
- note — заметка «на память» БЕЗ отслеживания и БЕЗ будущего времени ("запомни", "запиши", "заметь"). text = суть. "запомни что кофе закончился" → {"intent":"note","text":"кофе закончился"}.
|
||||
- query — вопрос, требующий ответа. text = вопрос.
|
||||
- act — команда выполнить действие на сервере. verb = глагол.
|
||||
- chat — свободный разговор. text.
|
||||
- system — вопрос о текущем времени/дате/дне недели.
|
||||
Ровно одно намерение: fact, reminder, note, query, act, chat, system.
|
||||
|
||||
ВАЖНО: «запомни/запиши» = note (просто сохранить), «напомни/не забудь» = reminder (напомнить позже). Нет будущего времени и это не «напомни» → note, НЕ reminder.
|
||||
Только JSON, без пояснений.`
|
||||
Классифицируй по цели пользователя. Порядок решения:
|
||||
1. Хочет напоминание в будущем → reminder
|
||||
2. Явно просит сохранить информацию → note
|
||||
3. Сообщает или обновляет текущее состояние/событие → fact
|
||||
4. Хочет получить информацию → query
|
||||
5. Просит выполнить работу → act
|
||||
6. Про ассистента, настройки или память → system
|
||||
7. Иначе → chat
|
||||
|
||||
Различия:
|
||||
- note — сохранить информацию, без напоминания. text = суть.
|
||||
- reminder — уведомить позже. text = что напомнить.
|
||||
- fact — неявное обновление: пользователь сообщает, что что-то в мире изменилось (текущее/изменённое состояние, случившееся событие). key/value.
|
||||
|
||||
Примеры:
|
||||
"запиши пароль" → {"intent":"note","text":"пароль"}
|
||||
"напомни купить молоко" → {"intent":"reminder","text":"купить молоко"}
|
||||
"запиши купить молоко" → {"intent":"note","text":"купить молоко"}
|
||||
"я выпил воду" → {"intent":"fact","key":"water","value":"выпил"}
|
||||
"мой любимый фильм — Интерстеллар" → {"intent":"note","text":"любимый фильм — Интерстеллар"}
|
||||
"что такое docker?" → {"intent":"query","text":"что такое docker"}
|
||||
"напиши письмо" → {"intent":"act","verb":"написать письмо"}
|
||||
"очисти память" → {"intent":"system"}
|
||||
"привет" → {"intent":"chat","text":"привет"}
|
||||
|
||||
Ответ — JSON-массив: по одному объекту на каждую просьбу. Обычно один. Если в реплике несколько просьб — по объекту на каждую. "напомни купить молоко, и запиши что кофе кончился" → [{"intent":"reminder","text":"купить молоко"},{"intent":"note","text":"кофе кончился"}]. Только JSON, без пояснений.`
|
||||
|
||||
type routeAction struct {
|
||||
Intent string `json:"intent"`
|
||||
@@ -59,10 +78,18 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
|
||||
if err != nil {
|
||||
return Decision{}, false, err
|
||||
}
|
||||
var a routeAction
|
||||
if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &a); err != nil {
|
||||
acts, err := parseActions(strings.TrimSpace(raw))
|
||||
if err != nil {
|
||||
return Decision{}, false, fmt.Errorf("llmrouter: parse %q: %w", raw, err)
|
||||
}
|
||||
if len(acts) == 0 {
|
||||
return Decision{}, false, fmt.Errorf("llmrouter: empty action list %q", raw)
|
||||
}
|
||||
// ponytail: contract is an array (compound utterances → N actions), but the
|
||||
// Router cascade still returns one Decision. Dispatch of all actions lands
|
||||
// with the engine turn-on (Router.Route → []Decision, both voice.go handlers
|
||||
// loop). Until then only the first ask is honored.
|
||||
a := acts[0]
|
||||
d := Decision{Utterance: utterance, Stage: 1, Confidence: 1.0}
|
||||
switch Intent(a.Intent) {
|
||||
case IntentFact:
|
||||
@@ -90,6 +117,20 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
|
||||
return d, true, nil
|
||||
}
|
||||
|
||||
// parseActions accepts the array contract (`[{...},...]`) or a bare object
|
||||
// (`{...}`) for robustness against a model that drops the wrapper.
|
||||
func parseActions(raw string) ([]routeAction, error) {
|
||||
if strings.HasPrefix(raw, "[") {
|
||||
var as []routeAction
|
||||
return as, json.Unmarshal([]byte(raw), &as)
|
||||
}
|
||||
var a routeAction
|
||||
if err := json.Unmarshal([]byte(raw), &a); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return []routeAction{a}, nil
|
||||
}
|
||||
|
||||
func firstNonEmpty(a, b string) string {
|
||||
if strings.TrimSpace(a) != "" {
|
||||
return a
|
||||
|
||||
Reference in New Issue
Block a user