claude
2b3e34c7e8
The plan calls the labeled set the whole project and names the stage 0 grammars as the label functions. cmd/labelgen runs them, the real ones in buildRouter order, so a rule change moves the training data with it. Gemma labels the rest at 334ms/call with nothing unparsed, which matches the plan's estimate. It agrees with the seed files on 197/277, and reading the disagreements is the finding: the seeds and the router prompt hold different definitions of system, of a world question and of a bare verb. V-626.
3.3 KiB
Gemma as a label function, and what it found in the seeds
06-08-2026. V-546. Measured on workpc against gemma-4-12b-it-qat-UD-Q4_K_XL.
docs/plans/18-routing-heads-on-e5-small.md puts the labeled set at 20k examples through
gemma, costing 2 to 4 hours of the card. This is the check before spending that. Gemma
labels the 344 hand-written classifier seeds. Agreement with the label a person already
chose is a precision number rather than a guess.
What ran
cmd/labelgen runs the stage 0 grammars. The real ones, in buildRouter order, minus
wakeword-act, whose allowlist is a deployment's enabled tool names. It labels 62 of 339
seed lines and leaves the rest.
The remaining 277 went to gemma through the daemon's own routeSystem prompt and
routeGrammar, both extracted from internal/router/llmrouter.go at run time rather than
retyped. Temperature 0.
Cost
334ms per call, 0 unparsed of 277. The GBNF held every time. At that rate the plan's 20k examples is under two hours of card, which matches its estimate.
The stage 0 rules as label functions
Agreement between the grammar's label and the seed file the line came from:
| seed intent | agree |
|---|---|
| reminder | 37/37 |
| query | 9/10 |
| system | 7/8 |
| act | 2/2 |
| chat | 0/4 |
| note | 0/1 |
ReminderGrammar at 37/37 is the evidence the plan wanted. The chat column is a defect
rather than a disagreement: chatNarrativeTopics is Russian-only, so tell me about yourself survives the decline and routes IntentQuery with topic yourself. Filed as
V-625, which also records that как дела у сервера appears verbatim in two seed files
under two intents.
Gemma against the seeds
197/277, 71.1%. By intent:
| seed intent | agree |
|---|---|
| note | 33/33 |
| act | 57/64 |
| fact | 37/40 |
| query | 51/54 |
| chat | 15/35 |
| system | 4/43 |
| reminder | 0/8 |
The number is not gemma's error rate. Reading the 80 disagreements, most are the seed files and the prompt holding different definitions of the same intent. Three boundaries carry 42 of them, and V-626 is the fix:
- system, 26 lines. The prompt restricts system to the clock, the calendar date and the assistant itself. The seeds also put sensor and host state there. That is the V-374 edit of 31-07-2026, which the seeds never received.
- world questions, 8 lines.
почему небо голубое,why is the sky blue. Written when chat was the only honest destination for a question nothing could answer, and external search now answers them. - bare verbs, 8 lines.
поставь напоминаниеwith nothing to remind about. The prompt calls that unknown. This one is not staleness. A nearest-neighbour centroid wants the bare verb phrase, and that is what a seed file is for.
Four intents have not been redefined since the seeds were written: note, fact, query and act. They agree at 178 of 191.
What this says about the plan
Gemma is usable as a label function on those four and not on system, chat or a bare verb. The plan already budgets a day of the owner reading the set. This says where to spend it.
It also says the two engines in the cascade are being taught different rules on 80 lines. A routing measurement that swaps between the classifier and the router is measuring some of that disagreement rather than the models.