Merge the example-led prompt finding
This commit is contained in:
@@ -103,6 +103,35 @@ but a large part of the jump is that failure now degrades into Russian instead o
|
||||
The two remaining failures: one `"..."` recurrence (`routine-stretch`) and one meal nudge
|
||||
that never says food.
|
||||
|
||||
## Tried and reverted: an example-led nudge prompt (#393)
|
||||
|
||||
The idea was that a 0.8B copies examples better than it follows rules, so the nudge prompt
|
||||
was rewritten to lead with five on-topic examples (water, break, pills, morning, service) and
|
||||
the prose rules were compressed to pay for the tokens: 1190 chars down to 986.
|
||||
|
||||
It measured **worse**, three runs each side, same llama-server, same fixture:
|
||||
|
||||
| run | before | after |
|
||||
|---|---|---|
|
||||
| 1 | 12/15 (address 14) | 11/15 (address 13) |
|
||||
| 2 | 13/15 (address 15) | 12/15 (address 15) |
|
||||
| 3 | 14/15 (address 15) | 11/15 (address 12) |
|
||||
|
||||
`feminine` and `hisgender` were 15/15 on all six runs, so they measure nothing here. The
|
||||
regression is all in `address`: 44/45 before, 40/45 after. Formal "вы"/"ваше" and plural
|
||||
imperatives came back, and so did `"..."`.
|
||||
|
||||
Two likely causes, both about the same thing — **examples do not carry a prohibition**. The
|
||||
old prompt spent a whole sentence on «говоришь на "ты", в единственном числе»; the new one
|
||||
demoted that to one item in a long "никогда" list, and the model stopped obeying it. And
|
||||
making the examples on-topic let their *wording* leak: a break case came back as
|
||||
«Вы давно не пили воду. Выпей стакан.» — the water example, verbatim, in the wrong slot.
|
||||
That is exactly the failure the laundry/laptop examples were chosen to avoid.
|
||||
|
||||
Change reverted. What survives is the measurement: a rule the model must obey needs its own
|
||||
sentence, and examples must stay off-topic. Also note the before side alone spans 12–14 of
|
||||
15 — this fixture cannot resolve anything smaller than about three cases.
|
||||
|
||||
## Broken, found, not fixed
|
||||
|
||||
1. ~~**`checkFeminine` only catches half the constraint.**~~ **Fixed** (#381). It scanned for
|
||||
|
||||
Reference in New Issue
Block a user