Tell the phraser to address him informally (#391) #35

2026-07-31T12:55:00+02:00

claude commented

2026-07-31 12:55:00 +02:00

Two commits, tiny. Vikunja #391 — PR 32 added a check for formal/third-person address, but nothing in the prompts ever asked for the right thing. This asks.

The prompts stated the feminine self-reference rule and stopped. They never said whom she is speaking to, so the model filled the gap itself and produced "Жду вас" and "Он не ел 11 дней". Two prompts now carry the address rule, right next to the feminine one — nudgeSystem in internal/phraser/llmphraser.go and replySystem in cmd/mavend/replier_llm.go. One line each. They belong side by side because they are the same rule seen from two ends: she is feminine, he is male, and she talks to him as ты. Split them across files and one of them drifts.

Kept short on purpose — this goes out on every turn to a 0.8B on laptop CPU, so tokens are latency, and a lecture makes a small model comply worse.

Prompts deliberately left alone: chatSystemPrompt, querySystemPrompt and router/knowledge.go are English and state no gender rule at all (persona gender is injected via config there), so adding an address rule alone would be the same split in reverse. Clarify questions are templated, not phrased by the model. internal/morning builds no prompt of its own — it goes through nudgeSystem and inherits the fix.

Numbers, and the honest read matters more than the numbers. Live llama-server, Qwen3.5-0.8B, two runs per side:

	before	after
`address`	13/15, 15/15	15/15, 15/15
all checks passing	73%, 60%	87%, 93%

I would not claim this is proven. The before side already hit 15/15 once without the change, so on a 15-case fixture a 13→15 move is inside sampling noise. What is real: the before-side failures were the exact reported shapes ("У вас 3 часа без воды", a plural imperative) and neither appeared after. The overall totals moved more, but the other failures were length/ontopic/a degenerate "..." — unrelated, so I am not crediting this line for them. The wording was written once and measured, not iterated against the fixture.

The second commit fixes the check, not the prompt — and it is the more useful half. The eval found a message that should have failed and didn't: "попробуй встать и отдохнуть — у него есть перерыв". The third-person half asks "was anyone else named before «он»?", and verbs were counted as possible people. A nudge is mostly verbs, so this was blind to a whole class of the exact break. Infinitives and imperatives now join past tense as words that cannot be a person.

What still gets past it: a plain noun before the pronoun — "выпей воды, он не пил" — because "воды" looks like somebody who was named. That needs a parser, not a suffix rule, and the comment says so instead of pretending otherwise. It's in the tests as a documented miss rather than quietly absent.

One thing found and not fixed: the 0.8B still sometimes emits a bare "..." despite the prompt forbidding it. Pre-existing, unrelated to address, and it showed up on both sides of the run.

Two commits, tiny. Vikunja #391 — PR 32 added a check for formal/third-person address, but nothing in the prompts ever asked for the right thing. This asks. **The prompts stated the feminine self-reference rule and stopped.** They never said whom she is speaking to, so the model filled the gap itself and produced "Жду вас" and "Он не ел 11 дней". Two prompts now carry the address rule, right next to the feminine one — `nudgeSystem` in `internal/phraser/llmphraser.go` and `replySystem` in `cmd/mavend/replier_llm.go`. One line each. They belong side by side because they are the same rule seen from two ends: **she is feminine, he is male, and she talks to him as ты.** Split them across files and one of them drifts. Kept short on purpose — this goes out on every turn to a 0.8B on laptop CPU, so tokens are latency, and a lecture makes a small model comply *worse*. **Prompts deliberately left alone:** `chatSystemPrompt`, `querySystemPrompt` and `router/knowledge.go` are English and state no gender rule at all (persona gender is injected via config there), so adding an address rule alone would be the same split in reverse. Clarify questions are templated, not phrased by the model. `internal/morning` builds no prompt of its own — it goes through `nudgeSystem` and inherits the fix. **Numbers, and the honest read matters more than the numbers.** Live llama-server, Qwen3.5-0.8B, two runs per side: | | before | after | |---|---|---| | `address` | 13/15, 15/15 | 15/15, 15/15 | | all checks passing | 73%, 60% | 87%, 93% | **I would not claim this is proven.** The before side already hit 15/15 once *without* the change, so on a 15-case fixture a 13→15 move is inside sampling noise. What is real: the before-side failures were the exact reported shapes ("У вас 3 часа без воды", a plural imperative) and neither appeared after. The overall totals moved more, but the other failures were length/ontopic/a degenerate `"..."` — unrelated, so I am not crediting this line for them. The wording was written once and measured, not iterated against the fixture. **The second commit fixes the check, not the prompt** — and it is the more useful half. The eval found a message that should have failed and didn't: *"попробуй встать и отдохнуть — у него есть перерыв"*. The third-person half asks "was anyone else named before «он»?", and **verbs were counted as possible people**. A nudge is mostly verbs, so this was blind to a whole class of the exact break. Infinitives and imperatives now join past tense as words that cannot be a person. **What still gets past it:** a plain noun before the pronoun — "выпей воды, он не пил" — because "воды" looks like somebody who was named. That needs a parser, not a suffix rule, and the comment says so instead of pretending otherwise. It's in the tests as a documented miss rather than quietly absent. **One thing found and not fixed:** the 0.8B still sometimes emits a bare `"..."` despite the prompt forbidding it. Pre-existing, unrelated to address, and it showed up on both sides of the run.

claude added 3 commits 2026-07-31 12:55:00 +02:00

Tell the phraser to speak to him informally, singular eef5d4da4f

The prompts stated the feminine self-reference rule but never said whom she is
speaking to, so the model produced formal plural ("Жду вас") and talked about
him in third person ("Он не ел 11 дней"). Adds the address rule right next to
the feminine one, in the nudge prompt and the confirmation prompt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ

Don't let a verb count as the person being talked about f4de2fc5e1

The third-person check asks whether anyone else was named before "он".
A nudge is mostly verbs, and they were counted as possible people, so
"попробуй встать и отдохнуть — у него есть перерыв" passed. Infinitives
and imperatives now join past tense as words that cannot be a person.

A plain noun before the pronoun still blinds it. That needs a parser,
and the comment says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ

Merge the informal address prompt rule a97f554802

kami added 2 commits 2026-07-31 13:41:04 +02:00

Record the example-led nudge prompt experiment (#393 ) — it made things worse 89d83c0b11

Tried rewriting the nudge prompt to lead with five on-topic examples instead
of rules. Three eval runs each side: before 12/13/14 of 15, after 11/12/11.
The loss is all in the address check — formal "вы" and plural imperatives came
back once the "говоришь на ты" rule stopped being its own sentence, and the
on-topic examples leaked their wording into the wrong cases.

Prompt reverted. Only the finding is committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ

Merge the example-led prompt finding ccc5cba2a3

kami commented

2026-07-31 20:22:29 +02:00

Superseded by #47, which landed this whole stack on master as one reviewed integration merge. This PR head is an ancestor of master — its commits are in, nothing here is lost. Closing as merged-by-proxy rather than merged, since the merge came in through #47.

Review threads on this PR were answered or acted on before the merge; the Russian wording fixes went in as #48.

Superseded by #47, which landed this whole stack on master as one reviewed integration merge. This PR head is an ancestor of master — its commits are in, nothing here is lost. Closing as merged-by-proxy rather than merged, since the merge came in through #47. Review threads on this PR were answered or acted on before the merge; the Russian wording fixes went in as #48.

kami closed this pull request

2026-07-31 20:22:30 +02:00

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kami/Maven#35