Catch formal and third-person address in the phrasing eval (#384) #32

2026-07-31T12:27:45+02:00

claude commented

2026-07-31 12:27:45 +02:00

Two commits. Vikunja #384 — two persona breaks the phrasing run scored clean.

She said "Приходите… Жду вас" (formal plural, as if to a stranger) and "Он не ел 11 дней" (talks about you, to somebody else). The hisgender check from PR 24 only looks at feminine past-tense forms, so both passed.

The new address check catches three things, and prints the word it tripped on every time:

the вы family as whole words — no exclusion list needed, none of those are ever another word
plural imperatives by ending: -ите -ете -йте -ьте ("приходите", "не забудьте")
он/его/ему when nobody else was named earlier in the message

Same honesty as hisgender: it is a keyword/suffix heuristic, not a parser, and the comment writes out where it gives up. One deliberate exclusion — a -ите word right after a preposition is skipped, because prepositional-case nouns share the ending ("в интернете"). Nothing else is excluded on purpose: a longer list starts swallowing real imperatives, which is how an earlier check in this file blinded itself.

The second commit is that same trap, caught once more. The check asks "was anyone else named before «он»?" — but time words weren't stoplisted, so "сегодня он не ел" passed. That is the recorded break with one word in front of it, and nudges open with those words constantly. Now сегодня/вчера/завтра/утром/вечером/опять/… don't count as the person being talked about.

What it still misses, stated plainly: any other noun before the pronoun blinds it the same way ("после обеда он не ел"). That's the real hole. Widening it further would start flagging messages that are legitimately about a third party, so it stops there. его is also a possessive, so "его задача" at the start of a message is a false alarm — dismissible, and that's the direction to err in.

Tests pin both recorded strings, since an eval run is sampled and may not reproduce them. Plus the negatives that must not trip: correct informal address, and "сервис упал, он не отвечает" — legitimately about a third party.

Not done here: this only measures. The phraser prompt in internal/phraser/llmphraser.go never actually tells her to address you informally in the second person singular — adding that is the fix for the behaviour, and it's a separate change.

Two commits. Vikunja #384 — two persona breaks the phrasing run scored **clean**. She said "Приходите… Жду вас" (formal plural, as if to a stranger) and "Он не ел 11 дней" (talks *about* you, to somebody else). The `hisgender` check from PR 24 only looks at feminine past-tense forms, so both passed. **The new `address` check catches three things**, and prints the word it tripped on every time: - the `вы` family as whole words — no exclusion list needed, none of those are ever another word - plural imperatives by ending: `-ите -ете -йте -ьте` ("приходите", "не забудьте") - `он/его/ему` when nobody else was named earlier in the message **Same honesty as `hisgender`: it is a keyword/suffix heuristic, not a parser**, and the comment writes out where it gives up. One deliberate exclusion — a `-ите` word right after a preposition is skipped, because prepositional-case nouns share the ending ("в интернете"). Nothing else is excluded on purpose: a longer list starts swallowing real imperatives, which is how an earlier check in this file blinded itself. **The second commit is that same trap, caught once more.** The check asks "was anyone else named before «он»?" — but time words weren't stoplisted, so **"сегодня он не ел" passed**. That is the recorded break with one word in front of it, and nudges open with those words constantly. Now `сегодня/вчера/завтра/утром/вечером/опять/…` don't count as the person being talked about. **What it still misses, stated plainly:** any *other* noun before the pronoun blinds it the same way ("после обеда он не ел"). That's the real hole. Widening it further would start flagging messages that are legitimately about a third party, so it stops there. `его` is also a possessive, so "его задача" at the start of a message is a false alarm — dismissible, and that's the direction to err in. **Tests** pin both recorded strings, since an eval run is sampled and may not reproduce them. Plus the negatives that must not trip: correct informal address, and "сервис упал, он не отвечает" — legitimately about a third party. **Not done here:** this only *measures*. The phraser prompt in `internal/phraser/llmphraser.go` never actually tells her to address you informally in the second person singular — adding that is the fix for the behaviour, and it's a separate change.

claude added 3 commits 2026-07-31 12:27:46 +02:00

Add an eval check for formal and third-person address (#384 ) 62d47d28ac

The phrasing run produced two persona breaks that scored clean:
"Приходите… Жду вас" (formal plural) and "Он не ел 11 дней" (talks
about him instead of to him). She is feminine, he is male, and she
speaks to him informally, one to one.

The new `address` check flags the "вы" family, plural imperative
endings, and a third-person "он" with no other subject named earlier in
the message. Like `hisgender` it is a keyword/suffix heuristic, not a
parser, and it prints the word it tripped on so a false alarm is easy to
dismiss. Limits are written out in the comment.

Both recorded strings are pinned as unit tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ

Don't let a time word blind the third-person check 9949b309b1

The check asks whether anyone else was named before "он". Time words
were not stoplisted, so "сегодня он не ел" read "сегодня" as the person
being talked about and passed — which is the recorded break with a word
in front of it, and nudges open with those words constantly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ

Merge the address-form eval check 2ad7635501

kami commented

2026-07-31 20:22:24 +02:00

Superseded by #47, which landed this whole stack on master as one reviewed integration merge. This PR head is an ancestor of master — its commits are in, nothing here is lost. Closing as merged-by-proxy rather than merged, since the merge came in through #47.

Review threads on this PR were answered or acted on before the merge; the Russian wording fixes went in as #48.

Superseded by #47, which landed this whole stack on master as one reviewed integration merge. This PR head is an ancestor of master — its commits are in, nothing here is lost. Closing as merged-by-proxy rather than merged, since the merge came in through #47. Review threads on this PR were answered or acted on before the merge; the Russian wording fixes went in as #48.

kami closed this pull request

2026-07-31 20:22:25 +02:00

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kami/Maven#32