# Rework manga character identity: tracklet spine + gemma resolver

Replace siglip-cosine as the primary identity signal. Priority: high.

## Problem
Identity quality plateaus because **siglip2 cosine is the wrong instrument**: it's a
semantic encoder ("man, office, manga panel"), not an instance re-id model. Different
people in the same setting score high; the same person across scenes scores low. No
threshold / gate / gallery fixes what the vector *means* — they only shave ~10% off the
error. Symptoms on the test chapter:
- male-brown colleague labeled as female-black **Choi Haeseon**
- speech bubble mis-attributed (MC credited, colleague actually speaking)
- "MC becomes Choi Haeseon" at the end (reconcile over-merge)

## Already done (guards against *new* contamination, not a fix for existing data)
- Gender gate in `worker_identity.match()` candidate filter + `_pending_match`
  (opposite decided genders never match).
- Reconcile gender gate in `service.run_stage_reconcile` (skip pair if decided genders differ).
- `db.create_character` name-dedup: skip fold if decided genders conflict
  (vision-hallucinated name on a wrong-gender crop).
- **NOTE:** existing roster is already polluted; guards only stop NEW contamination.
  Needs `/stage/clear identity reconcile scene script` + re-run to benefit.

## Target architecture — two-tier (from multi-object tracking)
**Tier 1 — tracklets (local, greedy, high-precision).**
Link per-panel detections into per-person tracklets across a 5–10 panel window. Local
association is the reliable regime (same page, adjacent shot, stable appearance, often the
same speaker). Don't touch global identity yet. A tracklet = ordered crops for one person →
carries an embedding gallery + merged attributes + associated speaker/dialogue.

**Tier 2 — resolve tracklet → global character (deliberate, max evidence).**
Once a tracklet is stable, resolve identity ONCE per person per scene, using a whole gallery
of views + gemma's opinion — not greedily per-crop. Multi-view (front/side/angry/crying)
falls out for free (a tracklet accumulates poses as it spans panels).

## Signal fusion — keep ORDINAL, not a 9-weight learned sum
No labeled data to tune weights → a hand-weighted 9-signal cost is a worse treadmill.
Make it ordinal:
- **HARD GATE:** gender, species → block impossible matches.
- **DECIDER:** gemma "same person?" vs a text character-sheet (reuse reconcile `/same`).
- **STRONG FEATURE:** hair color+style, name / honorific / alias from dialogue
  (Korean honorifics = gold anchors).
- **TIEBREAK:** embedding gallery max-sim (K~5), dialogue-speaker continuity, scene co-presence.

Only build a learned/weighted scorer AFTER labeling a couple chapters.

## Attributes to add (cherry-picked)
- Yes / cheap / high-value: honorifics, aliases, hair color/style, gender, species.
- Skip for now (noisy, marginal): age, body-shape, eye color — add only when a specific
  mislabel needs them.

## Reuses existing infra (rewiring, not greenfield)
- `/direct/window` = windowed multi-image gemma calls in reading order → tracklet window.
- reconcile `/same` = pairwise "same person?" gemma call → tier-2 adjudicator.
- appearance attrs, gender, dialogue+speaker already in the scene graph.

## Suggested sequencing
1. **Tier-2 first** (~1 day, most of the win): gemma resolver vs text character-sheet,
   gender-gated, gallery max-sim as tiebreak. Swaps cosine-decider for gemma-decider.
2. **Tier-1 tracklet spine:** link detections → tracklets, carry gallery + merged attrs,
   resolve once per tracklet. Robust across poses; contains errors to a tracklet not a crop.
3. Add dialogue-continuity / scene-co-presence tiebreaks.
4. (Later, if labeled data exists) learned association cost.

## Open question
Confirm "**gemma decides, tracklets give it evidence**" is the intended core
(vs the fusion score being the core).