memeval: scope the evaluator's note windows by source
Both windows the evaluator keeps over the notes table were row budgets over every writer. The dedupe read 200 recent notes and kept the eval ones, so after 200 ordinary notes an old observation left the window and the next evaluation wrote the same sentence again. The snapshot asked for MaxItems notes and then discarded her own, so once hourly evaluation had run for a few weeks the model saw almost no real notes. Both reads are now filtered in SQL, by RecentNotesBySource and RecentNotesExcludingSource. Two smaller things in the same area. The dedupe key stripped any trailing bracketed clause, so an observation ending in one hashed differently from its stored form; it now strips only the recorded action. The evaluation timeout was five minutes on the one llama-server that also answers voice turns, which made a collision a five-minute mute assistant, and is now sixty seconds. Found in review of #55.
This commit is contained in:
+22
-1
@@ -80,8 +80,29 @@ func (s *Store) QueryNotes(ctx context.Context, embedding []float32, k int) ([]N
|
||||
// embedding math; Score stays 0). This is the read surface for /dash: notes
|
||||
// captured by voice are otherwise only reachable through semantic query.
|
||||
func (s *Store) RecentNotes(ctx context.Context, n int) ([]Note, error) {
|
||||
return s.recentNotesWhere(ctx, "", n)
|
||||
}
|
||||
|
||||
// RecentNotesBySource returns the newest n notes written by one source, newest
|
||||
// first. The filter is in SQL, not in the caller, because a caller that reads n
|
||||
// rows and then keeps the ones it wants has a window measured in OTHER writers'
|
||||
// traffic: once n newer notes from anywhere have landed, the rows it was
|
||||
// looking for are gone. Anything that needs "the last n of mine" wants this.
|
||||
func (s *Store) RecentNotesBySource(ctx context.Context, source string, n int) ([]Note, error) {
|
||||
return s.recentNotesWhere(ctx, "WHERE source = ?", n, source)
|
||||
}
|
||||
|
||||
// RecentNotesExcludingSource returns the newest n notes NOT written by source.
|
||||
// Same reasoning inverted: a reader that wants n notes he wrote must not have
|
||||
// its budget eaten by rows it is about to discard.
|
||||
func (s *Store) RecentNotesExcludingSource(ctx context.Context, source string, n int) ([]Note, error) {
|
||||
return s.recentNotesWhere(ctx, "WHERE source <> ?", n, source)
|
||||
}
|
||||
|
||||
func (s *Store) recentNotesWhere(ctx context.Context, where string, n int, args ...any) ([]Note, error) {
|
||||
args = append(args, n)
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, ts, text, source FROM notes ORDER BY ts DESC LIMIT ?`, n)
|
||||
`SELECT id, ts, text, source FROM notes `+where+` ORDER BY ts DESC LIMIT ?`, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("recent notes: %w", err)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user