Skip to content

[bug] upvoting a never-recalled document demotes it in recall search #1278

Closed

Description

@SaulMoro

What happens

teamai recall feedback --positive <doc> on a document that has never been recalled lowers its search score once the vote is counted. In the run below the score went from 5.9 to 1.9, and the document dropped from RELEVANT to NOT_RELEVANT.

Why

A vote with no recall leaves last_recalled_at empty (src/votes.ts:343, src/votes.ts:345-346 for the upvote). computeConfidence reads an empty timestamp as 180 days old, so recency is 0 (src/maintenance/confidence.ts:31-36).

With one upvote and no recalls, confidence is base 0.3 × 0.4 = 0.12, plus ratio 1.0 × 0.3 = 0.3, total 0.42 (confidence.ts:40). That is below HOT_THRESHOLD 0.5 (src/maintenance/hot-cold.ts:4), so the entry gets COLD_PENALTY 0.3 (hot-cold.ts:23), and search multiplies its score by it (src/utils/search-index.ts:1160-1161).

Two related gaps:

  • The search index builds its factors at src/utils/search-index.ts:400-404 without lastUpvotedAt, so the upvote time is ignored there.
  • computeConfidence accepts lastUpvotedAt (confidence.ts:14) but never reads it.

#1256 caps the ratio at 1.0 and leaves recency alone, so it does not change this case.

Reproduction

Run on the PR #1276 build, not on main. Fixture: a document never recalled, then recall feedback --positive, then a pull. Before the vote the document scored RELEVANT score=5.9. After the vote was counted it scored NOT_RELEVANT score=1.9. All exit codes were 0.

On main the same fixture never counted the vote (the one-pull lag in #1233), so the drop on main is read from the code above, not observed in a run.

Possible fix

Decide what a vote without a recall should mean. Either take recency from the later of last_recalled_at and last_upvoted_at, which the factor interface already allows, or treat a missing timestamp as neutral instead of 180 days old. Pass lastUpvotedAt from the search index as well. The test at src/__tests__/confidence.test.ts:84-92 pins 0.7 for a document with no recalls and five upvotes, and it depends on the zero recency, so it changes with the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions