fix(core): chunk select_by_ids to stay under SQLite bound-parameter limit

[phernandez]

Summary

reindex --embeddings crashed with sqlite3.OperationalError: too many SQL variables at 0% on large projects, before any embedding work started.

Root cause

Repository.select_by_ids() built one WHERE primary_key IN (...) query for the entire id list. reindex_vectors() passes all entity ids at once, while older SQLite builds cap bound parameters at 999.

Fix

Chunk select_by_ids() into slices of 500 ids, leaving headroom for the project_id bind, execute one query per slice, and concatenate the results. Return type and empty-list behavior are unchanged.

Fixing the shared repository helper protects all bulk-id callers, including indexing/project_index_maintenance.py and indexing/note_content_batch_reconciliation.py.

Test plan

• A deterministic regression test lowers SELECT_BY_IDS_CHUNK_SIZE to 2, queries five rows, and asserts the real repository path issues exactly three statements and returns every row. This proves both full chunks and the tail without depending on the SQLite build parameter limit.
• uv run pytest -q tests/repository/test_repository.py — 14 passed.
• Ruff check and format — clean.
• just fast-check — passed.

Fixes #1045

🤖 Generated with Claude Code

[phernandez]

The production chunking change looks right, but the regression test does not currently prove chunking: modern SQLite builds allow far more than 1,100 bound parameters, so the pre-fix implementation can pass this test unchanged. It also inserts 1,100 rows into every backend/matrix job.

Please make the test deterministic by lowering the chunk size in the test and asserting the number of session.execute() calls (or otherwise observing the emitted SELECTs), while still asserting the complete result set. That should fail against the old one-query implementation and run much faster. I can fold this into the branch before merge.

[phernandez]

@codex review

recheck

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 28cdbfba5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".