You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
GITTENSORY-D (`ai_embed_http_400`) was partially fixed by #5029 (empty/whitespace-only chunks), but is STILL firing (492 occurrences since 2026-06-28, ongoing) — now visible with a real error body thanks to that same PR's response-body capture: "the input length exceeds the context length."
Root cause, traced live: `embedTexts` (`src/review/rag.ts`) sends up to `EMBED_BATCH` (96) chunk texts to the embedding provider in ONE call. If even a single text in that batch exceeds the embedding model's context window (a dense/minified file, a long unbroken line, etc. — chunking bounds by CHARACTER count, not token count, and token-per-char ratio varies a lot by content), the ENTIRE `inference.run()` call throws, and `embedTexts` returns `null` for the WHOLE call — discarding every OTHER, perfectly-fine chunk in that batch (and via `upsertChunks`, potentially chunks from many unrelated files sharing the batch). This silently degrades RAG review-context quality far beyond the one problematic chunk.
Fix
`embedTexts` now returns `(number[] | null)[] | null`: the outer `null` is preserved for "no adapter configured" / "invalid batch size" (unchanged); a `null` at a specific index means only THAT text failed to embed. When a batch call throws or comes back structurally invalid, it retries that batch one item at a time (sequentially, not concurrently, to stay gentle on a self-hosted inference server) so only the genuinely-unembeddable item(s) are lost.
`upsertChunks` filters out chunks whose vector came back `null` before the vector-index/DB write, instead of assuming every input chunk got a vector.
Problem
GITTENSORY-D (`ai_embed_http_400`) was partially fixed by #5029 (empty/whitespace-only chunks), but is STILL firing (492 occurrences since 2026-06-28, ongoing) — now visible with a real error body thanks to that same PR's response-body capture: "the input length exceeds the context length."
Root cause, traced live: `embedTexts` (`src/review/rag.ts`) sends up to `EMBED_BATCH` (96) chunk texts to the embedding provider in ONE call. If even a single text in that batch exceeds the embedding model's context window (a dense/minified file, a long unbroken line, etc. — chunking bounds by CHARACTER count, not token count, and token-per-char ratio varies a lot by content), the ENTIRE `inference.run()` call throws, and `embedTexts` returns `null` for the WHOLE call — discarding every OTHER, perfectly-fine chunk in that batch (and via `upsertChunks`, potentially chunks from many unrelated files sharing the batch). This silently degrades RAG review-context quality far beyond the one problematic chunk.
Fix