Skip to content

embedTexts: one oversized/malformed chunk fails the WHOLE embedding batch, not just that chunk #5072

Description

@JSONbored

Problem

GITTENSORY-D (`ai_embed_http_400`) was partially fixed by #5029 (empty/whitespace-only chunks), but is STILL firing (492 occurrences since 2026-06-28, ongoing) — now visible with a real error body thanks to that same PR's response-body capture: "the input length exceeds the context length."

Root cause, traced live: `embedTexts` (`src/review/rag.ts`) sends up to `EMBED_BATCH` (96) chunk texts to the embedding provider in ONE call. If even a single text in that batch exceeds the embedding model's context window (a dense/minified file, a long unbroken line, etc. — chunking bounds by CHARACTER count, not token count, and token-per-char ratio varies a lot by content), the ENTIRE `inference.run()` call throws, and `embedTexts` returns `null` for the WHOLE call — discarding every OTHER, perfectly-fine chunk in that batch (and via `upsertChunks`, potentially chunks from many unrelated files sharing the batch). This silently degrades RAG review-context quality far beyond the one problematic chunk.

Fix

  • `embedTexts` now returns `(number[] | null)[] | null`: the outer `null` is preserved for "no adapter configured" / "invalid batch size" (unchanged); a `null` at a specific index means only THAT text failed to embed. When a batch call throws or comes back structurally invalid, it retries that batch one item at a time (sequentially, not concurrently, to stay gentle on a self-hosted inference server) so only the genuinely-unembeddable item(s) are lost.
  • `upsertChunks` filters out chunks whose vector came back `null` before the vector-index/DB write, instead of assuming every input chunk got a vector.
  • Escalation stays low-noise: one ERROR-level `rag_embed_batch_degraded` summary per degraded batch (not one per skipped item), mirroring the per-attempt-warn/exhausted-error pattern already used elsewhere in the AI-review pipeline (fix(selfhost): AI-provider per-attempt logging amplifies one retry loop into up to 6 Sentry errors #5046).

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions