fix(core): defer superseded vector generations

[phernandez]

Why

An older embedding job can prepare physical vector-manifest row IDs, wait for embedding generation, and resume after a newer entity generation has replaced those rows. Core treated every missing prepared row as corruption, so stale work could exhaust worker retries even though the newer job already owned convergence.

The persistence boundary needs to distinguish a superseded logical chunk generation from a manifest row that is unexpectedly missing for the current generation. Broadly swallowing the error would hide real corruption.

Closes #1200.

What Changed

• Carry entity ID, chunk key, chunk text, and source hash through each pending embedding job instead of reducing it to a physical row ID and text tuple.
• When a prepared row is missing, rebuild the entity's current logical chunks from authoritative search-index rows and classify that exact generation as current or superseded.
• Skip superseded rows, return their classification from persistence, and report the old entity job as deferred rather than synchronized.
• Preserve the same generation-ownership contract through the PostgreSQL batched upsert and both built-in and external vector adapters.
• Keep missing current-generation rows and changed physical-row identity as visible correctness errors.

Implementation Details

PendingEmbeddingJob now contains the logical generation identity needed after the embedding await. _persist_embeddings validates that identity against the manifest when the physical row still exists. If the row disappeared, it reconstructs current chunks only for affected entities and compares (entity_id, chunk_key, source_hash).

EmbeddingPersistenceResult makes every row outcome explicit: persisted or superseded. The batch orchestrator fails if persistence does not classify every flushed row, and any superseded work keeps that entity out of the synchronized set. This preserves the newer generation's ownership without adding retries, a tenant-wide lock, or exception swallowing.

The existing external-adapter manifest lock still spans adapter I/O and ready publication. Built-in adapters retain their generation-safe write behavior.

Testing

• Pre-fix PostgreSQL regression: both pgvector and external-adapter cases failed because job A reported Vector manifest rows disappeared before write and was counted failed.
• BASIC_MEMORY_TEST_POSTGRES=1 LOGFIRE_IGNORE_NO_CONFIG=1 uv run pytest -p pytest_mock --no-cov -q tests/repository/test_vector_manifest_generation_ownership.py --tb=short — 4 passed.
• Default SQLite semantic/vector regression surface — 126 passed.
• PostgreSQL/pgvector semantic/vector regression surface — 138 passed, 2 skipped.
• just fast-check — Ruff format/lint and ty type checking passed.
• git diff --cached --check — passed before commit.

Risks / Follow-ups

• Reconstructing current chunks adds work only on the exceptional missing-row path; the normal persistence path is unchanged.
• A missing generation that is still current remains fatal by design, so manifest corruption is not masked.
• No Cloud-side policy or dependency workaround is included; Core owns the shared generation contract.