Skip to content

refactor(core): extract semantic chunk planning - #1128

Merged
phernandez merged 1 commit into
mainfrom
codex/1108-next-core-split
Jul 21, 2026
Merged

refactor(core): extract semantic chunk planning#1128
phernandez merged 1 commit into
mainfrom
codex/1108-next-core-split

Conversation

@phernandez

@phernandez phernandez commented Jul 21, 2026

Copy link
Copy Markdown
Member

Why

What Changed

  • Added repository/semantic_chunking.py as the typed, side-effect-free home for semantic chunk planning.
  • Kept thin compatibility delegates on SearchRepositoryBase while removing the planning implementation from the repository class.
  • Preserved project-aware duplicate chunk logging in the repository orchestration layer.
  • Updated the Postgres vector scheduling hook to use the shared VectorChunkRecord type.
  • Moved pure chunk-planning tests to the new module boundary and expanded edge-case and fingerprint coverage.
  • No user-visible semantic search behavior changes are intended.

Implementation Details

  • SemanticSourceRow defines the narrow search-row input contract.
  • VectorChunkRecord names the deterministic writer input, and VectorChunkBuildResult returns records with duplicate-key diagnostics.
  • Source composition, chunk splitting, record hashing, and entity fingerprinting are ordinary module functions with no repository or database dependencies.
  • SearchRepositoryBase still owns backend coordination, embedding calls, state inspection, logging, and persistence hooks.

Testing

Automated

  • just fast-check: passed (Ruff formatting/lint and ty type checking).
  • BASIC_MEMORY_ENV=test uv run pytest tests/repository/test_semantic_chunking.py tests/repository/test_semantic_search_base.py tests/repository/test_sqlite_vector_search_repository.py tests/repository/test_postgres_search_repository.py test-int/semantic/test_search_diagnostics.py --no-cov -q: 51 passed, 23 skipped after rebasing onto current main.
  • just fast-test: 4,594 passed, 41 skipped.
  • BASIC_MEMORY_ENV=test uv run pytest tests/repository/test_semantic_chunking.py tests/repository/test_semantic_search_base.py --cov=basic_memory.repository.semantic_chunking --cov-report=term-missing:skip-covered --cov-fail-under=0 -q: 33 passed; semantic_chunking.py reached 141/141 statements (100%).
  • git diff --check: passed.

Manual

  • just doctor: passed the local configuration, database, file/index/search, and FastEmbed vector-sync health checks.

Risks / Follow-ups

Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez
phernandez force-pushed the codex/1108-next-core-split branch from 30ad561 to 8c238ee Compare July 21, 2026 03:39
@phernandez
phernandez merged commit 656cac8 into main Jul 21, 2026
23 checks passed
@phernandez
phernandez deleted the codex/1108-next-core-split branch July 21, 2026 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant