[BUG] Forward references are not back-resolved when their target note is created (only on reindex)

[phernandez]

Summary

In the new event-index architecture (#1002), creating a note that forward-references a not-yet-existing target leaves the relation unresolved, and the relation is not back-resolved when the target note is later created. Resolution only happens on a full project-index pass (bm reindex or startup indexing). This contradicts the write_note success message, which tells users:

Note: Unresolved relations point to entities that don't exist yet. They will be automatically resolved when target entities are created or during sync operations.

"during sync operations" (reindex/startup) is accurate; "when target entities are created" is not for the incremental write path.

Reproduction

Local project, fresh DB. Using write_note:

1. Create note A with - relates_to [[Note B]] while B does not exist → A's relation is recorded unresolved (expected).
2. Create note B.
3. Inspect relations (recent_activity type=relation, or build_context). A's relates_to → B is still unresolved:
   • resolved relations render as A -> B with the full target permalink (.../relates-to/<project>/<b-permalink>)
   • unresolved render as just A with a short slug target (.../relates-to/note-b)
4. Run bm reindex → the relation resolves correctly.

Observed during PR #1002 regression testing: three relations authored before their targets (Accepted -> Search Index Behavior, Accepted -> Regression Harness Session, Search Index Behavior -> Regression Harness Session) all stayed unresolved across multiple later writes, and all resolved after a single bm reindex.

Root cause

The note-write path resolves the new note's own outgoing links against existing targets, but never triggers backward resolution of other notes' dangling links pointing at the newly-created note:

• knowledge_router note write schedules only entity_vector_sync, not a project index (src/basic_memory/api/v2/routers/knowledge_router.py:86).
• schedule_project_index is only invoked by the project-index route command with run_in_background (src/basic_memory/deps/services.py:959) — not on note creation.
• Single-file indexing runs with resolve_relations=False (src/basic_memory/indexing/file_indexer.py:162), so the watcher/inline single-file path doesn't back-resolve either.
• Full relation resolution runs only in the project-index coordinator (bm reindex, startup indexing), which is why reindex fixes it.

So in normal local use (even outside test mode), a forward reference created before its target persists as unresolved until the next reindex or server restart (startup indexing). There is no periodic/triggered sweep that resolves dangling relations as targets appear.

Suggested fix

Add a lightweight background resolver that periodically (or on a debounce after writes) sweeps unresolved relations and resolves any whose targets now exist — i.e. a background/polling task rather than requiring a full bm reindex. This keeps the write_note promise true without paying for a full project-index pass on every write. It should be a no-op in test mode (consistent with the other schedulers in deps/services.py), but enabled for dev/user runtimes.

Alternative / complementary: have the accepted-note write path enqueue a targeted "resolve inbound dangling refs for this entity's title/permalink" follow-up when a note is created, so resolution is event-driven instead of polled.

If neither lands soon, update the write_note message to say resolution happens "during sync/reindex operations" and drop "when target entities are created", to avoid over-promising.

Environment

• basic-memory branch codex/repository-explicit-sessions (v0.22.1), the refactor(core): add shared runtime orchestration #1002 event-index architecture
• SQLite backend, local runtime
• Reproduced through MCP tools and confirmed via recent_activity/build_context

Notes

• Should confirm whether the pre-refactor(core): add shared runtime orchestration #1002 sync architecture back-resolved these via its background relation-resolution task (the old "background relation resolution" behavior), to label this a regression vs. a known gap carried into the new architecture.

[claude]

Claude finished @phernandez's task in 2m 27s —— View job

---

Issue Triage

• Gather context and read issue
• Read relevant source files
• Check for duplicate issues
• Classify and assess issue
• Add appropriate labels
• Post triage summary

---

Analysis

Type: Bug report (code defect)

No duplicate found. Issue #955 (now closed) covered a related but distinct symptom — [[None]] rendering of unresolved refs in build_context. This issue is about the underlying resolution mechanism never firing on note creation.

---

Root Cause Confirmed

Reading the code confirms the report:

• create_entity in knowledge_router.py:506 calls entity_service.create_entity_with_content(), then search_service.index_entity(), then schedules only sync_entity_vectors via _schedule_vector_sync_if_enabled() (line 509). No inbound relation resolution is scheduled or triggered.
• batch_indexer.py:189 — index_markdown_file() accepts a resolve_relations: bool = True flag, and when called from the single-file/watcher path it is invoked with resolve_relations=False (per the issue's root cause section), so dangling inbound refs are never swept.
• write_note.py:378–381 explicitly promises: "They will be automatically resolved when target entities are created or during sync operations." — the first half of that promise is not implemented.
• The TaskScheduler in deps/services.py:460 only has one registered handler: sync_entity_vectors. There is no resolve_inbound_relations task registered.

---

Priority: Medium

• Not data loss or security, and a workaround (bm reindex) exists.
• However, it actively misleads users via the write_note success message, and degrades the core knowledge-graph feature (link traversal) in normal incremental use.
• Affects all users who create notes with forward references — a common pattern when building a knowledge base.

---

Component: mcp + database/sync

The bug spans two layers:

1. MCP layer (write_note.py) — incorrect promise in the success message.
2. Indexing/sync layer (knowledge_router.py, batch_indexer.py, deps/services.py) — no back-resolution triggered on entity creation.

---

Complexity: Medium–Complex

Two viable fixes (not mutually exclusive):

Option A — Immediate/low-risk (documentation fix):
Update write_note.py:381 to say "during sync/reindex operations" and drop "when target entities are created". This is a one-liner that stops over-promising.

Option B — Full fix (event-driven back-resolution):
After create_entity (and update_entity) persists the new entity, schedule a task (e.g. resolve_inbound_relations) that calls relation_repository.find_unresolved_relations_by_target_name(title/permalink) and resolves matching dangling refs against the new entity's id. The LocalTaskScheduler already supports registering new task handlers (deps/services.py:519), so the wiring exists — it just needs:

1. A new task handler resolve_inbound_relations registered alongside sync_entity_vectors.
2. A RelationRepository query for unresolved relations whose to_name matches the new entity's title/permalink.
3. A call to task_scheduler.schedule("resolve_inbound_relations", entity_id=entity.id, ...) from the create/update endpoints.

This keeps the request path fast (fire-and-forget) while making the promise true.

Option C — Polling sweeper (suggested in the issue):
A background periodic task that sweeps all unresolved relations. Higher overhead, slower to resolve, but catches cases not covered by Option B (e.g. renames). Can be layered on top of B.

---

Labels

The repo only has bug, enhancement, documentation, cloud, and v1.0 labels — the fine-grained priority/complexity labels listed in the triage template don't exist yet. The issue already has bug which is correct.

---

Recommended Next Steps

1. Quick win: Fix the write_note success message (src/basic_memory/mcp/tools/write_note.py:381) to accurately reflect that resolution happens "during sync/reindex operations", not on target creation.
2. Full fix: Implement Option B — add a resolve_inbound_relations task handler and schedule it from create_entity/update_entity_by_id in knowledge_router.py.
3. The RelationRepository.find_unresolved_relations (relation_repository.py:72) already exists; a targeted variant filtering by to_name is the only missing query.

[phernandez]

Update: confirmed in env: user (not a test-config artifact), with a precise root cause

Re-tested with the file watcher and background schedulers active (env: user). The behavior splits by ingress path:

Direct on-disk write (external editor / git / Obsidian) — back-resolution WORKS.
Wrote Source with relates_to [[Target ZZZ]] (Target absent) directly to disk → unresolved. Then wrote Target ZZZ directly to disk. The watcher indexed it and ran relation repair, and the dangling ref resolved:

index_markdown_file: Watcher Forward Target ZZZ.md
local_runtime:index_file_completed:147 - Local event-index relation repair completed

→ Source -> Watcher Forward Target ZZZ now resolves (full target permalink).

MCP write_note — back-resolution DOES NOT happen.
Wrote MCP Source with relates_to [[MCP Target QQQ]] (absent) via write_note → unresolved. Then wrote MCP Target QQQ via write_note. The source's relation stays unresolved (.../relates-to/mcp-forward-target-qqq, short slug). The logs for both MCP writes show the inline index + status_code=202 but no relation repair completed line.

Root cause

LocalInlineStorageEventResultRecorder.index_file_completed (src/basic_memory/index/local_runtime.py:135-146) is the only place that runs project-wide relation resolution on a single-file event:

relation_request = plan_index_file_relation_resolution(IndexFileRelationResolutionContext(..., status=result.status))
if relation_request is None:
    return
relation_result = await resolve_project_relations(self.relation_runtime)  # <-- back-resolves dangling inbound refs

• For a direct-disk write, the watcher is the first to index the file → result.status is a real index → relation_request is non-None → resolve_project_relations runs and back-resolves dangling inbound references.
• For an MCP/API write, the accepted-note write path inline-indexes the materialized file first and never calls resolve_project_relations. When the watcher then observes the file, the checksum already matches → the file is unchanged → plan_index_file_relation_resolution returns None → relation repair is skipped.

Net effect: creating a note via write_note that satisfies a dangling [[wikilink]] from an existing note does not resolve that link, but creating the identical note by saving a file in Obsidian does. Same logical operation, different outcome depending on ingress. Resolution otherwise waits for bm reindex / startup indexing.

Fix options (in addition to the background-sweep idea above)

1. Run resolve_project_relations (or a targeted inbound-ref resolution) in the accepted-note write path so MCP/API writes match the watcher path. Targeted resolution keyed on the new entity's title/permalink would avoid a full project pass per write.
2. Or: a periodic/debounced background sweep that resolves dangling relations as targets appear (the original suggestion) — covers both ingress paths uniformly.

The asymmetry is the actionable bug: the two ingress paths should agree.

Environment: branch codex/repository-explicit-sessions (v0.22.1), SQLite, local runtime, env: user (watcher + schedulers active).

[phernandez]

Fixed on main. The MCP/API write path now schedules relation resolution on every accepted note mutation (_schedule_post_write_followups in knowledge_router.py plus LocalRelationResolutionScheduler), so creating a note back-resolves inbound forward references that name it — parity with the watcher's relation repair. Resolution runs as a coalesced, debounced offline pass so it adds no per-write latency, and cloud enqueues a queue job for the same effect.

Regression test: test_create_schedules_relation_resolution_regardless_of_semantic in tests/api/v2/test_knowledge_router.py. Landed via the #1002/#1054 runtime work, building on the June 24–25 resolution fixes. Closing.