Skip to content

Catch up journals from Chronicle's change feed; versions only move forward - #125

Merged
keyxmakerx merged 1 commit into
mainfrom
claude/project-thread-bs6baq
Oct 3, 2026
Merged

keyxmakerx merged 1 commit into
mainfrom
claude/project-thread-bs6baq

Conversation

@keyxmakerx

Copy link
Copy Markdown
Owner

Fixes: none
Security implication: none. One new read (GET /sync/changes, owner and co-DM keys only on the server; a player's key gets 403 and falls back to the old rescan) and one read-back GET /entities/:id after a permissions push. No new writes.
Consumer-verified: Chronicle internal/plugins/syncapi/sync_changes_handler.go ListChanges (response {changes:[{seq,type,resourceId,op}], next, hasMore, resetRequired}), sync_change_repository.go List (holds back rows younger than 2 s), sync_change_bus.go (entity events recorded as type entity); entities/service.go SetEntityPermissions (bumps updated_at, answers {"status":"ok"}).
Foundry compatibility: no new Foundry APIs; uses the settings, document and flag calls the module already uses (v12–v14). Not run in a live world.
Mockup: n/a. Nothing on screen changes.

What this changes

Before:

  • Every time Foundry connected, the module downloaded every Chronicle page in the campaign to find what changed.
  • Foundry's own edits came back on the next connect as a "change". After a permissions push, the journal's recorded version went stale, so the next edit's version check was off as well.
  • Every journal edit re-sent the journal's permissions, even when they hadn't changed.

After:

  • On connect, the module asks Chronicle what changed since last time and fetches only those pages. A reopen with nothing new fetches no pages.
  • Foundry's own pushes are recognised and left alone. A journal's recorded version only moves forward.
  • Permissions are sent only when Foundry's sharing actually changed.

Why

This is step 3 of the approved sync rebuild (keyxmakerx/Chronicle#907, module #114), journals first, on the change feed from keyxmakerx/Chronicle#927. Key Maker's hard requirements were no duplicate copies, idempotent changes, and the two-sided bench before anything is called ready. Refs #114.

Load-bearing lines:

  • scripts/_change-feed.mjs (pure):
    • walkChangeFeed follows hasMore, returns resetRequired, and stops on no progress or after 200 pages (complete: false).
    • collapseChanges gives one outcome per page.
    • cursorFor reads a per-campaign cursor {campaignId, seq, areas, createdAfter}.
    • feedForArea gives an area the delta only if the cursor was saved while that area was syncing.
  • scripts/sync-manager.mjs _performInitialSync / _readChangeFeed:
    • The feed is read before modules apply, and at least 2.5 s after the socket opened. Chronicle holds back its newest 2 s of rows, so edits made just before connecting would otherwise be missed. The bench caught that.
    • The cursor is saved only when every feed area succeeded.
    • With no feed (an older Chronicle, or a player's key), journals rescan as before and no cursor is saved.
  • scripts/journal-sync.mjs _catchUpFromFeed:
    • Refetches only the listed pages and skips any already at the recorded version.
    • Sets a journal aside on deleted or a 404, with the same checks as before (_setAsideIfGone, now shared with the full walk).
    • Creates a journal only for a page whose own created_at is after the cursor's createdAfter.
    • Why that rule: the feed replays entries a previous connect already applied. Without it, a journal the GM deleted came back; the bench caught that too. createdAfter moves only with the cursor, so a failed catch-up still treats a new page as new on retry. The review caught that one.
  • Versions only move forward (_olderThanRecorded):
    • A live copy older than the recorded version is ignored. Equal versions still apply, because versions have one-second precision.
    • _recordPush and the "kept your unsent edit" path never roll the version back.
  • Permissions push (_pushPermissions):
    • Skipped when the body equals the last one sent for that journal (pushedPermissions flag).
    • After a real push, the module reads the page back for the version it stamped. Chronicle returns none, and its broadcast carries the old one.

Honest deviations:

  • If Chronicle-side permissions change while Foundry's ownership doesn't, a later Foundry text edit no longer re-sends Foundry's ownership. Only a real ownership change in Foundry pushes it.
  • The read-back after a permissions push has a sub-second window. A third-party edit landing inside it is recorded without being applied, and is lost to Foundry only if its live message is also dropped. It is fixed properly server-side by Change feed: report a cursor ahead of the head, a lost feed row, and the version a permissions save stamps Chronicle#985, filed with two other feed gaps the review found (a cursor ahead of the head after a restore, and a feed row lost on a failed insert).
  • Actors, notes and maps still rescan on connect. They move to the feed in their own PRs.

Test plan

  • node --test tools/test-*.mjs: 994 of 994 pass. New: test-change-feed.mjs (feed walk, collapse, cursor rules including the failed-catch-up replay and the cut-short walk, journal delta apply) and test-journal-versions.mjs (stale copy ignored, version never rolls back, permissions read-back and skip).
  • node tools/check-package-descriptor.mjs: OK
  • CHRONICLE_DIR=../Chronicle bench/run.sh (Chronicle main plus #957): 12 of 12 scenarios, run 3 times. The new scenario "a reopen reads only what changed" checks five things:
    • the reopen reads the feed and never walks the page list;
    • an untouched page is not fetched;
    • Chronicle's rename, delete and new page arrive;
    • Foundry's own pushed rename is not re-applied;
    • a third open fetches no page at all.
  • Manual verification in Foundry: covered by Live test session on Foundry v14 (after TESTING.md is brought up to date) #94 after release
  • CI passes (Sync bench included)

Tenet self-check

  • T-B1 security: no new trust boundary; the feed carries ids only, and content is still read through the normal visibility-filtered reads
  • T-B2 plugin isolation: module-only
  • T-B3 production UI: no UI change
  • T-B4 docs: .ai.md (change feed, versions), API-CONTRACT.md (GET /sync/changes, permissions save), CLAUDE.md convention

🤖 Generated with Claude Code

https://claude.ai/code/session_018uKa2E5HNdeubwBSD9Jn4n


Generated by Claude Code

…rward

On connect, read GET /sync/changes from a saved per-campaign cursor and
refetch only the pages it lists, instead of walking every page. The cursor
(with its own created-after floor) is saved only after the changes applied,
so a failed catch-up replays. Without a feed (older Chronicle, a player's
key) journals rescan as before.

A journal's recorded Chronicle version never moves backwards: stale copies
are ignored, identical permission pushes are skipped, and a real one reads
the page back for the version it stamped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018uKa2E5HNdeubwBSD9Jn4n
@keyxmakerx
keyxmakerx marked this pull request as ready for review October 3, 2026 05:41
@keyxmakerx
keyxmakerx merged commit 7f3550d into main Oct 3, 2026
2 checks passed
@keyxmakerx
keyxmakerx deleted the claude/project-thread-bs6baq branch October 3, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants