[garrytan]

Large-pages Wave B, spec §13 PR 2. Builds on PR 1 (#6787, v0.60.252.0). Every path that reads, rewrites, exports or derives from a page body now works on a page stored in segments. pages.segmented_storage takes effect only when no recent writer predates segment readers. The default stays off in this PR. The flip is a separate, held commit (037247627 on capy/large-pages-2s), to land after the switch-on blockers under Known gaps are fixed. GBRA-60's trust-accept fix is on master (#6807), and the both-engine test for an accepted held take or timeline removal on a segmented page is in this PR.

Stamped v0.60.290.0 on master v0.60.289.0 (15b1595): VERSION, CHANGELOG, every version stamp, SEGMENTED_WRITER_FLOOR = 0.60.290.0, and since: '0.60.290.0' on the 32 BEHAVIOR_CHANGES rows this PR adds. Migrations stay v252–v256 (master's latest is v251). Master's published v248 payload is restored unchanged; an earlier lane had edited its ANALYZE column list.

What's in it

• Reads: get_page none/auto/outline/window/markdown, fetch, entity, evidence and the CLI render all read segments through readBodyRange with the projection overlay (withdrawals, fences, credentials). full refuses with a fix pointing at body:'auto'. Every read that follows an authorization is pinned to the authorized generation; a visibility flip or trust demotion mid-read refuses. A manifest that doesn't tile the body refuses page_body_rows_unreadable for every byte-producing reader (export, backup, restore, digests, materialize, rewrites, projection), so a short read can't pass as complete. Remote callers see projected sizes only (get_page, get_versions).
• Privacy: RLS on the six segment tables, re-asserted on every Postgres schema replay, plus a source-scoped policy recipe in docs/ENGINES.md tested verbatim by a two-source NOBYPASSRLS e2e. With the switch off and no segmented pages, search, get_page, put_page, remember, sync, embedding effects and the unchanged-file check need no segment-table grant. Five maintenance paths do (purge-deleted, forget/purge_fact discovery, the sweep's facts pass, doctor's credential check, the segment storage sweep); a BEHAVIOR row and the docs name them.
• Forget/purge: purge works at any size. purge_fact rewrites segments and manifests across every generation, in keyset batches. Forget is scoped to the segment and occurrence. Withdrawal discovery joins split facts fences. Purge, the 72 h expiry and GC never delete a text an in-flight create holds (page_body_staging_texts, v255); purge_fact aborts a staging that holds the claim.
• Delete/restore: lossless up to pages.max_bytes on file-backed sources. The drift check streams per-region digests. Restore streams the file from segments. The pre-push secret scan reads large files in bounded windows, with claims owned by the window holding the match anchor. The recovery record stages the prior file by path and sha256 instead of embedding it, and the before copy is written to .copying and renamed after fsync.
• Export/backup: export streams segmented pages byte-identical to inline (BOM-safe decode), counted by exportStreamWork. PGLite backup restores segmented pages.
• Derived passes (full-pass mode): links, timeline, takes, facts reconcile, conversation facts (lossless tail window above 24 MiB), tags, evidence, dedup, think/compile leads, integrity, reconcile-links, reindex --aliases, the sweep (budgeted; a page that can't fit is deferred, excluded in SQL) and stale extraction (list == count). Block reads walk the manifest once; forced cuts at line boundaries start new blocks.
• Projection: a create up to 4 MiB seals inside its transaction and is searchable on return. A larger create reports searchable: "pending" with a retry interval and fix; one build runs under a 60 s queue claim plus master's in-process single-flight. projections drain rebuilds segmented pages. Retirement is lock-and-recheck per batch and runs on a skip scan over the v256 index.
• Chunking: on segments, a heading leads the next chunk, so an append after a full last chunk re-embeds one chunk.
• Writer gate: v254 stamps segment-reader marks on requests, publications, consumer heartbeats and ingest_log rows. Any unmarked, below-floor or unparsable writer in the last 7 days keeps the switch off; an unreadable gate counts as blocked. Doctor segmented_storage reports it.
• Semantic writes (decision feat: GBrain v0.1.0 — Postgres-native personal knowledge brain #1 d): add_tag/remove_tag flip metadata on a segmented page at any size: no segment, manifest or chunk write, and the hash resumes from the last manifest row's saved state. Timeline, takes, facts, sources reconcile and core add/diff read the page whole up to 24 MiB and write it as a normal put, matching the inline result. Above 24 MiB they refuse, naming the limit.
• Recovery budget (decision docs: expand brain schema with database architecture and OSS smoothing #4): with the switch on, worktree_recovery_bytes defaults to 2 × (pages.max_bytes + 1 MiB), never below 256 MiB (514 MiB on Postgres). A restore reserves the file twice, so a page at the ceiling now restores. An explicit value is kept.
• Doctor (decision GBrain v0.4.0 — production agent documentation + reference architecture #10): seven checks that read whole bodies (oversized_pages, scraper_junk_pages, markdown_body_completeness, link_resolution_opportunity, fence_integrity, core_memory, extract_health) and gbrain repair frontmatter report skipped_segmented: N / "skipped (segmented): N". The field is absent when N is 0, so the goldens are unchanged.
• Restore (decision Feature request: Org-mode (.org) ingestion and sync support #11): a restored segmented page is re-projected by the existing queue trigger and is searchable at commit or after one drain; a regression test pins this.
• Master interactions: purge_fact saves a preimage version for a segmented page (pointing at the purged generation, as the belief ledger's page-at-time expects); the restore-time purge replay and page-scope replay delete segment texts the way purge does. The core list and frontmatter re-import read segmented pages up to 24 MiB; above it the core list skips the page (doctor counts it) and the re-import is needs_review.
• Ceilings: 256 MiB (Postgres) and 128 MiB (PGLite) when the switch is on; 24 MiB otherwise.

Migrations

Master's v251 (take_horizons) comes first; this PR's five take the next free numbers, v252–v256, in their original order.

#	What	Lock
v252	GC index on segment texts by generation (retired_generation is kept; master's readers select it)	CONCURRENTLY on Postgres
v253	page_view_digests	new table
v254	segment-reader marks on persistence_requests, persistence_consumers, ingest_log	metadata only
v255	page_body_staging_texts; re-adds content_chunks.retired_generation if a test brain dropped it	new table; ADD COLUMN IF NOT EXISTS
v256	content_chunks_occurrence_generation_idx (page_id, occurrence_id, body_generation)	CONCURRENTLY on Postgres (0.34 s at a 100 MB page, 0.8 s at 1M chunks); PGLite blocks writes while it builds

Measured

100 MiB bench, final tree (scripts/bench-large-pages.ts --size 100MB, one seeded conversation page, MemoryMax=3G):

	PGLite	Postgres
Ingest	50.5 s	32.3 s
Peak RSS (over a 1 MiB import)	781 MB (+175 MB)	379 MB (+145 MB)
Rows	374 segments, 29,044 chunks	same
10 KB append	4.4 s, 1 segment, 10 chunks	3.6 s, 1 segment, 10 chunks
Small puts during ingest	p50 13 ms, p95 623 ms	p50 12 ms, p95 33 ms
Flip retiring 40k rows	1.2 s (1.9 s embedded)	0.6 s (1.2 s embedded)

Retirement plan (v256): half-rewrite selection at 26 MB went from 2.18M to 1.8k buffers on Postgres. The crash-recovery probe at 100 MB went from 421k to 3.8k.

N14 findings (gbrain-evals#165): the 2 MB chat append went from 2 chunks re-embedded to 1 (ratio 4.22 → 1.30); the other fixtures are unchanged. Time to searchable at 20 MB: the duplicate build is gone, and a page up to 4 MiB is searchable at commit.

All cost and linearity assertions use work counters, EXPLAIN rows/buffers, byte spies or retained heap after GC. None uses CPU time or wall clock.

Validation

• Full Ubicloud gate green on head 520df8a (v0.60.290.0, contains master 15b1595 with the 10-shard unit matrix): unit 3642, serial 429, slow 54, e2e 618, pg-arms 313, gitleaks, verify, 0 failed. pg-arms shards (8, 1,200 s budget each): 761/688/822/780/720/910/818/876 s.
• Stress flake fixed: recovery-budget-ceiling.slow read the recovery counters right after the purge's reply, but the reply is sent at commit and the recovery record is cleared after that (clearResolvedRecovery). The test and its 27/90 MiB sibling now wait for the source's recovery records to clear, then assert the counters are exactly 0. Forced probe: a 3 s stall at publication:after_commit fails the old test on PGLite and Postgres and passes the fixed one on both.
• Postgres arm weights re-mined from the 32 measured 8-shard arm logs of d406e03 and 717a152; every listed arm file is weighted. Projected file time per shard: Bun 1.4.0 at most 893 s (74% of the budget), Bun 1.4.3 at most 864 s.
• Unit weights: 79 of this PR's unit-lane files had no entry in master's re-mined test-weights.json and are added from the PR's green unit run 39026719688; the 10-shard matrix projects 808–903 s per shard. Stress estimates: 20 unweighted stressed files got measured weights, so e2e/large-pages-export (about 330–390 s per iteration) splits across four stress shards.
• Doctor goldens label the segmented writer floor <segmented-writer-floor>, so they don't flip between <version> and a literal when VERSION moves past the floor.
• GBRA-117's conditions after v0.60.278.0 #6803 wave: drainable projection queue, visible held embedding budgets, bounded doctor/status, extraction nudge (#6847) #6869: segmented rows count as pending in the text class (never failed), never get the media flag, are never sealed by sealImageProjections, and are drained by the resident and projections drain outside the 500-row window (forced probes on both engines in test/large-pages/gbra117-segmented-classes-window.test.ts).
• Memory-test flake: pass 1 of a streamed import now collects every 4 MiB (scanUncollected ≤ 5 MiB probe); the 32 MiB delta fell from ~310 to ~230 MiB with the 450 bound unchanged.
• The v0.60.253.0 W1 trust prerequisites: contest guard for lower-tier take/timeline removals; derived-tier down-propagation #6792/v0.60.254.0 fix(trust): accept keeps deactivated/re-kinded takes; accept works on segmented pages #6807 trust accept is covered by a both-engine test: a held take removal and a held timeline removal on a segmented page are accepted through gbrain trust confirm's path and the text stays out of segments, chunks and search.
• Master's Postgres doctor hot-checks golden pinned the soak-counter day to its capture date; it is now stored as <sync-day> like the fence trend day (forced probe in test/doctor-hot-checks-sync-day.test.ts).
• Two independent review rounds on every fix wave: gstack /review (Opus) and an outside voice (Grok 4.7; GPT-6 Astra refused under its cyber policy and Gemini rejected the request). Every finding was fixed with a forced probe that fails before and passes after on both engines, or shown not to reproduce.
• The final gstack review's one high finding (a second write sealed a page over the first write's unretired rows, keeping removed text searchable) is fixed with a forced probe on both engines, along with its two mediums (remote get_versions on demoted pages, managed stale-link extraction holding every timeline entry).
• A read-only audit of every user-reachable page_body_segmented refusal (about 900 probes on both engines) drove the 2r refusal fixes.

Contract golden changes

• Export-surface goldens (test/fixtures/goldens/exports/{runtime,types}.json) gain keywordBoundedRead, the one engine member added so search/dedup.ts segmentRegions runs its manifest probe and region read under the keyword knob's statement_timeout.

Known gaps (not in this PR)

• applySegmentedBody (segmented-apply.ts:184) runs hasStaleChunks inside the publication transaction, so a large segmented write on a loaded host can be cancelled at the 5 s statement bound and re-prepared; for the flip PR.

• A transient publication failure (57014 and the other transient codes) re-runs the whole prepare with no limit and outside max_preparation_attempts; for the flip PR.

• Switch-on blocker: the canonical file mirror (mirrorPage) skips segmented pages, so after purge_fact a segmented page's file on disk keeps the purged claim. It must be fixed before the flip.

• Switch-on blocker: the projection claim's renew in buildClaimedProjection refreshes the queue row without checking that this builder still owns the claim, so a build that outlived its 60 s claim keeps renewing and staging chunks next to the builder that took the claim over. It must be fixed before the flip.

• A restored archive holding a running request whose staging holds a purged claim makes the replay refuse with purge_blocked_pending_recovery, which fails the restore.

• Dev brains that ran this branch before either renumbering treat a master migration at a reused number as applied: before the first (old v249) master's v249 belief ledger, before the second (old v251, this PR's GC index) master's v251 take_horizons. Recreate them. Released brains are unaffected.

• revert_version on a segmented page refuses until PR 3 (Garry accepted; BEHAVIOR row added).

• edit_page and capture onto an existing slug over 24 MiB refuse (fence scan at admission, no-op check).

• Pages over 24 MiB stay keyword-only (oversized embed skip); imports of person/company/fenced pages over 24 MiB are held for the write gate.

• A brain that ran with the switch on and later turned it off falls back to the 256 MiB recovery budget, so its Postgres pages over about 128 MiB can be deleted but not restored until the switch is back on.

• gbrain repair stuck-recovery lists a large-file recovery as refused; the resident writer finishes it.

• Older binaries fail closed (unexpected_staging_bytes) on a recovery record that carries staging.before.

• The gate can't see an older binary that writes without managed persistence and leaves no request or ingest row; pages_body_storage_chk still fails old whole-body writes closed.

• Dropping retired_generation for real waits for a later migration, once the floor excludes master's binary.

• The sweep defers a page too large for any budget to gbrain extract --stale; the deferral memory is per process.