Skip to content

Auto create vector extension - #18

Closed
changergosum wants to merge 1 commit into
garrytan:masterfrom
changergosum:vector-extension
Closed

Auto create vector extension#18
changergosum wants to merge 1 commit into
garrytan:masterfrom
changergosum:vector-extension

Conversation

@changergosum

@changergosum changergosum commented Apr 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

DB connection is available at this point to auto enable vector extension instead of asking user to do it through another interface. This streamlines the install experience.

@garrytan garrytan mentioned this pull request Apr 11, 2026
Merged
4 of 5 tasks
@garrytan

garrytan commented Apr 11, 2026

Copy link
Copy Markdown
Owner

Included in the community fix wave (PR #38, v0.6.1)! Auto-create pgvector extension landed with a graceful fallback message. No more copy-pasting SQL. Thank you @changergosum! 🙌

@garrytan garrytan closed this Apr 11, 2026
garrytan added a commit that referenced this pull request Apr 20, 2026
@garrytan @claude
feat(progress): step 3b - embed + files + export stderr progress
3e5daea 
Replaces the \r-on-stdout progress pattern in the three worst offenders
(embed, files sync, export) with the shared reporter on stderr. Stdout
now carries only final summaries, so scripts and tests that grep for
counts ("Embedded N chunks", "Files sync complete", "Exported N pages")
still work when output is piped.

- embed.ts: runEmbedCore accepts an optional onProgress callback. The
  CLI wrapper builds a reporter and passes reporter.tick(); Minion
  handlers will pass job.updateProgress in Step 4. Worker-pool is
  single-threaded JS so no rate-gate race (per Codex review #18).
- files.ts syncFiles(): tick per file; summary preserved on stdout.
- export.ts: tick per page; summary preserved on stdout.

Also fixes a --quiet flag collision. `skillpack-check` has its own
--quiet mode (suppress all stdout). parseGlobalFlags strips --quiet
globally now, and skillpack-check reads the resolved CliOptions
singleton via getCliOptions() instead of re-parsing argv. Test updated
to match the stripping behavior.

1686 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
garrytan added a commit that referenced this pull request Apr 22, 2026
@garrytan @claude
feat: v0.15.2 - bulk-action progress streaming (stderr reporter, agen…
a4df40f 
…t-visible heartbeats) (#293)

* feat(progress): step 1 - shared ProgressReporter + CliOptions

Adds the foundation for v0.14.2's bulk-action progress streaming work:

- src/core/progress.ts: dependency-free reporter with auto/human/json/quiet
  modes, TTY-aware rendering, time+item rate gating, heartbeat helper for
  slow single queries, dot-composed child phases, EPIPE defense (both sync
  throw and async 'error' event), and a singleton module-level signal
  coordinator so SIGINT/SIGTERM emits abort events for all live phases
  without leaking per-instance listeners.

- src/core/cli-options.ts: parseGlobalFlags() for --quiet /
  --progress-json / --progress-interval=<ms> (both space and = forms),
  plus cliOptsToProgressOptions() that resolves to the right mode. Non-TTY
  default is human-plain one-line-per-event; JSON is explicit opt-in so
  shell pipelines don't suddenly see structured noise.

- test/progress.test.ts (17 cases): mode resolution, rate gating, no-fake-
  totals on heartbeat paths, EPIPE paths, SIGINT singleton, child phase
  composition.

- test/cli-options.test.ts (14 cases): flag parsing, invalid values,
  interleaved flags, mode resolution.

Follow-ups wire doctor/embed/files/export/extract/import/sync/migrate/
repair-jsonb/backlinks/orphans/lint/integrity/eval/autopilot/jobs plus
the apply-migrations orchestrators through this reporter, and route
Minion handlers to job.updateProgress instead of stderr. See the plan
in ~/.claude/plans/.

1682 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 2 - wire global flags into cli.ts

Parse --quiet / --progress-json / --progress-interval from argv BEFORE
command dispatch, strip them, stash resolved CliOptions on a module-level
singleton (same pattern as Commander's program.opts()) and on every
OperationContext created for shared-op dispatch.

- src/cli.ts: parseGlobalFlags(rawArgs) at the top of main(); setCliOptions
  once; dispatch sees only the stripped argv. Fixes the "gbrain
  --progress-json doctor" unknown-command case that Codex flagged.
- src/core/cli-options.ts: expose setCliOptions/getCliOptions/
  _resetCliOptionsForTest singleton. Commands that want progress call
  getCliOptions() to construct their reporter.
- src/core/operations.ts: OperationContext gains optional cliOpts field
  so shared-op handlers (and MCP-invoked ops that need a reporter) can
  read the same settings. MCP callers leave it undefined and consumers
  default to quiet.
- test/cli-options.test.ts: +4 cases covering singleton round-trip and
  an integration smoke spawning `bun src/cli.ts --progress-json --version`
  to prove the global flag survives dispatch.

45 relevant unit tests pass (progress + cli-options + cli.test.ts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 3a - doctor + orphans heartbeat streaming

Doctor on a 52K-page brain used to sit silent for 10+ minutes while the
DB checks ran, then get killed by an agent timeout. Wired through the
new reporter so agents see which check is running and the slow ones
heartbeat every second.

doctor.ts:
- Start a single `doctor.db_checks` phase around the DB section, with a
  per-check heartbeat before each step (connection, pgvector, rls,
  schema_version, embeddings, graph_coverage, integrity, jsonb_integrity,
  markdown_body_completeness).
- jsonb_integrity now scans 5 targets, not 4: added page_versions.
  frontmatter so the check surface matches `repair-jsonb` (per Codex
  review of the plan — the old 4-target scan missed a known repair site).
  Per-target heartbeat so 50K-row scans show incremental progress.
- markdown_body_completeness: wrap the existing query in a 1s heartbeat
  timer. The regex scan over rd.data ->> 'content' can't be paginated
  usefully; this just lets agents see life during the sequential scan.
  No fake totals — the LIMIT 100 query has no meaningful total count.
- integrity sample: same heartbeat pattern around the 500-page scan.

orphans.ts:
- findOrphans() wraps the NOT EXISTS anti-join in a 1s heartbeat.
  Keyset pagination was considered and rejected: without an index on
  links.to_page_id it's no faster than the full scan, and may re-plan
  the anti-join per batch. A schema migration adding that index is the
  right fix and is queued for v0.14.3.

Follow-ups:
- Step 3b: wire embed/files/export (the \r-only stdout offenders).
- Step 5: end-to-end progress test spawning `gbrain doctor --progress-json`
  against a fixture brain, asserting stderr events and clean stdout.

All existing unit tests continue to pass (76/76 in doctor + orphans +
progress + cli-options).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 3b - embed + files + export stderr progress

Replaces the \r-on-stdout progress pattern in the three worst offenders
(embed, files sync, export) with the shared reporter on stderr. Stdout
now carries only final summaries, so scripts and tests that grep for
counts ("Embedded N chunks", "Files sync complete", "Exported N pages")
still work when output is piped.

- embed.ts: runEmbedCore accepts an optional onProgress callback. The
  CLI wrapper builds a reporter and passes reporter.tick(); Minion
  handlers will pass job.updateProgress in Step 4. Worker-pool is
  single-threaded JS so no rate-gate race (per Codex review #18).
- files.ts syncFiles(): tick per file; summary preserved on stdout.
- export.ts: tick per page; summary preserved on stdout.

Also fixes a --quiet flag collision. `skillpack-check` has its own
--quiet mode (suppress all stdout). parseGlobalFlags strips --quiet
globally now, and skillpack-check reads the resolved CliOptions
singleton via getCliOptions() instead of re-parsing argv. Test updated
to match the stripping behavior.

1686 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 3c - extract + import + sync reporter streaming

Extract, import, and sync now stream per-file progress to stderr through
the shared reporter. All three kept their stdout summaries + JSON
action-events intact so existing tests + agent scripts are unaffected.

- extract.ts (4 paths: links/timeline × fs/db): replaced the ad-hoc
  `process.stderr.write({event:"progress"...})` lines with reporter
  ticks. Same channel (stderr), canonical schema now, visible in both
  text and --json modes. Stdout action-events (`add_link` /
  `add_timeline`) untouched — tests grep them.
- import.ts: the logProgress() function that printed every 100 files to
  stdout is now a progress.tick() call per file. Rate-gated by the
  reporter. Stdout still gets the final "Import complete (Xs)" summary
  and the --json payload.
- sync.ts: three new phases (`sync.deletes`, `sync.renames`,
  `sync.imports`) tick per file, so big syncs show each step rather than
  a single end-of-run summary. Phase hierarchy ready to be child()-chained
  into runImport / runEmbed later, per Codex review #26.

Updated the #132 nested-transaction regression test in test/sync.test.ts
to also accept the new hoisted-loop shape — the guarantee (this loop is
not wrapped in engine.transaction) still holds.

1686 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 3d - migrate/repair/backlinks/lint/integrity/eval

Wires the remaining bulk commands through the reporter:

- migrate-engine: phase starts (migrate.copy_pages, migrate.copy_links),
  per-page tick. Old \"Progress: N/total\" stdout logs replaced by
  stderr ticks; final stdout summary preserved.
- repair-jsonb: per-column start + a heartbeat timer while each UPDATE
  runs (minutes on 50K-row tables). CRITICAL: stdout stays clean so
  migrations/v0_12_2.ts's JSON.parse(child.stdout) still works. Per
  Codex review #12.
- backlinks: 1s heartbeat around findBacklinkGaps() (sync double-walk
  of the brain dir).
- lint: tick per page; per-issue stdout output preserved.
- integrity auto: tick per page in the main resolver loop. The separate
  ~/.gbrain/integrity-progress.jsonl resume marker is untouched (its
  role shifts from live progress reporting to resume-only).
- eval: add an onProgress option to core's runEval(), CLI wraps with a
  reporter. Phases: eval.single / eval.ab. Tick per query.

core/search/eval.ts gains a RunEvalOptions type so future callers (MCP
eval op, Minion handlers) can also hook in without the reporter.

1686 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 3e - onProgress callbacks on core libs

- src/core/embedding.ts: embedBatch() gains an optional
  EmbedBatchOptions.onBatchComplete callback, fired after each 100-item
  sub-batch. CLI wrappers pass reporter.tick; Minion handlers can pass
  job.updateProgress.
- src/core/enrichment-service.ts: enrichEntities() config gains
  onProgress(done, total, name) fired after each entity. Same split:
  CLI -> reporter, Minion -> DB-backed progress.

No CLI behavior change on its own. Wiring these callbacks into the
Minion handlers is Step 4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 4 - orchestrators + upgrade + minion handlers

- cli-options.ts: childGlobalFlags() returns the flag suffix to append
  to child gbrain subprocesses. Empty string by default, " --quiet
  --progress-json" when the parent has them set, so child behavior
  inherits the parent's progress-mode without scattering string-concat
  logic across every execSync site.

- migrations/v0_12_2.ts: each execSync inherits the parent's global
  flags. Phase C (repair-jsonb --dry-run --json) pins explicit stdio to
  ['ignore','pipe','inherit'] so child stderr streams straight through
  while stdout stays captured for JSON.parse. Per Codex review #12.
- migrations/v0_12_0.ts + v0_11_0.ts: same childGlobalFlags wiring at
  each gbrain-subcommand execSync.

- upgrade.ts: post-upgrade timeout bumped 300s → 30min (1_800_000 ms)
  with GBRAIN_POST_UPGRADE_TIMEOUT_MS override. The old 300s cap killed
  v0.12.0 graph-backfill migrations on 50K+ brains; the heartbeat
  wiring added in v0.14.2 makes long waits observable, so a generous
  ceiling no longer means users stare at a silent terminal.

- jobs.ts: the embed Minion handler passes job.updateProgress as the
  onProgress callback, so per-job progress is durable in minion_jobs
  and readable via `gbrain jobs get <id>`. Primary Minion progress
  channel is DB-backed — stderr from `jobs work` stays coarse for
  daemon liveness only. Per Codex review #20.

1686 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 5 - E2E doctor-progress test + CI guard

scripts/check-progress-to-stdout.sh greps src/ for the banned
`process.stdout.write('\r…')` pattern that v0.14.2 removed from the
bulk-action codepaths. Wired into the `bun run test` script so any
future regression that puts progress back on stdout fails fast. An
empty allowlist documents the position: every known call site was
migrated; new exceptions need a rationale in the allowlist.

test/e2e/doctor-progress.test.ts (Tier 1, needs Postgres + pgvector):
- `gbrain --progress-json doctor --json`: stderr carries JSONL progress
  events with the canonical {event, phase, ts} shape, starts + finishes
  for `doctor.db_checks`. Stdout stays parseable JSON — no progress
  pollution.
- `gbrain doctor` (no flag): human-plain progress goes to stderr only,
  stdout stays free of `[doctor.db_checks]`.
- `gbrain --quiet doctor`: reporter emits nothing; doctor still runs to
  completion.

test/cli-options.test.ts: +2 spawning integration tests. One verifies
`gbrain --progress-json --version` keeps stdout clean of progress events
(single-shot commands that don't use a reporter aren't affected). One
guards the skillpack-check --quiet regression — --quiet suppresses
stdout by reading the resolved CliOptions singleton, not re-parsing argv.

Full test matrix:
  bun run test           -> 1726 pass / 184 skipped (no DB) / 0 fail
  bun run test:e2e       -> 136 pass / 13 skipped / 0 fail

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(progress): step 6 - docs + v0.14.2 release bump

- VERSION + package.json bumped to 0.14.2.
- docs/progress-events.md (new): canonical JSON event schema reference.
  Stable from v0.14.2, additive only. Lists every phase name shipped
  in this release, the five event types (start/tick/heartbeat/finish/
  abort), the TTY/non-TTY rendering rules, subprocess inheritance
  semantics, and the Minion DB-backed progress model.
- CLAUDE.md: "Bulk-action progress reporting" section under the build
  instructions; Key files entries for src/core/progress.ts,
  src/core/cli-options.ts, scripts/check-progress-to-stdout.sh, and
  docs/progress-events.md; doctor.ts entry updated to note the v0.14.2
  5-target jsonb_integrity scan + heartbeat wiring.
- CHANGELOG.md v0.14.2: full release summary per project voice rules.
  The "numbers that matter" table, per-command before/after grid,
  backward-compat warnings for stdout→stderr moves, and an itemized
  changes section covering reporter/CLI plumbing/schema/Minion
  handlers/doctor fixes/upgrade timeout/CI guard/tests. No em dashes.
  Real file paths, real commands, real numbers.
- skills/migrations/v0.14.2.md (new): agent migration note. Mechanical
  step is "nothing" since v0.14.2 is purely additive. Walks agents
  through the three new global flags, the 14 wired commands, the event
  schema cheat sheet, Minion progress via job.updateProgress, and
  scripts/verification commands.

Full test matrix:
  bun run test (unit + guards) -> 1726 pass / 184 skipped / 0 fail
  bun run test:e2e (Postgres)  -> 141 pass / 8 skipped / 0 fail

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version to 0.15.2, restore master's [0.14.2] CHANGELOG entry

Master sits at 0.14.2 (reliability wave). This PR lands on top as 0.15.2
(progress streaming wave). Splits the merge-time combined CHANGELOG entry
back into two discrete release sections so history stays honest:

- [0.15.2] = progress reporter, CliOptions, 14 wired commands, Minion
  embed handler, doctor jsonb_integrity 5-target fix, upgrade timeout bump,
  CI guard, progress unit+E2E tests.
- [0.14.2] = master's eight root-cause bug fixes, restored verbatim from
  origin/master.

Touched files:
- VERSION + package.json: 0.14.2 -> 0.15.2 (next patch off master).
- skills/migrations/v0.14.2.md -> skills/migrations/v0.15.2.md (rename
  + rewrite frontmatter + body to v0.15.2).
- CHANGELOG.md: split into two entries; progress-wave refs renamed
  v0.14.2 -> v0.15.2; reliability-wave entry restored from master.
- src/core/progress.ts, src/commands/doctor.ts, src/commands/sync.ts,
  src/commands/upgrade.ts, docs/progress-events.md, test/sync.test.ts:
  progress-wave v0.14.2 references -> v0.15.2. The remaining v0.14.2
  references in test/e2e/migration-flow.test.ts (Bug 3 context) and
  CLAUDE.md (reliability-wave key commands, Bug 3 ledger move) correctly
  point at master's 0.14.2 release.

Test matrix after version bump:
  bun run test -> 1780 pass / 179 skipped / 0 fail

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
garrytan added a commit that referenced this pull request May 23, 2026
@garrytan @claude
v0.40.4.0 feat(search): selective graph signals + per-stage attributi…
d28be5d 
…on + audit-writer unification (#1300)

* v0.40.4.0 T1: shared audit-writer primitive

Extract createAuditWriter() helper. Five hand-rolled JSONL audit
modules (rerank-audit, shell-audit, supervisor-audit, audit-slug-
fallback, phantom-audit) duplicated the same ISO-week filename math,
best-effort write loop, and read-current-plus-previous-week loop.
T2 refactors all 5 onto this primitive.

Behavior preservation: filename format, JSONL line shape, mkdir
recursive, appendFileSync utf8, stderr-on-failure all byte-identical
to the existing modules so their tests pass unchanged.

resolveAuditDir() moves here from shell-audit.ts; shell-audit.ts
will re-export for back-compat (T2). Honors GBRAIN_AUDIT_DIR with
whitespace-trim, falls back to ~/.gbrain/audit/.

Test coverage: 22 cases covering ISO-week math + year-boundary edges
(2027-01-01 → 2026-W53), env override, mkdir-recursive, fail-open
stderr-warn shape, cross-week readback, corrupt-row skip, non-finite-
ts skip, round-trip with nested fields, computeFilename + resolveDir
accessors.

Plan ref: D5=B audit unification cathedral expansion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T2: refactor 5 audit modules onto shared writer

Replace the duplicated ISO-week filename math + best-effort write loop
+ read-current-plus-previous-week loop in:
  - src/core/rerank-audit.ts (rerank-failures-*.jsonl)
  - src/core/audit-slug-fallback.ts (slug-fallback-*.jsonl)
  - src/core/minions/handlers/shell-audit.ts (shell-jobs-*.jsonl)
  - src/core/minions/handlers/supervisor-audit.ts (supervisor-*.jsonl)
  - src/core/facts/phantom-audit.ts (phantoms-*.jsonl)

All five now delegate file I/O to createAuditWriter from T1. Public
API preserved bit-for-bit:
  - logRerankFailure, readRecentRerankFailures, computeRerankAuditFilename
  - logSlugFallback, readRecentSlugFallbacks, computeSlugFallbackAuditFilename
  - logShellSubmission, computeAuditFilename, resolveAuditDir
  - writeSupervisorEvent, readSupervisorEvents, computeSupervisorAuditFilename
    plus isCrashExit, summarizeCrashes, CrashSummary (domain-specific
    helpers stay in supervisor-audit.ts; only file I/O moves)
  - logPhantomEvent, readRecentPhantomEvents, computePhantomAuditFilename

Domain-specific behavior preserved:
  - audit-slug-fallback emits per-call stderr (D7 dual logging) in the
    caller; the shared writer is failure-only stderr
  - rerank-audit truncates error_summary to 200 chars before write
  - phantom-audit spreads optional fields conditionally (skip undefined)
  - supervisor-audit keeps single-file readback (no cross-week walk)
    to preserve pre-v0.40.4 doctor assertions

resolveAuditDir lives in src/core/audit/audit-writer.ts; shell-audit.ts
re-exports it so existing imports keep working (every other audit
module + gbrain-home-isolation.test.ts + minions.test.ts +
minions-shell.test.ts pull resolveAuditDir from shell-audit.ts).

Operator-visible drift: rerank-audit stderr line drops the
'rerank-failure audit' qualifier — was '[gbrain] rerank-failure audit
write failed (...)' now '[gbrain] write failed (...); search continues'.
Stderr is human-debugging, not machine-parsed; the file written gives
the qualifier away in `tail -f audit/*`.

Test coverage: 128/128 audit-touching tests pass unchanged.

Plan ref: D5=B audit unification cathedral expansion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T3: getAdjacencyBoosts engine method (PG+PGLite parity)

Add BrainEngine.getAdjacencyBoosts(pageIds) returning Map<page_id,
AdjacencyRow{hits, cross_source_hits}>. Returns ALL pages with
hits >= 1 (callers apply their own threshold).

Cross-source semantic (D15=A): cross_source_hits EXCLUDES the target
page's own source. A page in source A linked from 2 pages in source A
reports cross_source_hits = 0. Linked from 1 in source B + 1 in
source C reports 2.

Source-scope contract: pageIds MUST already be source-scoped by the
caller. Method does NOT filter by source_id. The in-set restriction
makes cross-source leakage impossible by construction. JSDoc spells
this out; same trust posture as cosineReScore's chunk_id handling.

COALESCE(p.source_id, 'default') on both target and from-page sides
for defense-in-depth even though pages.source_id is NOT NULL today.

JSDoc/SQL contract alignment (codex #2): HAVING >= 1 matches the
"returns ALL pages with hits >= 1" contract; threshold of 2 is the
caller's call in applyGraphSignals.

Known limitation (codex #15): cross_source_hits cannot distinguish
"genuinely linked from another team" from "mirrored imports from
another source." T-todo-4 captures the v0.41+ refinement.

SearchResult type extension (D4=A flat fields, D12=A attribution):
  - graph_adjacency_hits, graph_cross_source_hits,
    graph_session_demoted, graph_session_prefix
  - base_score, backlink_boost, salience_boost, recency_boost,
    exact_match_boost, graph_adjacency_boost, graph_cross_source_boost,
    session_demote_factor, reranker_delta
All optional; T4-T6 populate them.

Test coverage: 7/7 hermetic PGLite cases. Empty input, singleton,
same-source hub, cross-source attribution including the
"linked-only-from-other-source" case (widget in source b, linked
from alice+bob in source a → cross_source_hits=1), JSDoc HAVING>=1
contract. Postgres parity asserted by SQL-shape identity (will get a
mirror Postgres E2E in T10's eval gate work via DATABASE_URL when
set; PGLite hermetic case shipped now).

NULL source_id COALESCE branch noted as untestable in current PGLite
schema (pages.source_id is NOT NULL); kept as defense-in-depth.

Plan ref: T3 in v0.40.4.0 wave plan; D1=A, D3=A, D15=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T4+T11: applyGraphSignals 4th stage in runPostFusionStages

New file src/core/search/graph-signals.ts. Three signals:

  1. Adjacency-within-top-K (×1.05): hits >= 2 inbound from in-set.
  2. Cross-source adjacency (×1.10, stacks): cross_source_hits >= 2.
     Dormant on single-source brains.
  3. Session diversification (×0.95): if multiple top-K share a slug
     prefix, keep highest scoring, DEMOTE the rest. NOT amplify —
     codex caught the original framing was backwards (amplification
     of redundancy makes the cited "weak chunks compete for budget"
     problem worse, not better).

Conservative magnitudes (D14=B): 1.05/1.10/0.95. Score-distribution
probe (onScoreDistribution) collects min/p25/p50/p75/p95/max +
reorder_band_width to feed T-todo-2 magnitude calibration wave.

Slot: 4th stage inside runPostFusionStages (hybrid.ts:248), AFTER
backlink/salience/recency, pre-dedup. Inherits the v0.35.6.0
floor-ratio gate from computeFloorThreshold — this is the structural
protection that prevents a low-cosine hub from outranking a strong
non-hub (codex T2 / D1=A).

PostFusionOpts extends with graphSignalsEnabled, onGraphMeta,
onScoreDistribution. Caller (hybridSearch in subsequent T5 work)
resolves graph_signals from the mode bundle.

Source-scope contract preserved: getAdjacencyBoosts takes raw
page_ids, no source filter. Adjacency is in-set restricted so
cross-source leakage is impossible by construction (D3=A).

Fail-open: engine throw → JSONL audit row via shared createAuditWriter
(T1/T2 primitive, featureName='graph-signals-failures') + meta.errored
+ caller's results unchanged. Session diversification ALSO skips on
failure (predictable all-or-nothing posture).

Mutation note (codex #9): score mutated in place. base_score must be
stamped at runPostFusionStages entry BEFORE this stage so eval-capture
sees pre-boost score (T6 attribution wave).

Test coverage (24 cases, including T11 IRON RULE regression):
  - sessionPrefix multi/single/empty cases
  - computeScoreDistribution percentile math
  - Disabled + empty short-circuits
  - Adjacency hit, no-hit, cross-source stacking, cross-source alone
  - Session diversification 3-share + single-segment + singleton
  - Test seam injection (no engine call)
  - Fail-open: throw → audit row + meta.errored + unchanged
  - Empty Map → session still runs
  - Score-distribution always emits when enabled
  - Meta carries fire counts + duration_ms
  - Missing page_id silently skipped from dedup set
  - **T11 IRON RULE regression (3 cases):**
    * weak hub BELOW floor_threshold does NOT get boosted past
      above-floor non-hub (the bug class the floor gate exists for)
    * hub AT floor still gets boosted (gate is < not <=)
    * NaN score → NaN >= threshold is false → no boost

Plan ref: T4 + T11 in v0.40.4.0 wave plan; D1=A, D2=A, D11=B, D14=B,
D9=A, D5=B. Codex outside-voice #1 + #2 + #6 + #8 + #9 addressed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T5: graph_signals mode-bundle knob + KNOBS_HASH bump 3→4

ModeBundle gains graph_signals: boolean. Per-mode defaults:
  - conservative: false (cost-sensitive tier)
  - balanced:     true (the wave's primary surface for default-on)
  - tokenmax:     true (power-user tier, capstone fit)

SearchKeyOverrides + SearchPerCallOpts gain optional graph_signals
field. resolveSearchMode picks via the standard per-call → config
override → mode bundle chain.

loadOverridesFromConfig parses 'search.graph_signals' from the config
table ('1' or 'true' → true). SEARCH_MODE_CONFIG_KEYS adds the key
so `gbrain search modes --reset` clears it alongside other knobs.

KNOBS_HASH_VERSION bump 3→4 (append-only per CDX2-F13). New `gs=`
parts entry appended AFTER cross-modal + column + prov entries. A
graph-on cache write cannot be served to a graph-off lookup —
mid-deploy hit-rate dip clears within cache.ttl_seconds (3600s).

src/commands/search.ts KNOB_DESCRIPTIONS gains graph_signals entry
so `gbrain search modes` dashboard renders the new knob.

Test coverage:
  - test/search-mode.test.ts (+ 8 new cases): per-mode defaults
    canonical, config override both directions, per-call override
    wins, knobsHash distinct for on/off, config key registered,
    attributeKnob reports per-call + mode sources correctly.
  - test/search/knobs-hash-reranker.test.ts: version assertion
    bumped 3→4 with v0.40.4 rationale comment.
  - test/cross-modal-phase1.test.ts: version assertion bumped
    3→4 with v0.40.4 rationale comment.
  - Canonical-bundle assertions updated to include graph_signals
    in expected shape (3 cases).

50/50 search-mode tests pass. 45/45 cross-modal pass. 17/17
knobs-hash-reranker pass. 10/10 balanced-reranker pass.

Plan ref: T5 in v0.40.4.0 wave plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T6: per-stage attribution stamping in every boost

Every boost stage that mutates SearchResult.score now stamps a field
recording WHAT it multiplied:

  - applyBacklinkBoost  → backlink_boost (skipped when count == 0)
  - applySalienceBoost  → salience_boost (skipped when score == 0)
  - applyRecencyBoost   → recency_boost (skipped on evergreen prefix)
  - applyExactMatchBoost → exact_match_boost (skipped on no-match
    OR when intent's exactMatchBoost == 1.0 no-op)
  - runPostFusionStages → base_score stamped ONCE at entry, BEFORE
    any boost mutates r.score. Idempotent: caller-pre-stamped value
    preserved. Empty-results short-circuit unchanged.
  - applyReranker → reranker_delta = original_index - new_index
    (positive = rank improved; raw rerank score stays in rerank_score)
  - applyGraphSignals → graph_adjacency_boost, graph_cross_source_boost,
    session_demote_factor (T4 already stamped these)

Why: feeds the T7 `gbrain search --explain` formatter so it can
attribute the final score to its components. Without these stamps,
"why did this rank where it did?" is grep-and-guess.

SearchResult.reranker_delta doc updated to clarify it's a RANK delta
(positive = improved), not a score delta. The raw relevance score
stays in `rerank_score` (untyped, for back-compat with telemetry that
already reads it).

Test coverage: 16 new cases in test/search/attribution-stamping.test.ts.
Pins: every boost stamps when it fires AND skips stamping when it
doesn't (no false attribution on no-op stages). base_score idempotency
preserved. reranker_delta computed correctly across rank-improved +
rank-degraded cases.

All 178/178 search tests pass (no regressions).

Plan ref: T6 cathedral expansion in v0.40.4.0 wave plan; D12=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T7: gbrain search --explain per-stage attribution

New file src/core/search/explain-formatter.ts renders SearchResult[]
as a multi-line breakdown of how the final score was formed:

  1. people/alice (score=12.4)
     base=10.2 (rrf+cosine)
     + backlink ×1.08
     + salience ×1.05
     + adjacency ×1.05 (hits=3)
     + cross_source ×1.10 (other_sources=2)
     ↑ reranker rank +2
     = final 12.4

Reads the boost_* / base_score / *_hits fields populated by T4 + T6.
Empty path: "no boosts applied" when no stage stamped anything.
Session demote rendered with `-` prefix (not `+`) so the demotion
direction is visually distinct from boosts.

CliOptions gains `explain: boolean`; parseGlobalFlags recognizes
`--explain` anywhere in argv. cli.ts formatResult for `search` +
`query` cases reads CliOptions.explain via the module-level
singleton and routes to formatResultsExplain when set. Lazy import
keeps the hot path narrow for the common non-explain case.

Number formatting: 4-decimal precision, trailing zeros stripped
('1.0000' → '1', '0.1234' → '0.1234'). NaN preserved as 'NaN'.

Test coverage:
  - test/search/explain-formatter.test.ts: 19 cases pin output
    format. Each boost type renders correctly, every-stage stacking
    composes, reranker_delta=0 doesn't render, empty list short-
    circuits, rank numbering 1-based, number formatting edge cases.
  - test/cli-options.test.ts: 3 new cases for --explain parsing
    (basic, absent default, any-argv-position).

Existing CliOptions literals in test/cli-options.test.ts +
test/thin-client-upgrade-prompt.test.ts updated for new required
explain field.

JSON envelope unchanged — the same attribution fields surface in
existing --json output via JSON.stringify; no separate JSON formatter
needed.

Plan ref: T7 cathedral expansion in v0.40.4.0 wave plan; D12=A + D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T8: doctor check graph_signals_coverage

New checkGraphSignalsCoverage in src/commands/doctor.ts. Wired into
both runDoctor (local engine) and doctorReportRemote (HTTP MCP /
JSON path) so local AND remote-server brains both surface the metric.

Logic:
  1. Resolve active graph_signals setting: config override
     'search.graph_signals' wins, else mode bundle default
     ('search.mode' → conservative=false, balanced/tokenmax=true).
  2. When disabled → silent ok ("disabled — coverage not checked").
     Avoids polluting doctor output on installs that don't use the
     feature.
  3. When enabled, compute global inbound-link density:
     COUNT(DISTINCT to_page_id) / COUNT(*) across non-deleted pages.
  4. <10% → warn ("signal will rarely fire") with paste-ready
     `gbrain extract all` fix hint.
  5. >=30% → ok ("fire on most queries") with metric.
  6. 10-29% → ok ("fire occasionally") with metric.

Known limitation (codex outside-voice #14): global density is an
imperfect proxy for "top-K subgraphs have enough edges to fire."
T-todo-5 captures the v0.41+ refinement that measures actual fire
rate from search-stats after 30 days of data.

Best-effort: SQL errors → warn with the underlying message. Never
breaks doctor.

Test coverage (7 new cases in test/doctor.test.ts):
  - conservative mode → silent ok regardless of coverage
  - balanced default + 0 links → warn at 0% with fix hint
  - balanced default + 40% inbound → ok "fire on most queries"
  - balanced default + 20% inbound → ok "fire occasionally"
  - explicit search.graph_signals=false overrides mode default
  - empty brain → ok with explanation
  - check is wired into runDoctor (source-grep regression guard)

All 55/55 doctor.test.ts cases pass.

Plan ref: T8 in v0.40.4.0 wave plan; D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T9: gbrain search stats graph_signals section

runStatsSubcommand in src/commands/search.ts gains a graph_signals
section in both --json and human output:

  Graph signals:
    enabled:    true (mode default)
    failures:   3 fail-open event(s)
      ECONNREFUSED         2
      timeout              1

Data sources:
  - config: 'search.graph_signals' override → enabled + source=config,
    otherwise mode-bundle default → enabled + source=mode_default.
  - JSONL audit: readRecentGraphSignalsFailures(days) returns events;
    failures_count is len, failures_by_reason buckets by first word of
    error_summary (e.g. 'ECONNREFUSED', 'timeout').

JSON envelope (schema_version 2 unchanged; graph_signals is a new
sibling property of stats, so consumers reading the existing fields
keep working):

  {
    "schema_version": 2,
    ...stats...,
    "graph_signals": {
      "enabled": bool,
      "source": "config" | "mode_default",
      "failures_count": int,
      "failures_by_reason": { reason: count }
    },
    "_meta": { metric_glossary: { ..., graph_signals_enabled: ..., graph_signals_failures_count: ... } }
  }

Fire-rate metrics (adjacency_fires, cross_source_fires,
session_demotions) and score-distribution stats are NOT in this
section yet — they require telemetry-table writes from the
applyGraphSignals onMeta callback. Wired in v0.41+ via T-todo-2
calibration wave (the wave that needs them). For v0.40.4: status +
error count is the actionable surface for "is graph_signals on, and
is it failing?"

Human output: prints the section after the existing stats block.
Edge case: when total_calls is 0 BUT graph_signals is enabled OR
has historical failures, still prints the section so operators
don't lose the signal on a brain with no telemetry yet.

Test coverage (6 cases in test/search/search-stats-graph-signals.test.ts):
  - search.graph_signals=true → enabled true, source=config
  - mode=conservative → enabled false, source=mode_default
  - no config → enabled true (balanced default), source=mode_default
  - JSONL failures bucketed by first word of error_summary
  - empty audit → failures_count 0, empty failures_by_reason
  - human output includes "Graph signals:" header

Plan ref: T9 in v0.40.4.0 wave plan; D6=A.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T10: eval gates (longmemeval-mini A/B + paired bootstrap)

New test/e2e/graph-signals-eval.test.ts runs each longmemeval-mini
question twice (graph_signals off, graph_signals on) and asserts:

  Gate 1 (QUALITY) — paired bootstrap, 10,000 resamples:
    - If signals-on is significantly WORSE than off
      (delta < 0 AND p < 0.05) → fail.
    - Otherwise pass. p>=0.05 either direction OR delta >= 0 → ok.

  Gate 2a (CHANGE-MAGNITUDE): mean Jaccard@5 over result-set overlap
    must be >= 0.5. If results overlap less than half, the change is
    too large and needs human review before default-on.

  Gate 2b (CHANGE-MAGNITUDE): top-1 stability rate >= 0.7. If 30%+
    of top picks change, hard look required.

  Gate 3 (HARD ABSOLUTE FLOOR): recall@5 drop <= 5pt. Catastrophic
    regression catch (codex outside-voice #18 — addresses the "top-5
    must not drop at all" brittleness on tiny fixtures).

Bootstrap implementation:
  - Per-question observation is binary (recall@5 hit/miss).
  - Paired pairing on question_id between on/off branches.
  - Centered distribution under null (subtract observed mean) per
    standard paired-bootstrap-shift approach for binary outcomes.
  - Two-tailed p-value: |resampled delta| >= |observed delta|.
  - Deterministic seeded RNG so test runs are stable across CI.

pairedBootstrapPValue exported as a pure function with separate
tests for edge cases (empty input, all-equal, strong positive, strong
negative, determinism). Reusable from future calibration waves.

Hermetic: in-memory PGLite via createBenchmarkBrain + resetTables
between questions. No API keys needed (--no-embed import path
exercises keyword-only retrieval). Skips gracefully via describe.skip
when the fixture is missing.

Plan ref: T10 in v0.40.4.0 wave plan; D7=C absolute floor + D13=A
paired bootstrap; codex #4 + #18 stability-vs-quality distinction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 T12: VERSION + package.json + CHANGELOG + TODOS

VERSION: 0.37.11.0 → 0.40.4.0
package.json: 0.37.11.0 → 0.40.4.0
CHANGELOG.md: top entry for v0.40.4.0 in ELI10-lead voice per
  CLAUDE.md release rules. Lead is plain-English ("Your search now
  notices when a page is a hub for your query"); precise file paths
  / SQL semantics / numbers live in the "Itemized changes" section
  below. Includes the cathedral-expansion notes (D5=B audit
  unification, D12=A per-stage attribution, D13=A eval gates) and
  the "To take advantage of v0.40.4.0" verify-and-fix block.

TODOS.md: 5 new items captured under "v0.40.4 graph signals —
deferred follow-ups (v0.41+)":
  - T-todo-1: profile graph-signal SQL latency, merge if hot (D8=C)
  - T-todo-2: magnitude calibration wave from probe data (D14=B / D17)
  - T-todo-3: DB-backed audit table for cross-deploy observability (codex #15)
  - T-todo-4: sync-topology-aware cross-source signal (codex #11)
  - T-todo-5: replace doctor's global density with fire-rate (codex #14)

Verified the 3-line audit: VERSION + package.json + CHANGELOG topmost
all match 0.40.4.0. `bun install` ran (lockfile unchanged — root
package version isn't stored in bun.lock). `bun run build:llms`
refreshed llms.txt + llms-full.txt for the next commit.

Plan ref: T12 in v0.40.4.0 wave plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 TODO: document pre-existing shard-2 flake noticed during ship

3 isCacheSafe test failures in shard 2 reproduce on stashed clean
master. Confirmed pre-existing — not introduced by v0.40.4. Filed
under "Pre-existing flake on master (noticed during v0.40.4 ship)"
with reproduction commands + remediation options. Shipping v0.40.4
through it; future wave can fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 privacy scrub: replace wintermute → media in example slugs

CLAUDE.md line 550 bans the private OpenClaw fork name in public
artifacts. Example session prefix in sessionPrefix() docs + 3 test
fixtures swept to 'media/chat/...' instead. Pre-existing
scripts/check-privacy.sh in `bun run verify` caught it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: wire graph_signals from mode bundle to runPostFusionStages

CRITICAL: pre-landing review (codex outside-voice via /ship Step 9)
caught that hybrid.ts's `postFusionOpts` literal at line 566 was
building PostFusionOpts WITHOUT threading `resolvedMode.graph_signals`
to `graphSignalsEnabled`. The gate at hybrid.ts:358 read the field
from a literal that never set it.

Result before this fix: the entire v0.40.4 graph-signals wave was
dead code in production. Mode bundles set
`balanced.graph_signals = true` and `tokenmax.graph_signals = true`,
but no production call site ever reached applyGraphSignals. The
KNOBS_HASH bump 3→4 correctly varied the cache key by the flag, so
contamination was prevented — but the feature itself never fired.

All shipped infrastructure (engine SQL, fail-open audit, attribution
stamps, --explain formatter, doctor coverage check, search-stats
section) was reachable only through the unit-test seam
(`opts.adjacencyFn`). The CHANGELOG-advertised behavior never
landed in user-visible search.

Fix: thread `graphSignalsEnabled: resolvedMode.graph_signals` into
the postFusionOpts literal (1 line). Inline comment names codex's
catch so future refactors see the regression class.

Tests: new test/search/graph-signals-wire-integration.test.ts pins
the wire end-to-end. Three cases:
  1. balanced mode → hybridSearch on a seeded brain with adjacency
     hub produces a result with base_score stamped (proves
     runPostFusionStages actually ran).
  2. search.graph_signals=false config override → no graph_* fields
     stamped (proves the gate honors the override path).
  3. Source-grep regression guard pinning the
     `graphSignalsEnabled: resolvedMode.graph_signals` literal in
     hybrid.ts so a future refactor can't silently disconnect.

All 57 existing v0.40.4 wave tests still pass. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: pre-landing review AUTO-FIX findings (audit msg drift + deleted_at)

Two informational findings from /ship pre-landing review (Step 9):

1. Stderr message qualifier drift (rerank/slug-fallback/phantom audits)
   Pre-v0.40.4 messages included a per-feature qualifier:
     [gbrain] rerank-failure audit write failed (...)
     [gbrain] slug-fallback audit write failed (...)
     [gbrain] phantom audit write failed (...)
   The T2 refactor dropped the qualifier (plan promised "byte-identical"
   operator-visible behavior, but stderr lines did drift). Restored via
   new `errorMessagePrefix` option on `createAuditWriter` (optional, ''
   default). Three modules pass the per-feature qualifier; shell-audit
   and supervisor-audit unaffected (their pre-v0.40.4 messages didn't
   have a separate qualifier — label already carried the feature name).

2. Defense-in-depth `deleted_at IS NULL` on getAdjacencyBoosts
   SQL was previously protected by-construction (hybridSearch's
   visibility filter ensures input pageIds are live), but matches the
   v0.35.5.0 findOrphanPages pattern and closes the bug class if a
   future caller bypasses hybridSearch. Added to both Postgres and
   PGLite engines for parity. Three JOIN sites guarded (targets CTE,
   FROM-pages join). One inline comment per engine cites the codex
   review and the v0.35.5.0 precedent.

Plan ref: /ship pre-landing review v0.40.4.0 (codex finding C and F).

All 84 audit+graph-signals tests pass. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 fix: adversarial review HIGH findings (codex H1+H2 + Claude F1)

Three HIGH-severity issues from /ship adversarial pass:

H1 (Codex): Eval gate was a no-op.
  Test passed `graph_signals: graphSignalsOn` via `as any` cast, but
  SearchOpts had no field and hybridSearch's perCall didn't thread it.
  Both off/on branches resolved to the mode-bundle default — gate
  measured identical behavior, could pass while detecting nothing.

  Fix: add `graph_signals?: boolean` to SearchOpts (types.ts:794).
  Thread `opts.graph_signals` into perCall in both hybridSearch
  (hybrid.ts:425) AND hybridSearchCached (hybrid.ts:1027) so the
  cache-key resolver also sees the override. Drop the `as any` from
  the eval test — types are real now.

H2 (Codex): Session diversification fired on entity directories.
  sessionPrefix() used "any shared parent directory" as the session
  signal. Result: a search for "people in SF" returned `people/alice`
  + `people/bob` + `people/charlie` and the latter two got demoted
  to 0.95×. Every common entity-search query silently penalized
  legitimate same-type results. Default-on for balanced/tokenmax
  means production behavior was wrong.

  Fix: narrow sessionPrefix() to fire ONLY when the slug contains a
  session-like marker (`chat`/`session`/`sessions` segment OR a
  `YYYY-MM-DD` date segment). Entity directories (`people/`,
  `companies/`, `docs/`) return null → diversification skips.
  Returns NULL (not the slug itself) so the loop skips clean.
  Examples in JSDoc:
    your-agent/chat/2026-05-20-foo → 'your-agent/chat/2026-05-20-foo'
    daily/2026-05-20/journal-entry-1 → 'daily/2026-05-20'
    transcripts/chat/funding-discussion → 'transcripts/chat/funding-discussion'
    people/alice → null  ← codex H2 regression
    docs/quickstart → null

F1 (Claude adversarial subagent): case-sensitivity drift across 3 sites.
  loadOverridesFromConfig in mode.ts is case-insensitive +
  whitespace-trimmed for 'search.graph_signals' values. But
  doctor's checkGraphSignalsCoverage (doctor.ts:899) AND
  search-stats's readGraphSignalsStats (search.ts:288) used
  case-sensitive compare. User sets `search.graph_signals TRUE`:
  production enables the feature, but doctor + search-stats both
  silently report disabled. Operators lose the only observability
  surface for the new feature on values like 'True'/'TRUE'.

  Fix: trim + lowercase parity at both sites. Mirror the parser's
  semantic. Also case-normalized `search.mode` reads at both sites
  for the same divergence class.

Tests:
  - sessionPrefix block rewritten with 7 cases covering chat marker
    + date anchor + entity dirs (now-NULL) + degenerate (no /).
  - Added regression test pinning codex H2: people/alice +
    people/bob + people/charlie do NOT get diversified.
  - graph-signals-eval.test.ts drops `as any` — typed field works.
  - Existing tests using `chat/a`/`chat/b` updated to session-shaped
    `media/2026-05-20/chunk-a` so the date anchor actually fires.

111/111 graph-signals + doctor + search-stats tests pass. Typecheck clean.

Plan ref: /ship adversarial review v0.40.4.0 (codex H1, H2; Claude F1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.40.4.0 TODOs: capture 11 LOW adversarial findings for v0.41+

Codex L1 (audit window underreport) + Claude F2/F3/F5-F8/F11/F12/F14/F16
from /ship adversarial review. None are load-bearing; all captured under
'v0.40.4 adversarial review LOW findings — captured for v0.41+'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.40.4.0

- README: surface v0.40.4.0 graph signals + --explain in Hybrid search capability
- CLAUDE.md: annotate engine.ts getAdjacencyBoosts, new graph-signals.ts /
  explain-formatter.ts / audit/audit-writer.ts, plus hybrid.ts post-fusion
  4th stage, mode.ts graph_signals knob + KNOBS_HASH 3→4, cli-options.ts
  --explain flag, search stats + doctor coverage check
- llms-full.txt: regenerated from CLAUDE.md per the build:llms chaser rule

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(ci): pin bun-version to 1.3.13 across all workflows

setup-bun action with `bun-version: latest` calls the GitHub API
(https://api.github.com/repos/oven-sh/bun/git/refs/tags) to resolve
the tag. CI started failing today with HTTP 401 "Bad credentials"
even though the action receives a token (visible as `token: ***`
in the run log). Pinning the version eliminates the API call
entirely.

Affected workflows: test.yml, e2e.yml, release.yml, heavy-tests.yml
(5 invocations total). Pinned to 1.3.13 — matches package.json
engines (`bun >= 1.3.10`) and the version v0.40.4.0 was developed
against.

Bump cadence: when a new bun version is required, update this
pin in one PR. Trading "always-latest" for "always-deterministic"
is the right trade for a 5-shard CI matrix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@garrytan garrytan mentioned this pull request May 27, 2026
Merged
5 of 7 tasks
garrytan added a commit that referenced this pull request May 27, 2026
@garrytan @claude
v0.41.17.0 feat: --workers N on every bulk command + facts dim doctor…
8ab7334 
… parity (#1519)

* feat(worker-pool): shared sliding pool + bounded semaphore + PGLite-clamp wrapper

T1 + T2 of the v0.41.16.0 workers cathedral. New src/core/worker-pool.ts is
the canonical primitive every --workers N bulk command in this wave (and
future bulk commands) builds on. Atomic-claim invariant enforced by
scripts/check-worker-pool-atomicity.sh (wired into bun run verify).
BudgetExhausted bypass + AbortSignal composition baked into the helper so
budget caps are a structural ceiling under concurrency, not a per-caller
convention.

The new resolveWorkersWithClamp wrapper composes existing autoConcurrency
with PGLite-clamp + per-(command, requested) stderr dedup. Deliberately
NOT a modification to shared autoConcurrency (silent today, used by sync
+ import); embed.ts keeps GBRAIN_EMBED_CONCURRENCY || 20 default per
codex #13.

23 + 12 + 9 = 44 hermetic tests pin every contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: structural + dim-check regression suites for v0.41.16.0 wave

- test/embed-helper-migration.test.ts (T3): asserts embed.ts's two
  sliding-pool sites are migrated to runSlidingPool, pre-migration
  shapes (let nextIdx = 0, Promise.all(Array.from(...))) are gone,
  GBRAIN_EMBED_CONCURRENCY || 20 default preserved, failureLabel
  threads page.slug. Per codex #16/#17 these are invariant assertions,
  not byte-equality on progress event ORDERING.
- test/embedding-dim-check-facts.test.ts (T6): readFactsEmbeddingDim
  covers vector(N) + halfvec(N), halfvec-before-vector regex ordering
  pinned (codex #19), buildFactsAlterRecipe emits DROP INDEX + ALTER
  USING + CREATE INDEX (codex #18, not bare REINDEX),
  FactsEmbeddingDimMismatchError tagged class shape,
  assertFactsEmbeddingDimMatchesConfig PGLite skip + Postgres absent-
  column skip, doctor check + insert-cast wiring assertions.
- test/extract-conversation-facts-workers.test.ts (T5): helper
  exports (extractConversationFactsLockId, PER_PAGE_LOCK_TTL_MINUTES),
  structural wiring (runSlidingPool, resolveWorkersWithClamp,
  withRefreshingLock, LockUnavailableError, delete-orphans-first
  before segment loop, preflight before pool, exit 3 when lock_skipped
  > 0), Minion handler round-trip.
- test/extract-workers.test.ts (T7): --workers wiring on all 3 inner
  fs-walk loops (extractForSlugs, extractLinksFromDir,
  extractTimelineFromDir) + CLI parse + opts threading through
  runExtractCore.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump v0.41.16.0 → v0.41.17.0 (queue collision with PR #1510)

PR #1510 (garrytan/dynamic-regex-conversation-formats) claimed v0.41.16.0
on master in parallel. Advancing this wave to v0.41.17.0 so both can land
cleanly. Pure mechanical version bump:

- VERSION + package.json → 0.41.17.0
- CHANGELOG.md header + "To take advantage of v0.41.17.0" block
- TODOS.md section header + v0.41.18+ forward references
- CLAUDE.md inline version tags
- Regenerated llms-full.txt / llms.txt

No code changes. The actual workers cathedral feature set is unchanged
from the two prior commits in this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): search-image-column probes column dim at runtime

CI shard 5 failed on `searchVector column routing (v0.27.1)` with:
  error: expected 1280 dimensions, not 1536

The test had a hardcoded `fakeText1536` helper that seeded chunks at
1536-d vectors. Master's default embedding model switched from OpenAI
text-embedding-3-large (1536) to ZeroEntropy zembed-1 (1280) so a fresh
PGLite brain on CI now sizes content_chunks.embedding at 1280; the
test's 1536-d INSERT trips pgvector's CheckExpectedDim.

Fix: probe `content_chunks.embedding` width via
`readContentChunksEmbeddingDim(engine)` in `beforeAll`, store in
`TEXT_DIM`, and build `fakeTextDefault(seed)` at that width. The test
now passes regardless of which default ships (the model has flipped
twice and may flip again). Local dev (1536 from older config) and CI
fresh-install (1280 from new default) both pass.

Image-side vectors stay at 1024 (matches Voyage multimodal-3 + the
column's fixed width on the image side).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): bump PGLite hook timeout for shard-4 deep-process files

facts-anti-loop.test.ts and ingest-capture.test.ts were timing out in CI
shard 4 with "beforeEach/afterEach hook timed out" after the v0.41.16.0
master merge brought migration count to 99. When these files run deep in
a shard process that has already created ~20 PGLite engines, the WASM
cold-start + 95-migration replay legitimately exceeds bun's 5s default
hook timeout (observed 5.6s and 7.3s locally when reproducing).

Bun's --timeout=60000 from scripts/test-shard.sh covers TEST timeouts
but NOT hook timeouts; those default to 5s and must be set per-hook via
the optional 2nd arg to beforeAll/afterAll.

Reproduced locally by running the first 21 shard-4 files via
  head -21 /tmp/shard4-list.txt | xargs bun test
  → 179 pass, 2 fail (both with hook-timeout error)

After fix:
  → 198 pass, 0 fail (the 4 anti-loop + 15 ingest-capture tests recover)

Full shard 4 with fix:  955 pass, 0 fail.
Full shard 5 with fix:  1261 pass, 0 fail.

Also added a defensive diagnostic to the two put_page tests: if
facts_backstop is missing in the response payload, throw with the full
payload + isError so future failures surface the actual handler error
instead of a bare "expected {...} got undefined" assertion. No-op when
the test passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@time-attack time-attack mentioned this pull request Jul 28, 2026
Closed
@garrytan garrytan mentioned this pull request Sep 29, 2026
Merged
garrytan added a commit that referenced this pull request Oct 16, 2026
@garrytan @capy-ai
belief ledger lane B: fix review #5, #18, #25, #31, #32, #34, #35, #40,
947f24f 
#43 with regression tests

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@changergosum @garrytan