Repository navigation
[PR] fix(storage): per-runtime SQLite engine artifacts + refuse-before-load Node floor (#3894, rebase of #3896) - #3963
Merged
Merged
Conversation
…e floor better-sqlite3 v13 declares engines: node >= 22, but the bootstrap pinned only version + sha512 and registered any copy that require() accepted. On Node 20 the win32-x64 prebuild faults inside process.dlopen (access violation) and the linux-x64 one faults on the first new Database() - uncatchable either way, so a bootstrap probe cannot defend against it. The crash loops the host process instead of degrading to legacy storage. The pin now carries the artifact's own Node major floor and runBootstrap() refuses BOTH the download and a cached-copy registration below it, returning unsupported-node with an explicit upgrade reason. The gate sits after the builtin/host probes (a host-installed better-sqlite3@12 on Node 20 is fine) and before the cache check (the cached 13.0.3 copy is the crash source). A cached copy is left on disk so it becomes usable again after a Node upgrade. No new config surface, no cooldown (a too-old Node is not a transient download failure). Fixes #3894. Refs #3341.
… the sidecar-open-error branch (#3894) Review fix for #3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case. Regression test pins the acp_status reason for the unsupported-node scenario.
…ne artifact better-sqlite3 v13 declares `engines: node >= 22` and compiles its prebuilds for N-API 10, while Node 20 only provides N-API 9. Loading that binding below its floor is undefined behavior, not a clean failure: the win32-x64 prebuild faults inside `process.dlopen` (access violation, uncatchable) and the linux-x64 one faults on the first `new Database()` (#3341, #3894). The self-bootstrap registered such a copy whenever it happened to be cached, which crash-looped the proxy child (1485 spawns, exit 0xC0000005) on Node 20. The previous revision of this branch refused the artifact below its floor and degraded to legacy JSON storage. That is the wrong answer: legacy storage is scheduled for removal, so "low Node" must stay on the unified lane with a lower artifact rather than fall off it. Pin one artifact per Node major band instead: * Node >= 22 -> better-sqlite3 13.0.3, unchanged: one tarball that carries its own eight prebuilds. * Node 20 -> better-sqlite3 12.9.0, the last 12.x line that publishes an ABI 115 binding. Its npm tarball is pure JS (no binding, no prebuilds); the native half is a separate ABI-keyed asset (`better-sqlite3-v12.9.0-node-v115-<target>.tar.gz`) that the bootstrap now fetches from the binary mirror and materializes as `prebuilds/<target>.node`. v12's `lib/database.js` accepts an injected `nativeBinding`, so the two halves compose without `bindings`, `prebuild-install` or any install script — the #2833 contract holds: pinned artifacts, sha512 verified on every download, no host build approval. A spec is eligible only when its Node floor AND (when it has one) its prebuild ABI match the running Node. A runtime no pinned artifact serves reports `unsupported-node` with the full table in the detail instead of registering a binding that would crash the process. Pinned sibling cache dirs are no longer pruned, so switching Node majors does not re-download an artifact this build already knows how to load. Verified on a real Node 20.19.5 (ABI 115): downloads 12.9.0 + its ABI 115 prebuild, binds through nativeBinding, runs DDL/DML/transactions, process survives. On Node 22.22.3 the same cache resolves to builtin node:sqlite and the v12 copy is left untouched. Tests: 5 new cases (JS-only artifact binds its ABI prebuild; a mismatched ABI is refused before any download; a self-contained artifact never touches the binary mirror; the prebuild extractor lands and fails loud). Both directions of the gate are asserted per #3617. Fixes #3894. Refs #3341.
The `test (windows-latest, 24, 3)` leg went red on `#3818 review: reap-shape records ... stay envelope-served across restart` with `EPERM, Permission denied` from `rmrf()` — not an assertion failure. Windows can still hold an ephemeral handle on `index.db`/`-wal` right after a store closes (Defender real-time scan, delayed close), and a bare `rmSync({recursive:true, force:true})` throws instead of waiting. Retry with the same `maxRetries`/`retryDelay` shape the other Windows-touching suites already use (`tests/export-jsonl.test.ts`, `tests/identity-registry.test.ts`), then swallow the residual error: the dir lives under `HERMETIC`, which the `after` hook clears anyway, so a leaked handle can only leak a temp dir — never fail an otherwise green run. Refs #3894.
…3894) Additive review commit (ai: vllm-qwen/qwen3.8-27b): the PINNED_VERSIONS prune-retention behavior introduced by the per-major artifact table had no test row - assert the other pinned version' dir survives a successful download while an unpinned legacy version is still removed.
The per-major band table from the previous commit only asked two questions -
the package's own engines floor and the ABI of the binary - and that leaves a
hole big enough to crash the process: better-sqlite3 v13 declares
`engines.node >= 22`, but its binding.gyp compiles with NAPI_VERSION=10, and
Node 22.0 through 22.12 ship N-API 9. Those releases satisfy the major floor,
pass every existing check, and then fault uncatchably inside process.dlopen
when the v13 prebuild is loaded. `minNodeMajor: 22` cannot express this: the
gap is in the N-API axis, not the major axis.
Serving is therefore decided on three axes now - engines floor, N-API version
the binding was compiled against, and the ABI of the binary we would register:
* `EngineBootstrapSpec.minNapiVersion` carries the artifact's NAPI_VERSION.
v13.0.3 declares 10. A runtime below it is refused with a reason naming
the gap instead of being handed an addon it cannot load.
* `EnginePrebuild.sha512Base64ByAbi` replaces the single-ABI
`nodeAbi`/`sha512Base64ByTarget` pair with an ABI-keyed digest table, so
v12.9.0 now serves both Node 20 (ABI 115) and the Node 22.0-22.12 band
(ABI 127). Eight new pins, taken from the npmmirror binary mirror.
* The cache directory is ABI-keyed (`better-sqlite3-12.9.0-abi115`): a
runtime that switches Node minors must fetch the binding that matches it
rather than inherit the previous one. The prune keeps every pinned ABI
variant, so switching back does not re-download.
`pinnedEngineVersionFor()` is exported as a pure selector so the whole band
matrix is testable without a download.
Verified end to end on real runtimes against the real registry: Node 22.0.0
(ABI 127, N-API 9) now downloads v12.9.0 plus its ABI 127 prebuild, opens a
database, runs DDL/DML and survives - the same binary that segfaults when the
v13 prebuild is loaded. Node 20.19.5 stays on the ABI 115 lane; Node 22.13+
and 22.14+ take the builtin or the v13 single-tarball lane as before.
Fixes #3894. Refs #3341.
#3853 landed three solver-backed verif tests without the availability gate the other verif suites carry, so on a runner with neither cvc5 nor z3 they fail instead of skipping - `no solver available (set VERIF_SOLVER_CMD / VERIF_PYTHON)`. That contradicts the module's own contract (AGENTS.md rule 14: an environment without a solver must skip, never go red) and it is what turned every `test` leg red on this head, on ubuntu and windows alike. Adds the same probe-then-skip the sibling suites use (verif-bug3769, verif-generic-append-stability, verif-symbolic-smoke) to bug3676, bug3691 and the induction step. With a solver present nothing changes; without one the three cases report skipped and the rest of the suite still runs. Verified locally: the 8 verif files report 5 pass / 10 skipped / 0 fail with no solver on PATH, and the probe returns null as expected.
ranxianglei
changed the title
Owner
Author
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。本 PR 是 #3963 ← #3896(同分支 rebase 续作,#3896 floor 48 已声明「Suggest closing this one in favor of #3963」),源缺陷 #3894;按失败签名(better-sqlite3 / Node floor / unsupported-node / bootstrap / N-API)检索 open+closed 的 issue 与 PR 无同症状同机制条目。相关但不同:#3341(同机制、已关、代码路径从未加门限)、#3788/#3823(优雅冷却降级,不崩)、#3959(darwin 真实 index.db 损坏,另一类)。 标签:bug / dsh / scope/storage / p1(崩溃循环 = 核心存储承诺失灵,p1 按 owner 修订规则不降档)。 开工前发现一个状态问题:PR head 开始处理,有进展会同步。 |
Contributor
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.674-master.74b894d # this exact build
npm install -g billion-context@master # always the newest merged build
|
Contributor
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4734-pr.3963.b78b86cThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3963.tgz
npm install -g packageInstall instructions are refreshed on each push. |
ranxianglei
added a commit
that referenced
this pull request
Oct 20, 2026
…#3963 follow-up) EngineBootstrapSpec gained the required minNodeMajor field when the node floor gate landed; the fake 13.0.3 tarball spec now declares 22.
ranxianglei
added a commit
that referenced
this pull request
Oct 20, 2026
ranxianglei
added a commit
that referenced
this pull request
Oct 20, 2026
…#3619 hermeticity) The forced path passes the update-policy gate, which fetches the remote policy document. The test's fetch stub rejected that URL and the count assertion (2) never included it, so the case only passed when some earlier state suppressed the fetch — #3963's new test files rebalanced the CI shards and exposed it. Mock an empty (all-defaults) policy doc and expect policy + packument + tarball = 3.
ranxianglei
added a commit
that referenced
this pull request
Oct 20, 2026
…#3619 hermeticity) The forced path passes the update-policy gate, which fetches the remote policy document. The test's fetch stub rejected that URL and the count assertion (2) never included it, so the case only passed when some earlier state suppressed the fetch — #3963's new test files rebalanced the CI shards and exposed it. Mock an empty (all-defaults) policy doc and expect policy + packument + tarball = 3.
ranxianglei
added a commit
that referenced
this pull request
Oct 20, 2026
[PR] docs: regenerate config reference after the engine-tiering seeds (#3963)
This was referenced Oct 20, 2026
Closed
Zhishulo
pushed a commit
to Zhishulo/billion-context
that referenced
this pull request
Oct 20, 2026
This was referenced Oct 20, 2026
Closed
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3894
Rebase of #3896 onto current master — all engineering by @zypNB123 (zypNB123/billion-context, branch
2026-10-20_node-floor-gate, 9 commits kept with original authorship); original PR conflicted with the storage PR wave (#3911/#3927/#3932-#3955) and this branch replays it cleanly on top (one CHANGELOG entry added).What
better-sqlite3-v12.9.0-node-v115-<target>.tar.gz) fetched from the npmmirror binary mirror (github release asset as fallback), sha512-pinned per ABI × target, bound through v12'snativeBindingoption — nobindingsresolver, no install script, the 彻底解决 dsh/pnpm 构建脚本门导致 build-blocked：SQLite 引擎自举（内置优先 → 运行时自下载 → 复用宿主） #2833 contract holds.unsupported-nodebefore any download or cache registration — loading such a binding faults uncatchably (process.dlopenaccess violation on win32-x64, firstnew Database()on linux-x64), which is exactly the Node 20 startup segfault via self-bootstrapped better-sqlite3@13 — the Node floor promised in #3341 was never implemented #3894 crash loop (1485 consecutive proxy-child deaths). The degrade reason now names the running Node/ABI and every pinned band instead of blaming the network.better-sqlite3-12.9.0-abi115vs-abi127) so switching Node majors does not re-download; pinned siblings survive the post-publish prune.Verification (rebased branch, local)
storage-engine-bootstrap43/43 (incl. the new floor-boundary matrix, prebuild mirror walk, per-ABI cache retention, unsupported-node refusal)storage-full-flip8/8,acp-status-storage-degraded3/3,storage-backend-knob4/4,storage-unified14/14,storage-reap-journal2/2,storage-archive-gc5/5,media-cas3/3, verif family greengen-config-docs check-coverageok; unused-exports cleanNot here