Repository navigation
ml: resume equivalence proven to the bit + portable host IO shim - #13168
Merged
Merged
Conversation
The resume-equivalence lane's stated baseline is master+portable-prof_now. On bare origin/master the profile test dies with SIGSYS on macOS (the raw Linux svc in prof_now), which fails test-report and blocks seal --save on this host, so the two ml files are taken byte-for-byte from lane/meter-repair (c9bab00 + e52263f). Content-identical bytes: whichever of the two branches lands first, the other's merge of these files is a no-op. Measured here: train 37/37 (both profile tests pass), eval 10/10 re-run below. Closes #13132 phi^2 + 1/phi^2 = 3 | TRINITY
…inery (lane P) The resumable .state checkpoint now carries an FNV-style digest of its whole payload (every parameter, LAMB m and v bit, in file order) in the header's reserved bytes at offset 52 -- written zero and never validated before. The layout is otherwise unchanged: same 64-byte header, same magic, same version 1, same payload offsets. tr_load_state recomputes the digest over what it decoded and refuses a mismatched checkpoint with RC_RESUME, so one flipped payload byte is refused instead of silently resuming into corrupted weights. tr_state_digest_chunk exposes the loader's fold so tests can run the same check against buffers; the_state_payload_digest_catches_a_flipped_byte covers header-digest consistency, flipped-byte detection and chunked-fold order (38/38 here). ckpt_dir_tmp (tests only, also declared in eval_hslm for the spliced tr_path) redirects checkpoint files into /tmp itself: a bare sandbox has no /tmp/hslm and host_open cannot create directories. The resume equivalence helpers (tr_resume_*) and tr_state_write_broken (the corrupted/wrong-version writer) land with this commit; the three file-level resume tests themselves ride the lane/resume-equivalence-io-tests branch because the host IO shim they are the first to exercise does not compile against zig 0.16 on macOS (the production trainer build fails there too, on master, before this change). Measured here (macOS, stable t27c, zig 0.16.0): t27c gen specs/ml/train_hslm.t27 -> exit 0 t27c test-report specs/ml/train_hslm.t27 -> 38/38, 0 vacuous t27c test-report specs/ml/eval_hslm.t27 -> 10/10 t27c fmt specs/ml/train_hslm.t27 -> clean Closes #13132 phi^2 + 1/phi^2 = 3 | TRINITY
…n, version) resume_then_continue_matches_the_control_to_the_bit: one step, the REAL tr_write_state, a reset to what a fresh process sees (the same tr_init path tr_main runs before tr_load_state), the REAL tr_load_state, then the same second step -- the full-state digest (params + LAMB m + v + rng + beta powers + loss) must equal the control's to the bit. Bounded by: same backend (speed 2), CPU one thread, fixed windows and lr, same seed; stated in the spec before the result, no tolerance. a_corrupted_checkpoint_is_refused / a_wrong_version_checkpoint_is_refused: the same broken-writer at the same step with only the defect varied (the A/B control), asserting RC_OK clean and RC_RESUME refused. NOT RUN YET, ON PURPOSE: these tests are the first in the corpus to exercise the emitted host IO shim, which does not compile against zig 0.16 on macOS (std.posix.system.open resolves to variadic std.c.open; read/write return isize vs the shim's usize). The production build fails the same way on master: t27c compile --backend ml specs/ml/train_hslm.t27 -> zig build-exe failed, before this lane touched anything. On Linux the shim compiles (CI installs zig there); this branch expects 41/41 in a Linux environment and stays BLOCKED on macOS until the compiler lane patches HOST_IO_ZIG in bootstrap/src/compiler.rs (about three casts, foreign-budget-eligible, owner call). Closes #13132 phi^2 + 1/phi^2 = 3 | TRINITY
…cast + openat zig 0.16 on macOS resolves posix.system.open to the variadic std.c.open and posix read/write/close disagree with the shim's usize signatures, so every emitted test binary that touches host IO failed build-exe on this host (train_hslm's resume tests are the first to exercise the shim; the production trainer build failed the same way on master before this lane). The emitted shim now branches on the OS tag: macOS reads/writes/closes through std.c with explicit @intcast on fd and byte-count widths, and opens through posix.openat(AT.FDCWD, std.mem.span(path), ...) with a catch-to--1; every other target keeps the posix.system path unchanged. Foreign diff: bootstrap/src/compiler.rs +4/-7, net negative, self-balancing. FROZEN_HASH resealed to the patched compiler.rs (sha256 f32f2982..., verified against the staged tree; build.rs reads the first token). Closes #13132 phi^2 + 1/phi^2 = 3 | TRINITY
Contributor
Contributor
PR DashboardGenerated at: 2026-11-03 13:27:00 UTC
Summary
Seal Status
|
This was referenced Nov 3, 2026
Merged
This was referenced Nov 15, 2026
Master's shim (evolved via #13625) is CI-proven on both native runners. The lane's macOS-branch portability layer is superseded; lane's resume-equivalence specs and tests merged cleanly and are untouched. Lane owner may re-apply portability branches if tests demand them.
Collaborator
Author
|
Merged origin/master into this lane to clear DIRTY. One real conflict in bootstrap/src/compiler.rs (embedded Zig host-IO shim): resolved to master's version — it has evolved via #13625 and is CI-proven on both native runners; the lane's macOS-branch layer is superseded for now. Your resume-equivalence specs/tests merged cleanly and are untouched. If native tests show the macOS branches are still needed, re-apply them in a follow-up commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The battery: control vs restart, to the bit
resume_then_continue_matches_the_control_to_the_bit(train_hslm.t27): onestep, the REAL tr_write_state, a reset to what a fresh process sees (the same
tr_init path tr_main runs before tr_load_state), the REAL tr_load_state, then
the same second step - the full-state digest (params + LAMB m + v + rng + beta
powers + loss) must equal the control's to the bit.
Bounding, stated in the spec before the result: same backend (speed 2),
CPU one thread, fixed windows and lr, same seed 7. No tolerance.
Corrupted / version refusals: the same broken-writer at the same step with
only the defect varied (the A/B control) -
a_corrupted_checkpoint_is_refusedand
a_wrong_version_checkpoint_is_refusedassert RC_OK clean and RC_RESUMErefused. The checkpoint payload now carries an FNV-style digest of its whole
payload at header offset 52; tr_load_state recomputes and refuses a mismatch
(RC_RESUME), so one flipped payload byte is refused instead of silently
resuming into corrupted weights.
the_state_payload_digest_catches_a_flipped_bytecovers header-digestconsistency, flipped-byte detection and chunked-fold order.
The shim fix (this branch's compiler commit)
zig 0.16 on macOS resolves posix.system.open to the variadic std.c.open and
posix read/write/close disagree with the shim's usize signatures - every
emitted binary touching host IO failed build-exe on this host; the production
trainer build failed the same way on master before this lane. The emitted
HOST_IO shim now branches on the OS tag: macOS reads/writes/closes through
std.c with explicit @intcast, opens through posix.openat(AT.FDCWD, ...) with
catch-to--1; other targets keep the posix.system path. Foreign diff
bootstrap/src/compiler.rs +4/-7, net negative, self-balancing. FROZEN_HASH
resealed (sha256 verified against the staged tree; build.rs reads the first
token).
Measured from this worktree (which binary, honestly)
train_hslm.t27: 41/41 EXECUTED, 0 vacuous - with t27c REBUILT fromthis branch's tree (
target/release/t27c, 0.5.2, HEAD 4716457). Thestable slot
~/.cache/t27/bin/t27c(built 2026-11-03 17:08) predates theshim fix and BLOCKS train on exactly the variadic-open error above - the
shim fix has to be inside the compiler that emits the test binary, which is
why the verification rebuild was needed.
eval_hslm.t27: 10/10 EXECUTED, 0 vacuous - with the stable slotbinary (2026-11-03 17:08), which carries the meter lane's use_resolve
var-splice (PR fix(use_resolve): splice module-level vars; portable prof_now - the meter runs again #13161, not yet merged at train time); eval's test build does
not exercise the host IO shim. The branch tree itself carries only the
prof_now carry commit from lane/meter-repair, byte-for-byte, so whichever
of the two branches lands first, the other's merge of these ml files is a
no-op.
(shim) both land; the two fixes are disjoint (use_resolve parsing vs
emitted host IO) and this PR does not rebase away any of fix(use_resolve): splice module-level vars; portable prof_now - the meter runs again #13161's content.
Commits
Closes #13132
phi^2 + 1/phi^2 = 3 | TRINITY
Note
Medium Risk
Touches compiler-emitted host I/O and checkpoint resume semantics; corrupted checkpoints are now rejected instead of silently loading, which is intentional but changes resume failure modes.
Overview
Adds bit-exact resume equivalence for HSLM training: resumable
.statecheckpoints now store an FNV-style payload digest at header offset 52;tr_load_staterecomputes it while decoding and returnsRC_RESUMEon version errors, short reads, or a single flipped payload byte. Newtr_state_write_broken,ckpt_dir_tmp(tests write to/tmp/ckpt_*when/tmp/hslmcannot be created), and resume helpers/tests prove control vs restart through the second training step with no tolerance.prof_nowis split at comptime: Linux keeps rawclock_gettimesyscalls; macOS usesstd.c.clock_gettimeso profiling no longer SIGSYS on Darwin.The compiler’s emitted
HOST_IOshim branches on OS: macOS usesposix.openatandstd.cread/write/close with explicit casts; Linux keeps the existingposix.systempath—fixing Zig 0.16 macOS build failures for binaries that use host I/O.eval_hslm.t27only mirrorsckpt_dir_tmpand comment updates for splicedtrain_hslmstate.Reviewed by Cursor Bugbot for commit f4cdd16. Bugbot is set up for automated code reviews on this repo. Configure here.