Repository navigation
Description
Goal
Build "IGLA-Coder": a small, deployable code-generation model that emits code in any programming language, trained on the IGLA RACE Rust stack (gHashTag/trios-trainer-igla) and wired into the t27 spec-first toolchain as a downstream consumer for the non-sealed language targets. Find the best training parameters with evidence (control ablation on a code loss), not assertion.
Why it matters
t27 already produces a parallel, compiler-verified multi-language corpus: one .t27 spec is deterministically emitted into the same module across many targets (gen/c and gen/verilog ~33 modules each; gen/numeric one catalog in 16 languages). This is a built-in spec->code and code<->code translation signal that no public small code model has. The current Python emitter is an explicit stub (bootstrap/src/codegen_python.rs) -- exactly the gap an ML coder fills.
Source of truth
- Plan: t27_coder_model_plan.md (workspace, to be committed under docs/)
- SOTA scan: research_code_llm_sota.md
- Trainer: gHashTag/trios-trainer-igla (champion BPB=2.2111, standard AdamW arm, not phi-canonical)
- Companion epic: Phase F / F2: BPB head-to-head -- phi-ladder vs format zoo (BLOCKED on real BPB pipeline + compute) #1021 (Phase F/F2 BPB head-to-head, blocked on real BPB pipeline + compute)
Deliverable
A pilot 50-200M coder with measured MultiPL-E pass@1 across >=6 languages, an evidence-backed best-parameter table (phi vs standard control), a t27-native executable eval, and an integration path t27c gen --backend ml --lang <X> for non-sealed targets.
Sub-tasks (checkboxes)
- P0 Decision and honesty foundation — [IGLA-Coder] P0 Decision and honesty foundation #1033 (CLOSED)
- P1 Tokenizer and data spine — [IGLA-Coder] P1 Tokenizer and data spine #1034 (partial: byte-level tokenizer decided, t27 parallel corpus extracted; general corpus + FIM packing pending)
- P2 Architecture port in the IGLA stack — [IGLA-Coder] P2 Architecture port in the IGLA stack #1035 (CLOSED: shipped as PR fix(seals): Update JonesPolynomial ring to 51, fix SacredPhysics hash #186)
- P3 phi-vs-standard control ablation on code — [IGLA-Coder] P3 phi-vs-standard control ablation on code #1036 (CLOSED: null result, all knobs audited)
- P4 Pilot pretraining at 50-200M — [IGLA-Coder] P4 Pilot pretraining at 50-200M #1037 (blocked on compute budget)
- P5 Multi-language evaluation harness — [IGLA-Coder] P5 Multi-language evaluation harness #1038 (in progress: resumable curriculum + n=32 Wilson power front-end; compile@1 still 0 at 493K params)
- P6 Scale-up to deployable 0.5B-1.5B (budget-gated) — [IGLA-Coder] P6 Scale-up to deployable 0.5B-1.5B (budget-gated) #1039 (blocked on explicit approval)
- P7 Low-bit / ternary track (parallel, optional) — [IGLA-Coder] P7 Low-bit / ternary track (parallel, optional) #1040 (open)
- P8 Integration into t27 and publication — [IGLA-Coder] P8 Integration into t27 and publication #1041 (pending)
Done when (acceptance)
A reproducible pilot coder beats a documented baseline on MultiPL-E and the t27-native eval; the best-parameter choice is backed by a control ablation with confidence intervals and Gamma_tip robustness; phi claims carry claim-status labels.
How to verify
MultiPL-E via the BigCode evaluation harness; t27-native eval runs emitted test/invariant/bench blocks as oracles; F2 dual-mediation CSV attributes BPB deltas; reproducibility capsule (seeds + frozen config + sha256).
Risks / blockers
Real-BPB code pipeline + compute + corpus access are the recurring prerequisites (same blockers as #1021). The deterministic Verilog/silicon emitter is never replaced by the ML model (L6 CEILING). Honesty mandate: the method survives, phi does not (yet); only phi^2 + phi^-2 = 3 is Verified.
Status update -- 2026-06-16
Now: P0-P3 complete. P0 baseline frozen; P1 corpus extracted; P2 architecture shipped; P3 closed with honest null result (no phi-specific optimizer benefit at sub-1M params). P5 evaluation harness advancing (resumable curriculum, n=32 eval bundle, Wilson power front-end).
Next: P4 pilot pretraining gated by compute budget; P5 capacity scan (hidden=192/256) to locate first non-zero compile@1.
Blocked: P4/P6 compute budget approval pending.
Links
Companion #1021; trios-trainer-igla README; The Stack v2; MultiPL-E; Qwen2.5-Coder, OpenCoder, Phi-1, BitNet b1.58 (see research_code_llm_sota.md).
Anchor: phi^2 + phi^-2 = 3
Activity
gHashTag commentedon Jun 1, 2026
Compute decision (2026-06-01): Railway control plane + rented GPU worker
Directive: use Railway. Honest constraint: Railway has no GPU (CPU/RAM/volumes/egress only - Railway Pricing, Railway docs). So:
tri deploy,tri race)GPU plan, cheapest-first:
Net: keep Railway where it is strong (orchestration + ledger + storage); GPU is a disposable rented worker. Start P0-P2 now at $0. Do not put the trainer on Railway.
Open input: confirm a per-month GPU spend ceiling for the pilot (e.g. cap $500).
gHashTag commentedon Jun 1, 2026
CORRECTION to the previous compute comment (2026-06-01)
The earlier "rented GPU worker" comment was wrong and is retracted. Audited facts:
cpu_train; Cargo.toml: "Portable Rust-only pipeline that runs on any CPU machine, Railway, Fly, or Docker."AdamWCpu(pure Rust, no CUDA), used by all trainers.gpufeature; the canonical path never enables it.railway upacross 3 seeds with Dockerfile + entrypoint.sh.Corrected decision: Railway CPU IS the trainer. No GPU.
railway up --service igla-seed-$s)AdamWCpu(phi-armwith_phi_defaults, standard armwith_params)Model size + token budget are CPU-bound, set empirically in P4 by timing a real
cpu_trainrun on a code corpus. P6 "scale-up" = more steps / wider hidden / more seeds on Railway CPU, NOT GPU. 0.5B-1.5B GPU-scale is out of scope unless the CPU pipeline is explicitly replaced.Open input: per-seed CPU wall-clock / Railway spend budget, and number of parallel seed services (champion used 3).
gHashTag commentedon Jun 1, 2026
IGLA-Coder P3 update -- full backprop verified, honest phi-vs-standard ablation
Code: branch
feat/igla-coder-v1ingHashTag/trios-trainer-igla(PR #186).What landed
gradcheck: analytic vs f64 numerical gradient -> PASS, 14 checks, 0 fails(emb, w1, w2, wq, wv, wo). Fixed a real transposed-matmul bug in the MLP-input
gradient found during this pass.
ablate: phi vs standard AdamW, N seeds, code-BPB + CI95. Equal learning rate;only beta1/weight_decay differ.
Honest ablation result (CPU, hidden=64, 3 seeds, each arm at its own best LR)
Verdict [Empirical fit]: phi-specificity for IGLA on code BPB is NOT supported
by this ablation. Tuned standard AdamW reaches lower BPB and is more stable; the
phi optimizer prior even loses on its converged seeds (disjoint CI95). Recorded as
a negative result, stated first. Only
phi^2 + phi^-2 = 3stays [Verified].Caveats: code BPB only (not pass@1); small n=3; single 300-step horizon; NOT
comparable to the tiny_shakespeare champion BPB=2.2111 (different data/vocab/
model/scale). Positional embeddings fixed (v-series scope). muP LR-transfer probe
inconclusive at this horizon -- reported as a limitation, no transfer claimed.
The method survives (verified backprop + ablation harness); the phi prior does not
(here).
gHashTag commentedon Jun 1, 2026
Loop update: pre-norm RMSNorm + trainable positional embeddings
Closing two audit weak-spots and re-running the control ablation with normalization.
Engineering (the durable asset):
pre-logit norm, each with a learnable per-channel gain. Full analytic backward.
normalization) and W2 (pos-emb not trained).
Verification:
gradcheck:checks=23 fails=0(analytic vs f64 central-difference, gateabs < 2e-3 OR rel < 5%). New families covered and passing: pos-emb, final-norm
gain, pre-attn gain, pre-mlp gain.
cargo fmt --all -- --checkclean;cargo clippy --all-targets -- -D warningsclean.
Empirical result (negative, stated first). hidden=64, heads=4, layers=2,
300 steps, 3 seeds (42/43/44), LR re-swept per arm, held-out code-BPB. Not
comparable to the tiny_shakespeare champion BPB=2.2111.
RMSNorm did not rescue the phi prior; the failure mode merely shifted from
divergence (no-norm, prior loop) to under-fitting/freezing (with-norm).
Attribution (new finding): the phi^-3 weight_decay anchor (~6x the standard
0.04) is the dominant failure cause under pre-norm -- the
phi_wdarm isnear-frozen, while the
phi_b1(momentum-only) arm still learns. Consistent withthe weight-decay-dominates-LR-transfer literature. The phi optimizer prior is
falsified at this scale by a fair equal-LR control. The method survives; phi
does not (yet). Anchor:
phi^2 + phi^-2 = 3.Refs #181.
gHashTag commentedon Jun 1, 2026
Loop+1 update -- decay isolation (A) + power (B) + F2 mediation (C)
Commit:
bd2b06eonfeat/igla-coder-v1. gradcheck 23/0 PASS; fmt + clippy clean; F2 CSV schema PASS.Headline (negative result first): phi-momentum does NOT beat tuned standard AdamW. At matched low weight decay the arms are a statistical tie; at canonical wd=0.04 the phi arm is +0.89 BPB worse. The method survives, phi does not (yet). Only
phi^2 + phi^-2 = 3is [Verified].A -- weight-decay isolation (hidden=64, 300 steps, lr=0.03, 3 seeds, bootstrap 95% CI)
Low wd helps both arms; phi-momentum closes to a tie, never overtakes. The phi^-3 wd magnitude (0.236), not phi-momentum, was the prior-loop killer.
B -- power run (hidden=128, 800 steps, lr=0.01, wd=0.02, 5 seeds)
phi_b1 4.313 (sd 0.457) vs standard 4.181 (sd 0.109); diff +0.132 BPB, 95% CI [-0.152, +0.548], p~0.56 -> TIE. Power floor at n=5 is ~0.66 BPB; phi_b1 is ~4x more variable (one unstable seed at 5.11).
C -- F2 long-form CSV + stratified-CDE mediation
2x2 factorial, mediators M1=momentum (beta1), M2=decay (wd). CSV is F2-contract compliant (W3C-PROV preamble +
INPUT STRATUM = canonicalbanner + ASCII snake_case headers); schema validator PASS. Pearl CDE + VanderWeele 4-way decomposition, bootstrap CI + Gamma_tip:Decomposition identity exact: CDE_dec + CDE_mom + interaction = TE = +2.858. The phi^-3 decay path is the dominant harmful path; the two phi knobs are sub-additive (negative interaction). Refs Pearl 2001, VanderWeele 2015, VanderWeele & Ding 2017, Zhao & Luo 2024.
Full report + reproducible scripts in
coder_ablation/. Three options (A more seeds/curves; B real F2 toolchain bridge + wd0 stratum; C lr x wd frontier + phi-lr arm) incoder_ablation/LOOP_REPORT.md.Refs #181
gHashTag commentedon Jun 2, 2026
IGLA-Coder Loop+2 -- power (A) + real F2 bridge (B) + lr x wd frontier (C)
CPU-only. Outcome = code_val_bpb (lower better), not comparable to the
tiny_shakespeare champion (different corpus). Negative result first: phi
still does not beat tuned standard AdamW. Only
phi^2 + phi^-2 = 3is[Verified]. The method survives, phi does not (yet).
A -- power (hidden=128, 1000 steps, lr=0.01, wd=0.02, 6 seeds, best-checkpoint)
Best-checkpoint readout collapses the final-step gap to zero. phi_b1's only
real difference is variance (max best-vs-final instability 1.46 BPB vs 0.33).
phi-momentum is less stable, not better. [Conj]
B -- real F2 Rust bridge + wd0 stratum
New
src/bin/f2_dual_mediation.rs(registered in Cargo.toml) reproduces thenumpy stand-in on the canonical CSV (CDE_decay +3.544 / Gamma 64.8;
CDE_momentum +0.888 / Gamma 8.2; TE +2.858; interaction -1.574; all
[Verified]). New wd0 stratum (weight_decay pinned to 0, Pearl
single-mediator CDE along non-decay paths):
Sign flip: phi-momentum's controlled direct effect is entirely conditional
on the weight-decay regime -- harmful at canonical wd=0.04, helpful at wd=0.
Real training uses non-zero wd, so the operative regime is the one where
phi-momentum hurts. 4 unit tests, all green.
C -- lr x wd frontier + phi-lr arm (hidden=64, 300 steps, 3 seeds)
phi-lr = 0.03 * phi^-1 = 0.0185410197. Frontier-reach test vs standard
(lr=0.03, wd=0.02, mean 3.775):
No phi-derived config reaches the standard frontier; all CIs above zero. The
MDL "one phi constant generates a competitive config" hypothesis is
rejected at this scale. [Efit -> falsified]
Verification
gradcheck 23/0;
cargo fmt --all -- --checkclean;cargo clippy --all-targets -- -D warningsclean;cargo test --release --bin f2_dual_mediation4/4 pass; F2 CSV schema PASS on canonical + wd0.Claim-status summary
wd0 CDE_momentum sign flip. These describe phi's regime-dependent / harmful
effects, not a benefit.
Refs #181
gHashTag commentedon Jun 2, 2026
IGLA-Coder Loop+3 -- third stratum (mom_std) + cross-stratum CDE stability
CPU-only. Outcome = code_val_bpb (lower better), not comparable to the
tiny_shakespeare champion (different corpus). Negative result first: phi
still does not beat tuned standard AdamW. Only
phi^2 + phi^-2 = 3is[Verified]. Method survives, phi does not (yet).
Audit decision: warmup0 dropped, mom_std added
The proposed warmup0 stratum is fictitious for this trainer --
igla_coderhas no LR warmup and no schedule (lr is constant). Pinning a non-existent
mediator would violate the f2-mediation-loop rule (a stratum must pin a real
mediator carrying >= 50% of the IE). Replaced with the honest symmetric
completion of the 2x2 factorial: mom_std pins momentum (beta1=0.9) and
varies decay (0.04 -> phi^-3), giving a Pearl CDE of decay along non-momentum
paths.
The three strata
Cross-stratum stability (the Loop+3 finding)
The two phi knobs behave categorically differently:
4 dp whether momentum is free or pinned, Gamma ~65 in both. No regime
rescues it.
the canonical wd=0.04 that real training uses.
This is the strongest causal statement the coder track supports, and it is a
statement about phi's liabilities and conditionality, not its benefits. The
NeurIPS 2024 warmup paper (arXiv:2410.23922) explains the mechanism: warmup
normally absorbs Adam's beta1 bias-correction shock and interacts with weight
decay; this trainer lacks warmup, so the phi^-1 momentum arm is the unstable
one measured in Loop+2.
Implementation
f2_dual_mediation: single-mediator branch generalised to detect whichmediator is pinned (decay-pinned -> momentum CDE; momentum-pinned -> decay
CDE). +1 unit test (5 total).
coder_ablation/emit_f2_mom_std.py,coder_ablation_f2_mom_std.csv.coder_ablation/stratum_compare_coder.py: cross-stratum comparator withstable_across_strataper mediator.validate_f2_csv.py: KNOWN_STRATA += mom_std.Verification
gradcheck 23/0;
cargo fmt --all -- --checkclean;cargo clippy --all-targets -- -D warningsclean;cargo test --release --bin f2_dual_mediation5/5 pass; F2 CSV schema PASS on canonical + wd0 + mom_std.Claim-status summary
mom_std decay CDE; both cross-stratum stability verdicts. All describe phi's
harmful or conditional effects, not a benefit.
frontier.
Refs #181
gHashTag commentedon Jun 7, 2026
IGLA-Coder Loop+9 -- iso-everything axis test (pure beta1 momentum, isolated)
CPU-only. Outcome = code_val_bpb (lower better), not comparable to the
tiny_shakespeare champion. Negative result first: phi still does not beat
tuned standard AdamW -- and the pure momentum prior turns out to be a complete
non-event. Only
phi^2 + phi^-2 = 3is [Verified]. Method survives, phi doesnot (yet).
Code: branch
feat/igla-coder-v1@430f71b.The question
Loop+8 (iso-lr) pinned the learning rate and the four momentum axes collapsed
to a ~0.14 BPB band with phi worst by +0.09. But each arm still carried its
OWN weight_decay, and the F2 mediation work had shown decay (
CDE_decay ~+3.5)dominates momentum (
CDE_momentum ~+0.9). So that residual phi gap could stillhave been a decay effect. Loop+9 removes the last confound: lr AND
weight_decay are both pinned to constants (lr=0.002, wd=0.04) for every arm;
only
beta1varies.Result (hidden=64, 800 steps, 3 seeds, lr=0.002 AND wd=0.04 pinned for all arms)
Provenance sha256[:16] =
bef01eab741f9162.Verdict [Efit]: phi-momentum NOT distinguishable from standard at
iso-everything -- CIs overlap, delta -0.027 BPB within noise, whole-band spread
0.030 BPB (about half a single arm's CI width). With both lr and decay
controlled, the choice of Adam
beta1in [0.37, 0.90] has no detectable effecton code BPB at sub-1M params;
phi^-1is one neutral point among several.The confound ladder (the actual finding)
-> a learning-rate sweep in disguise.
decay.
residual gap WAS the decay confound. The pure momentum prior is neutral.
The only operationally relevant phi liability the coder track ever measured is
the heavy
phi^-3weight decay (0.236, ~6x standard) -- confirmed here as thesource of the Loop+8 gap. The momentum and (Loop+8) lr-controlled effects are
non-events.
Verification
cargo build --release --bin igla_coderclean;gradcheck checks=23 fails=0 PASS; all 12 runsgen_rc=0valid BPB, no divergence.ablate_generator_axis_isoall.py,coder_generator_axis_isoall.csv,LOOP9_REPORT.md, andbuild_t27_parallel_corpus.py(closes the priorworkspace-only corpus reproducibility gap; corpus regenerated from t27
gen/,1,044,046 train tokens, vocab 263).
Claim-status delta
controlled. Neutral, not a benefit.
phi^-3weight_decay is the dominant harmful knob.Loop+10 options in
coder_ablation/LOOP9_REPORT.md: (A) publish the confoundladder as the methodology's headline honesty figure [P8.3]; (B) C-only
curriculum +
--resumeto first compile@1 > 0 [P4/P5 gate]; (C) symmetricbeta2/grad_clip sweep to close the per-knob phi audit.
Refs #181. Anchor: phi^2 + phi^-2 = 3.
gHashTag commentedon Jun 7, 2026
Loop+10 update (A + B + C) -- commit
4bbd5a9onfeat/igla-coder-v1User picked all three Loop+9 options. Negative-first, as always.
Headline: the per-knob phi-optimizer-prior audit is now CLOSED, and it is a null. Across every isolated AdamW knob the coder track can vary -- lr (P3/Loop+7/8), weight_decay (F2 mediation), beta1 (Loop+9), and now beta2 (Loop+10) -- phi structure is either neutral or harmful, never a clean benefit. Only
phi^2 + phi^-2 = 3is[Verified]. The method survives, phi does not (yet).(C) beta2 isolation -- no phi benefit. With lr (0.002), wd (0.04), AND beta1 (0.9) all pinned to standard, varying only the new
--beta2flag: phi-native0.966 (~1-phi^-7)tested -0.164 BPB vs the0.999null -- but a non-phi control sweep (0.98/0.95/0.92) beats the null by an identical margin (-0.146 to -0.166). It is a generic "0.999 is too high for a tiny noisy run" effect, not phi. Aggressive phi-native beta2 (0.618,0.382) is catastrophically worse (+2.3, +32 BPB). The control sweep is exactly the trap the critical-honesty discipline exists to catch.grad_clipreported INERT (printed in the prior but never applied inopt_step) -- not swept, not fabricated as a mediator.(A) confound ladder (figure + P8.3 write-up). Published the Loop+7->8->9 sequence as a single figure: the phi-standard delta collapses +0.566 -> +0.092 -> -0.027 BPB as the lr then the decay confound are each controlled. The phi advantage was a confound at every rung; honest endpoint is neutrality. Drop-in P8.3 subsection included in the report.
(B) first resumable C-only curriculum at hidden=128. New
--resumeflag warm-starts weights across short CPU sessions (honest: weights only, fresh optimizer state -- no bit-exact-optimizer claim). Lifted the 296K hidden=64 ceiling to 493K params, 12200 steps in 4x3000 resumable chunks. val BPB descended monotonically to a record-low 1.038 -- but compile@1 = 0/4 and pass@1 = 0/4. Doubled capacity bought a better likelihood model, not valid C. The capacity ceiling for compiling C sits above 493K params at this corpus/budget.Verification:
cargo buildOK;gradcheckchecks=23 fails=0 PASS (unchanged by new flags); 30/30 beta2 sweep runs returned valid BPB, no NaN. New files ASCII-clean.Full report:
coder_ablation/LOOP10_REPORT.md. Phase comments on #1036 (C+A) and #1038 (B).Three options for Loop+11
grad_clipintoopt_step(it is currently inert dead metadata), re-run gradcheck, then run the symmetric isolation sweep on the LAST AdamW knob. Medium cost.STOP. Pick A / B / C.
Anchor:
phi^2 + phi^-2 = 3.fix(ci): Coq admitted check covers all .v files; add allow-list mecha…
fix(rtl): R-SI-1 compliance — barrel shifters + Wallace tree multiplier
fix(build): resolve P0 build failure — Cyrillic purge + HeaderValue i…
fix(infra): P0 infrastructure hardening — duplicate workflows, Coq al…
fix(ci): Coq workflow rename + infrastructure cleanup
docs: Wave Loop 14 — Loop 9 report, spec foundation, competitive intel
feat(igla): Loop 10 — spec implementations, codegen fixes, security p…
docs: Wave Loop 14 — Loop 10 report, codegen fixes, spec implementati…
gHashTag commentedon Jun 17, 2026
Wave Loop 92 audit: closing long-running EPIC with no close criteria — work tracked in newer milestones.