Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/REFEREE-PAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ manuscript; these are the measurements produced by the autonomous audit
| 1y | **`gft_signed_dot4` is incomplete, for two unrelated reasons (W974).** nextpnr hit a **600 s cap — reported `ABSENT`, not `FAIL`**, so nothing was proven unroutable; and a genuine **BSCAN chain/site mismatch** — `JTAG_CHAIN(3)` enabled while the wrapper wires `BSCAN4` | same | **"not measured" and "measured negative" are separate columns**; one red line is evidence about the design, the other about the time budget |
| 1z | **THE SILICON ANSWERED (W974).** On `xc7a200tfbg676-1`, IDCODE `0x3636093`, board `1:5`: **`Done = 1`** and **`0xa5a5a5a7`** read back through `USER2` — magic `0xA5A5A5A`, **`ok=1`, `beat=1`**. Spec → Verilog → yosys → nextpnr → FASM → frames → bitstream → load → read, every stage demonstrated | `die_verdict_w974.json`, derived by `verify_numbers.py` (270 checks) | **the path is proven; the claim is not.** What answered is a **classifier**, and its verdict is a liveness/integrity check — **not** a measurement of the number system. The format's operators are built and timed (1x) and **unread** |
| 1aa | **The first attempt failed on the default cable address, not the artefact (W974).** Identical bitstream: `no magic on any cable`, `index 0: ALL ZERO -- dead chain or no BSCANE2 in this bitstream`. The service defaults to the **three-cable bench** (`1:4/1:6/1:8`); this bench has **one** cable at **`1:5`** | same | **a read of all zeros must implicate the address before the artefact** — "wrong cable" and "no BSCANE2" are indistinguishable at the wire. A default encoding a physical layout expires **silently** |
| 1ab | **THE FORMAT'S SIGNED MAC FAILS ON SILICON (W975).** Same bench, control satisfied — a foreign bitstream forced `Done=0` first. `gft_sadd`: **3/3** in simulation, **`clauses=1111`, `ok=1`** on the die. `gft_signed_mac`: **2/2** in simulation, **`clauses=0011`, `ok=0`**, `beat=1` — loaded, alive, **answering incorrectly** | `operators_die_w975.json`, derived by `verify_numbers.py` (281 checks) | **timing is not the explanation**: closed at 9.14 MHz against a 2.21 MHz target, **4.1× margin** |
| 1ac | **The die check is STRONGER than the suite the spec ships (W975).** The MAC carries **two** simulation tests and its on-die check evaluates **four** clauses; the two that fail are **not covered in simulation** | same | **not "the simulator lies" — a coverage gap made visible by hardware**, found after synthesis, PnR, bitstream and a physical load. L4 is satisfied by *having* tests, not by their being as strong as the project's own stronger oracle |
| 2 | **TNF4 is the only sub-8-bit format measured that works under PTQ**: fp4 e2m1 and GF4 lose 70.5 / 71.3 pp | same | GF4 and fp4 are one lattice (identical to the digit on every seed) |
| 2a | **The advantage is a range, not a number.** Without retraining: **13–65 pp**. With max-scale QAT: **+0.19 / +0.89 pp**. With a **learned** scale: **+1.58 / +0.91 pp** — a better recipe *widened* it eightfold on MNIST | `qat_2026-08-20.json`, `lsq_2026-08-20.json` | epoch budgets differ (4 vs 3) across the two QAT runs; paired tests are within-run |
| 2c | **WITHDRAWN and replaced (W945).** The 4-bit comparison was not width-matched: TNF4 is physically **6 bits / 57 grid values**, fp4 e2m1 is 4 bits / 15. Against real 6-bit floats the advantage is **+0.11 (MNIST, t 2.2)** and **+0.17 (Fashion, t 1.2 — not significant)**, and with quantised activations on Fashion **fp6 e3m2 wins by 0.42** | `lsq_width_matched_2026-08-20.json` | four waves of headline numbers were measured against a competitor two bits narrower |
Expand Down
47 changes: 47 additions & 0 deletions research/arxiv_tnf/measurements/operators_die_w975.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"wave": "W975",
"part": "xc7a200tfbg676-1",
"idcode": "0x3636093",
"board": "1:5",
"control": {
"A1_wrong_part_done": 0,
"A1_expected": 0,
"note": "a foreign bitstream forced Done to 0 first, so Done=1 afterwards is meaningful"
},
"operators": {
"gft_sadd": {
"spec": "specs/ternary/gft_sadd.t27",
"LUT": 1312,
"CARRY4": 257,
"fmax_mhz": 18.24,
"clock": "slowclk",
"target_mhz": 8.85,
"sim_passed": 3,
"sim_failed": 0,
"die_word": "0xa5a5307f",
"die_design": 1,
"die_clauses": "1111",
"ok": 1,
"beat": 1,
"verdict": "PASS -- 4 of 4 clauses on the die"
},
"gft_signed_mac": {
"spec": "specs/ternary/gft_signed_mac.t27",
"LUT": 6466,
"CARRY4": 1237,
"fmax_mhz": 9.14,
"clock": "slowclk",
"target_mhz": 2.21,
"sim_passed": 2,
"sim_failed": 0,
"die_word": "0xa5a5334e",
"die_design": 13,
"die_clauses": "0011",
"ok": 0,
"beat": 1,
"verdict": "FAIL -- 2 of 4 clauses on the die, with the control satisfied",
"control_run": true
}
},
"finding": "gft_signed_mac passes every test it has in simulation (2/2) and satisfies only 2 of the 4 clauses its on-die check evaluates. The die check is STRONGER than the simulation suite the spec ships with -- the failing clauses are not covered in simulation, so this is a coverage gap made visible by hardware, not a simulator that lies. beat=1 and Done=1 with a wrong-part control at Done=0: the design is alive, loaded, and answering incorrectly. Timing closed with 4.1x margin (9.14 MHz against a 2.21 MHz target), so a timing violation is not the obvious explanation."
}
18 changes: 18 additions & 0 deletions research/arxiv_tnf/verify_numbers.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,5 +505,23 @@ def drop(d, task, fmt, key="formats", base="baseline"):
check("тот же битстрим, что в W973", dv["build"]["bitstream_bytes"], 9730834, tol=0)
check("Fmax совпадает с W973", dv["build"]["fmax_mhz"], 80.35, tol=0.01)

# W975: the format's operators read off the die, with the control satisfied.
od = rec("operators_die_w975.json")
if od:
print("\n== операторы формата на кристалле (W975)")
check("контроль: чужой битстрим уронил Done", od["control"]["A1_wrong_part_done"], 0, tol=0)
sa, mc = od["operators"]["gft_sadd"], od["operators"]["gft_signed_mac"]
check("sadd: клаузы на кристалле == 1111", sa["die_clauses"] == "1111", True, tol=0)
check("sadd: ok", sa["ok"], 1, tol=0)
check("sadd: симуляция без падений", sa["sim_failed"], 0, tol=0)
check("mac: клаузы на кристалле == 0011", mc["die_clauses"] == "0011", True, tol=0)
check("mac: ok", mc["ok"], 0, tol=0)
check("mac: жив (beat)", mc["beat"], 1, tol=0)
check("mac: симуляция без падений", mc["sim_failed"], 0, tol=0)
check("mac: тестов в симуляции меньше, чем клауз на кристалле",
mc["sim_passed"] < 4, True, tol=0)
check("mac: запас по частоте, раз", mc["fmax_mhz"] / mc["target_mhz"], 4.13, tol=0.02)
check("контроль прогонялся", mc["control_run"], True, tol=0)

print(f"\n ИТОГ: сошлось {ok}, расхождений {bad}, пропущено блоков {skip}")
sys.exit(1 if bad else 0)
Loading