Reduce H25 side-verification time by 29% with exact native GF(2)

research/gf2-side-verification/CODE-SOURCES.json

@@ -0,0 +1,15 @@
+{
+  "check_artifacts.py": "472211cc36381b6ec097f19cd98a7dd04d3d3158b5917acd7f2bf37ec5d144ac",
+  "compare_side_verifier.py": "6e507bb0c7e00d090a60289fddb6c55e90c946d0850c42c3c5e3e15c27604a68",
+  "finite_controls.py": "df989a831aac43cc1af95169dad8c64a7548760e6e1f96976da996703f2c8f7a",
+  "full_B.py": "db552a7a9561b01289dff84fa15b53e095ec4740b7bb94a8b88d118d79822c90",
+  "full_C.py": "82bc9c2bf905fa8fa05f327c68ed3f021c7ba20d4d8e6d35accbd041bd706d44",
+  "guard.py": "7540f8cc4ae4d12ed86c844ef0837a74184d3ada653979608c295c09f200a751",
+  "native_backend.py": "e473c38245c3668715b5a78dfe980b2fe3ed0d0c9b8c97c666fbae6ecf1fffd5",
+  "native_gf2.c": "e9c3c9194a08718bd1575c1cf08c6c6772d3b644c2e274c25a84a1f9249d06cc",
+  "profile_side_verifier.py": "9c994378171c20a91539418d93cb8387e1daa93465d6862beefea90e12999c2a",
+  "replay.py": "b121f83fd6e451d56a3095c611a7b0a2e0ccd24e7dd94447ecec37b7bce9408c",
+  "source/complex_circuit.py": "d623617f7c5cb3189580883a8ab30c0bd3832688ae3f1f0c1f881bbbbff8e0c1",
+  "source/complex_network.py": "0da395e67ac04fa213494e71012c1b19b544596d34cdce5a3b172b1fb6af5f1b",
+  "verify.py": "5983283dab626963a5c69e78a1cdf69d1d195f4d7dda38c3309be3c8588c4b4a"
+}

research/gf2-side-verification/README.md

@@ -0,0 +1,161 @@
+# Exact native GF(2) for the frozen H25 side verifier
+
+The complete frozen H=25 **complex-side verification segment** runs in
+**24.807329 rather than 35.075541 seconds median wall time (29.2746% less)**
+and **24.76 rather than 35.01 seconds median CPU (29.2773% less)** on the
+recorded Linux host. All six fresh cold processes retain identical complete
+results, structural/basis/role witnesses, every Gram-rank observation and every
+ordered complement input/output. Compilation, linking, loading and every added
+finite/binding/fallback check are paid in each candidate command. Both arms pay
+the same complete witness serialization and complement-stream hashing.
+
+This is a scoped verification implementation improvement. It is **not the full
+complex-network certificate**, a PR5864 speedup, a hosted performance factor,
+a new mathematical construction, or a kappa/selected-frontier claim. Percentages
+from these different workloads cannot be added. The further PR5864 target is
+not established by this package.
+
+## Frozen workload and change
+
+Canonical base is `de5ec2e7a6074068452f45b1a191b5c72832df4a`. The unchanged
+caller `scripts/complex_network.py:certificate` fixes H=25. The isolated scope
+constructs `ComplexSideCircuit(25)`, initializes `Checks`, compiles physical
+roles and performs **all** of `verify_map`, `verify_labels` and
+`verify_role_frames`, in the caller's original order. No parameter is searched
+and no source instruction is deleted or weakened. The two complete original
+source files are copied byte-for-byte and pinned in SOURCES.json.
+
+The opt-in C backend changes only `Label.complement_in` and
+`Label.nondegenerate`. It uses packed unsigned 64-bit GF(2) elimination and
+Gram construction. Checked input widths and augmentation bounds precede all
+shifts and array accesses. Unsupported widths, negative/nonexact values and
+container cases use the original Python path rather than truncating. The
+complete H25 workload takes 238,685 native complements and 59,959 native Gram
+calls, with zero fallback. The reference `echelon`, `reduce`, `contains`, source
+construction and all verification predicates remain active.
+
+The complement kernel preserves the source's pivot selection, all-row
+elimination, insertion order and returned-vector order. The Gram path retains
+the exact GF(2) field and all packed rows/ranks. This is unrelated to using a
+modular lower bound as an exact rational-rank certificate.
+
+## Complete identity and qualification bill
+
+Each baseline and candidate command recomputes these source-pinned identities:
+
+| Observation | SHA-256 or count |
+| --- | --- |
+| Full result | `342e99212b12e518902ae9598613f5484113bacee8747c27a13b58a4b1d3986c` |
+| Full structural/basis/role witness | `4b170c1dbfc33c7cb141a2284c6fd69eac7b6d100971d3e9ee625502b5893f82` |
+| Ordered Gram-rank transcript | `85006ca332bbc57c2ba9fd06f61a0281d4cee600c9f3b449342379bf2aedec8f` |
+| Rank calls / distinct matrices | 59,959 / 252 |
+| Ordered complement stream | `7dec2bf33303c37e1c699609e83768d61df7e19104d34a26a148f66d5ccc4838` |
+| Complement calls / canonical bytes | 238,685 / 55,839,507 |
+
+The complete witness is 58,143,622 canonical bytes; the rank transcript is
+2,674,272 bytes. Each timed arm writes the same full bytes. One compressed
+canonical copy is retained under `evidence/canonical/`; its decompressed bytes
+and hashes are authoritative, while gzip hashes describe transport only. All
+raw per-run artifacts remain in the donor's checkpoint, with source/evidence
+receipts retained here. No witness or complement was sampled away. The
+complement stream is hashed in constant memory, including call index, ordered
+input vectors, exact list/None output type and every output vector.
+
+New finite controls include 5,329 exhaustive ordered small complement pairs,
+585 small Gram cases, an independent dense-list elimination oracle, exact
+ordered outputs, 64/65-bit and augmented-width boundaries, unsupported
+type/iterator/fallback behavior, source/live builtin/helper/class/dispatch
+drift rejection and zero-side-effect native-proxy rejection. Normal and
+optimized finite modes retain the same explicit checks. Complete measurements
+require normal assertion mode, preserving every assertion in the unchanged
+source; optimized full verification is deliberately rejected.
+
+The initial profile showed that rank alone was too small to fund construction;
+the measured wider boundary was complement/Gram work. Its instrumented 78-second
+outer diagnostic is not a performance benchmark. Independent static review
+found two gaps in v1: resolved helper freshness and missing complete complement
+observations. V2 fixes both. Every v1 source/finite/baseline record is retained;
+no v1 candidate complete timing was performed. One explicitly declared literal
+v2 anchor constructed the new complement identity before the fixed timing
+sequence; the literal digest was then frozen into **both** arms. Candidate mode
+requires this expected digest before creating output or building native code.
+
+## Fixed cold comparison
+
+The predeclared order is B,C,C,B,B,C. All six science identities match. GNU
+time includes compiler-child CPU. Every process starts with fresh mutable
+circuit state, labels, verdicts, transcripts and native build; no answer/PASS
+cache crosses a process or backend. The same source snapshots retain authenticated bytes/modes, with live method
+environment guards under the declared serial stable-runtime contract.
+
+| Complete side segment | Median wall seconds | Median CPU seconds |
+| --- | ---: | ---: |
+| Unchanged Python baseline | 35.075541058 | 35.01 |
+| Native complement/Gram candidate | 24.807329345 | 24.76 |
+| Absolute saving | 10.268211713 | 10.25 |
+| Reduction | 29.2746% | 29.2773% |
+| Maximum within-arm spread | 1.115459876 | 1.10 |
+
+Both savings exceed 15%, 0.50 seconds and each arm's repeat spread. The first
+pair passed the same declared screen before the remaining fixed samples were
+run. No sample was dropped, no failed timing was replaced, and no automated
+retry was made. See [the complete gate](evidence/MATRIX-COMPLETE-GATE.json),
+[declared plan](evidence/MATRIX-PLAN.json), and retained raw receipts.
+
+## Reproduction
+
+Tested on Linux x86-64, CPython 3.12.3 and GCC 13.3. Existing `cc` and Python
+development headers are required. No third-party dependency is installed and
+no compiled binary is committed. This uses the tested CPython 3.12 C ABI;
+abi3, free-threaded and subinterpreter support are not claimed.
+
+Fresh registered finite/source/native checks, about two seconds per mode here:
+
+```sh
+python3.12 -B scripts/research_package.py --package research/gf2-side-verification
+```
+
+One complete guarded candidate replay, with a new external evidence directory:
+
+```sh
+python3.12 -B research/gf2-side-verification/replay.py C /tmp/gf2-side-new-evidence
+```
+
+Use arm B for the complete unchanged baseline. The portable guard applies one
+serial runner, 180 seconds per command, 1 GiB sampled process-group RSS and
+zero automatic retries; the recorded round used 600 seconds initial feasibility
+and a 1,500-second shared cap. New evidence directories prevent overwriting
+retained runs. Parent source-inventory/transport checks are separate from the
+timed cold scientific process, as documented by the raw guard receipts.
+Metadata, committed bytes/modes, export, readonly replay and exact-head hosted
+correctness CI are publication checks, not hosted performance evidence.
+
+## Credit
+
+Original complex-side source/caller and all retained CrocSwap construction,
+verification and contributor notices remain unchanged. Ali Romman and the
+retained predecessor authors are credited through the source history. Existing
+GMP/FLINT/native precedents [PR5168](https://github.com/CrocSwap/integer-mult-bounds/pull/5168),
+[PR5382](https://github.com/CrocSwap/integer-mult-bounds/pull/5382),
+[PR2853](https://github.com/CrocSwap/integer-mult-bounds/pull/2853) and
+[PR3143](https://github.com/CrocSwap/integer-mult-bounds/pull/3143) are adjacent
+engineering precedents; this implementation adds no dependency or novel linear
+algebra claim. The inherited Ali Romman PR5761 routing-test repair remains
+unchanged and separately credited.
+
+Prepared by Chris R / Clear1nsight with substantial OpenAI Codex implementation,
+user-requested GPT-6-astra xhigh design and separate static review. Only the
+parent performed compiler/numerical execution, serially. Apache-2.0. Source
+translation, finite independent comparisons and complete hashes are assisted
+engineering evidence, not human or proof-kernel acceptance of a theorem.
+
+## Open next questions
+
+The smallest useful follow-up is a bounded profile of the complete original
+complex-network caller before attempting any wider integration or claiming a
+whole-certificate factor. Preserve the exact field, every additional stage and
+its independent checks. That caller was not replayed here; its cost and resource
+requirements remain unpriced. No shared helper or selected construction is
+changed by this opt-in package. A broader matrix basis kernel would also need
+canonical pivot/sign/vector and full witness contracts, not merely equal rank.
+See [structured questions](TASK.json) and the retained independent source review.

research/gf2-side-verification/SOURCES.json

@@ -0,0 +1,52 @@
+{
+  "schema_version": 1,
+  "base_commit": "de5ec2e7a6074068452f45b1a191b5c72832df4a",
+  "files": [
+    {
+      "path": "docs/research/agent-task-catalog.json",
+      "usage": "provenance",
+      "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec",
+      "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81"
+    },
+    {
+      "path": "certificates/selected-result.json",
+      "usage": "provenance",
+      "git_blob": "581f8d87008354eb5b869f7ed9de1317ad837ce8",
+      "sha256": "2a0220e866417774606e0f991e41b9c1d26b1f43b97857238cf478dbc7d330b5"
+    },
+    {
+      "path": "scripts/complex_circuit.py",
+      "usage": "provenance",
+      "git_blob": "9e83a1065cd5b5f337b39de58d61bac335feb7ce",
+      "sha256": "d623617f7c5cb3189580883a8ab30c0bd3832688ae3f1f0c1f881bbbbff8e0c1"
+    },
+    {
+      "path": "scripts/complex_network.py",
+      "usage": "provenance",
+      "git_blob": "af9bb6d7b31f3006189761d99bdd919da84656a8",
+      "sha256": "0da395e67ac04fa213494e71012c1b19b544596d34cdce5a3b172b1fb6af5f1b"
+    },
+    {
+      "path": "research/gf2-side-verification/source/complex_circuit.py",
+      "usage": "input",
+      "git_blob": "9e83a1065cd5b5f337b39de58d61bac335feb7ce",
+      "sha256": "d623617f7c5cb3189580883a8ab30c0bd3832688ae3f1f0c1f881bbbbff8e0c1",
+      "pin": "head:b47911b3054242b95d5e0de259c7e6ff05942ff3:research/gf2-side-verification/source/complex_circuit.py"
+    },
+    {
+      "path": "research/gf2-side-verification/source/complex_network.py",
+      "usage": "input",
+      "git_blob": "af9bb6d7b31f3006189761d99bdd919da84656a8",
+      "sha256": "0da395e67ac04fa213494e71012c1b19b544596d34cdce5a3b172b1fb6af5f1b",
+      "pin": "head:b47911b3054242b95d5e0de259c7e6ff05942ff3:research/gf2-side-verification/source/complex_network.py"
+    }
+  ],
+  "predecessors": [
+    5168,
+    5382,
+    2853,
+    3143
+  ],
+  "native_source_sha256": "e9c3c9194a08718bd1575c1cf08c6c6772d3b644c2e274c25a84a1f9249d06cc",
+  "scope": "No fullnetwork/PR5864/kappa/selected-frontier claim"
+}

research/gf2-side-verification/TASK.json

@@ -0,0 +1,60 @@
+{
+  "schema_version": 1,
+  "task_id": "replay-runtime",
+  "original_card": {
+    "id": "replay-runtime",
+    "question": "Can one measured replay bottleneck be sped up without dropping coverage?",
+    "starting_points": [
+      "scripts/run_isolated_tests.py",
+      "scripts/verify_replay_coverage.py",
+      "certificates/selected-result.json"
+    ],
+    "methods": [
+      "Profile one isolated replay and optimize the measured bottleneck.",
+      "Design source-bound reuse with explicit dependency invalidation and negative controls."
+    ],
+    "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.",
+    "first_step": {
+      "command": [
+        "python3",
+        "-B",
+        "scripts/research_selected_baseline.py"
+      ],
+      "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
+      "inspect": "Resolve the current selected review, separate complex/ordinary BIT/admitted BIT/outer budgets, and compare the card against live handoffs before starting. Historical ceilings require exact source/model equivalence.",
+      "scope": "Source-binding orientation only; not a scientific replay or all-size proof. Materialize selected inputs if this checkout is sparse."
+    },
+    "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.",
+    "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
+    "allowed_variations": [
+      "Profile one stage or design one source-bound cache boundary."
+    ],
+    "scope_change_triggers": [
+      "Skipping coverage or comparing different fixtures to claim speedup."
+    ],
+    "overlap_terms": [
+      "replay runtime",
+      "cache invalidation",
+      "isolated tests"
+    ]
+  },
+  "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
+  "author": "Chris R / Clear1nsight",
+  "base_commit": "de5ec2e7a6074068452f45b1a191b5c72832df4a",
+  "scope_status": "refined",
+  "scope_change_reason": "Donor requested all four C verification opportunities and at least15% improvement. This separate frozenH25side workload passes29.27%complete wallCPU afterallnewbills; noPR5864/fullnetwork/kappa claim.",
+  "actual_scope": "frozenH25/complete-side-verification/native-GF2-complement-Gram",
+  "stop_rule": "One bounded source-pinned profile, fixed finite controls, complete anchor then BCCBBC. Preserve all samples/drafts and no automatic retries; stop on field/witness/source/resource/noise mismatch.",
+  "time_budget_minutes": 60,
+  "outcome": "completed",
+  "evidence": "README.md and evidence/MATRIX-COMPLETE-GATE.json; allfourwitness identities, finite normalO, exact source bytes and complete cold resource receipts.",
+  "ai_assistance": "Substantial OpenAI Codex implementation; user-requested GPT-6-astra xhigh design and separate static review. One parent serial numerical runner; no human/proof-kernel acceptance.",
+  "open_questions": [
+    {
+      "question": "Does this native side kernel improve the complete original complex-network certificate after every other stage and control is priced?",
+      "why_it_matters": "The measured29.27%scope is the entire side-verification segment, not the full network orPR5864.",
+      "first_step": "One unchanged complete-caller source/resource profile before proposing integration or another timing round.",
+      "blocked_by": "Full caller dependencies/costs/resources unpriced; no whole-network replay performed."
+    }
+  ]
+}

research/gf2-side-verification/VALIDATION.json

@@ -0,0 +1,20 @@
+{
+  "status": "PASS_LOCAL_COMPLETE_SIDE_15_PERCENT_AND_NOISE_GATES",
+  "normal_optimized_finite": "PASS",
+  "finite_exhaustive_ordered_complements": 5329,
+  "finite_small_Gram_cases": 585,
+  "complete_cold_processes": 6,
+  "complete_witnesses_all_equal": true,
+  "ordered_complement_calls": 238685,
+  "ordered_complement_bytes": 55839507,
+  "rank_calls": 59959,
+  "distinct_rank_matrices": 252,
+  "wall_reduction_percent": 29.27456399324708,
+  "CPU_reduction_percent": 29.2773493287632,
+  "same_measured_scientificcore_bytes": true,
+  "readonly_export_full_candidate": "PASS_FULL_CANDIDATE_AND_FINITE_NORMAL_OPTIMIZED_SOURCE_MODES_UNCHANGED",
+  "metadata_and_committed_bytes": "PASS_105_FILES_BEFORE_TRANSPORT_EVIDENCE_ADDITION",
+  "hosted_correctness": "UNCLAIMED_SEE_EXACT_HEAD_PR_RUN",
+  "primary_PR5864_target": "UNMET_BY_THIS_SEPARATE_SCOPE",
+  "full_complex_network_caller": "UNRUN"
+}

research/gf2-side-verification/VERIFY.json

@@ -0,0 +1,40 @@
+{
+  "schema_version": 1,
+  "status": "ready",
+  "python": "3.12",
+  "requirements": [],
+  "inputs": [
+    "research/gf2-side-verification/source/complex_circuit.py",
+    "research/gf2-side-verification/source/complex_network.py"
+  ],
+  "commands": [
+    {
+      "argv": [
+        "{python}",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 30,
+      "checks": "Fresh cold native independent dense/source/binding/fallback finite controls; no full or timing run."
+    },
+    {
+      "argv": [
+        "{python}",
+        "-O",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 30,
+      "checks": "Same explicit finite controls in optimized mode; complete source assertions are never disabled."
+    },
+    {
+      "argv": [
+        "{python}",
+        "-B",
+        "{package}/check_artifacts.py"
+      ],
+      "timeout_seconds": 30,
+      "checks": "Separate byte-transport check: complete decompressed canonical witness and ordered Gram-rank transcript; no benchmark or mathematical acceptance."
+    }
+  ]
+}

research/gf2-side-verification/check_artifacts.py

@@ -0,0 +1,19 @@
+"""Byte-transport check of the complete canonical witness, separate from timing."""
+import gzip
+from hashlib import sha256
+import json
+from pathlib import Path
+
+root = Path(__file__).resolve().parent / 'evidence' / 'canonical'
+manifest = json.loads((root / 'MANIFEST.json').read_text())
+for name, row in manifest.items():
+    path = root / (name + '.gz')
+    digest = sha256()
+    size = 0
+    with gzip.open(path, 'rb') as stream:
+        for block in iter(lambda: stream.read(1024 * 1024), b''):
+            size += len(block)
+            digest.update(block)
+    if size != row['canonical_bytes'] or digest.hexdigest() != row['canonical_sha256']:
+        raise ValueError('complete canonical artifact changed: ' + name)
+print('PASS_COMPLETE_CANONICAL_WITNESS_AND_RANK_TRANSPORT')