Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -589,4 +589,6 @@ MIT

**Wave 18 (2026-05-22):** Stochastic Rounding import — [`rings/ring-091-rust`](rings/ring-091-rust) lands the fourth honestly-authored Wave-11 crate (462 LOC, 19 tests). Implements stochastic rounding (`sr_round_f32_to_i32`, `sr_quantize_f32`, `sr_quantize_batch`) on top of a deterministic seedable `SplitMix64` PRNG (Vigna 2014; reference seed-0 value `0xE220A8397B1DCDAF` checked in test). Two statistical tests verify unbiasedness over 10 000 draws against a 3-sigma bound. `sr_quantize_phi_unbiased` is the **third cross-kernel anchor test** in the project after Wave 15's `mac_dot_phi_identity` and Wave 16's `cpu_phi_identity_integer_projection`. `#![no_std]`, zero deps, all 19 tests green on first run. See [COMPILE_STATUS](rings/COMPILE_STATUS.md).

**Wave 19 (2026-05-22):** Attention import — [`rings/ring-092-rust`](rings/ring-092-rust) lands the fifth honestly-authored Wave-11 crate (760 LOC, 28 tests). Mirrors `specs/nn/attention.t27` (SacredAttention) for the `no_std`-realizable subset: sacred constants (`NUM_HEADS=3`, `HEAD_DIM=81`, `EMBED_DIM=243`, `CONTEXT_LEN=81`, `ROPE_PAIRS=40`, `SACRED_GAMMA = phi^-3`, `SACRED_SCALE = 81^(-SACRED_GAMMA)`); `Trit` enum; primitives `ternary_matmul`, `add_residual`, `apply_softmax` (numerically stable max-subtract per head), `compute_scores` (Q.K^T with causal mask + sacred scaling), `weighted_values`, `cache_kv`. A private `exp_f64` (range-reduced Taylor series) makes softmax viable without libm. `attention_phi_identity_via_softmax_matmul` is the **fourth cross-kernel anchor test** in the project, routing `phi^2 + 1/phi^2 = 3` through softmax-style normalization and ternary matmul. RoPE table init (cos/sin) and the full `sacred_attention_kernel` orchestrator are out of scope (R5-HONEST). Wave-11 narrative claimed 847 LOC; honest measurement is 760 LOC. All 28 tests green on first run (Rust 1.83.0). See [COMPILE_STATUS](rings/COMPILE_STATUS.md).

**DOI:** [10.5281/zenodo.19456875](https://doi.org/10.5281/zenodo.19456875)
15 changes: 15 additions & 0 deletions docs/NOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

Last updated: 2026-05-22

## wave-19 -- Attention import: ring-092-rust (this PR, Closes #725)

- **NEW** (rings-only, additive): `rings/ring-092-rust/` lands with `Cargo.toml` + `src/lib.rs` (760 LOC) + `README.md`. Zero edits under `gen/`, `coq/`, `trios-coq/`, `proofs/`, `bootstrap/`, `specs/`, `conformance/`, `architecture/`, root `Cargo.toml`, or any other crate. Doc-only updates to `rings/COMPILE_STATUS.md`, `README.md` (Wave 19 footer), and this file.
- **What ring-092 actually does:** Faithful Rust mirror of the realizable subset of `specs/nn/attention.t27` (SacredAttention). (a) Sacred constants byte-for-byte: `NUM_HEADS=3`, `HEAD_DIM=81`, `EMBED_DIM=243`, `CONTEXT_LEN=81`, `ROPE_PAIRS=40`, `SACRED_GAMMA = phi^-3 ~= 0.2360679774997897`, `SACRED_SCALE = 81^(-SACRED_GAMMA) ~= 0.3543788557382518` (the spec calls for `pow(81, -SACRED_GAMMA)`; we embed the literal because `powf` is unavailable in `no_std` without libm, and add `attn_sacred_scale_matches_reference` to lock the value to 1e-6). (b) `Trit` balanced-ternary weight enum `{Neg, Zero, Pos}` with `value() -> i8`. (c) `ternary_matmul(input, weights, output, in_dim, out_dim)` -- matrix-vector product with ternary weights, identical algorithm to spec's `ternary_matmul`. (d) `add_residual(output, input)` -- in-place residual add, length-clamped. (e) `apply_softmax(scores, seq_len)` -- per-head softmax over a `NUM_HEADS * CONTEXT_LEN` buffer, max-subtract numerical stabilization. (f) `compute_scores(q, cache_k, position, seq_len, scores)` -- Q.K^T per head, multiplied by `SACRED_SCALE`, with a causal mask (positions `j > position` forced to zero). (g) `weighted_values(scores, cache_v, seq_len, concat)` -- softmax-weighted V sum. (h) `cache_kv(k_buffer, v_buffer, position, cache_k, cache_v)` -- KV cache store at offset `position * EMBED_DIM`. (i) `identity_witness()` for the universal anchor.
- **no_std exp:** softmax requires `exp`, which is unavailable in `no_std` without libm. The crate embeds a private `exp_f64` using range reduction (`exp(x) = (exp(x / 2^20))^(2^20)`) plus a 12-term Taylor series. Verified to better than 1e-9 across the working range against the standard library (`exp_negative_small`, `exp_negative_large`), with explicit underflow handling (`exp_underflow_returns_zero` at `x < -700`).
- **No new spec (L6 CEILING + L2 GENERATION):** every sacred constant, the per-head matmul shape, the causal mask convention, and the softmax+matmul structure are direct mirrors of `specs/nn/attention.t27`. No file under `specs/`, `coq/`, `proofs/`, `bootstrap/`, `gen/` is touched. The constants are duplicated, not edited.
- **Tests (28, all pass on first run on Rust 1.83.0):** sacred constants (`attn_num_heads_is_trinity`, `attn_head_dim_is_three_pow_four`, `attn_embed_dim_is_heads_times_head_dim`, `attn_rope_pairs_is_context_len_div_two`, `attn_sacred_gamma_is_phi_cubed_inv`, `attn_sacred_gamma_positive_less_than_one`, `attn_sacred_scale_in_range`, `attn_sacred_scale_matches_reference`); Trit (`trit_values`); ternary matmul (`attn_ternary_matmul_identity`, `attn_ternary_matmul_negation`, `attn_ternary_matmul_zero_weights`); residual (`attn_add_residual_identity`, `attn_add_residual_length_clamped`); softmax (`attn_softmax_normalization_single_head`, `attn_softmax_positive_all_entries`, `attn_softmax_uniform_input`, `attn_softmax_all_heads_normalized`); compute_scores (`attn_compute_scores_applies_sacred_scale`, `attn_compute_scores_causal_mask`); cache (`attn_cache_kv_stores_at_offset`); weighted values (`attn_weighted_values_uniform_attention`); exp helper (`exp_at_zero_is_one`, `exp_negative_small`, `exp_negative_large`, `exp_underflow_returns_zero`); identity (`identity_witness_holds`); and the cross-kernel anchor (`attention_phi_identity_via_softmax_matmul`).
- **Fourth cross-kernel anchor test:** `attention_phi_identity_via_softmax_matmul` is the fourth time `phi^2 + 1/phi^2 = 3` is exercised through actual numeric kernels (after Wave 15 `mac_dot_phi_identity`, Wave 16 `cpu_phi_identity_integer_projection`, Wave 18 `sr_quantize_phi_unbiased`). Construction: total = phi^2 + 1/phi^2 + 1 must equal 4 by the identity; weights w0 = phi^2/total, w1 = 1/total, w2 = (1/phi^2)/total sum to 1; routing these weights through `ternary_matmul` with all-positive weights recovers the sum 1.0, which multiplied back by total = 4.0 confirms the identity end-to-end.
- **R5-HONEST LOC observation:** the Wave-11 narrative quoted **847 LOC** for ring-092; the honest Wave-19 measurement is **760 LOC**. Pattern across the Wave-15..19 import series: ring-088 claimed 961 -> 439, ring-089 claimed 334 -> 635, ring-090 claimed 2143 -> 547, ring-091 claimed 409 -> 462, ring-092 claimed 847 -> 760. The honesty work is replacing guesses with measurements.
- **R5-HONEST out of scope:** RoPE table init (`sacred_attention_init`) is omitted because it requires `cos`/`sin` which are not available in `no_std` without libm. The `ROPE_PAIRS` constant and per-head dimensional layout are still exposed for downstream composition. The full `sacred_attention_kernel` orchestrator is also omitted; the primitives this crate ships are exactly the building blocks that orchestrator composes.
- **Compile semantics unchanged:** ring-092 lives outside `[workspace].members` (Wave-14 `exclude = [..., "rings"]` covers it automatically). `rings-rust.yml` discovers it via the matrix generator and runs `cargo check` + `cargo test`, both `continue-on-error: true`. The CI run triggered by this PR will be the first to exercise these 28 tests in public.
- **COMPILE_STATUS promotion:** ring-092 moves from `claimed-only` to `check` + `test`. The remaining 7 Wave-11 rings (ring-093..ring-099) stay `claimed-only` and the section preamble is updated to reflect the new boundary.
- **L1 TRACEABILITY:** PR cites `Closes #725` in title and body; every commit message carries it. **L2 GENERATION:** zero edits under generated trees. **L3 PURITY:** ASCII source, English doc-comments. **L4 TESTABILITY:** 28 `#[test]`s. **L5 IDENTITY:** anchor exercised both at f64 level and through softmax + ternary matmul. **L6 CEILING:** zero numeric kernel / spec changes; sacred constants mirror existing spec byte-for-byte. **L7 UNITY:** no new `*.sh` -- all new files are `.toml`, `.rs`, `.md`.
- Closes #725

## wave-18 -- Stochastic Rounding import: ring-091-rust (this PR, Closes #723)

- **NEW** (rings-only, additive): `rings/ring-091-rust/` lands with `Cargo.toml` + `src/lib.rs` (462 LOC) + `README.md`. Zero edits under `gen/`, `coq/`, `trios-coq/`, `proofs/`, `bootstrap/`, `specs/`, `conformance/`, `architecture/`, root `Cargo.toml`, or any other crate. Doc-only updates to `rings/COMPILE_STATUS.md`, `README.md` (Wave 18 footer), and this file.
Expand Down
41 changes: 33 additions & 8 deletions rings/COMPILE_STATUS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# rings/ -- Living Compile Status (Wave 13 gate, Waves 14-18 promotions)
# rings/ -- Living Compile Status (Wave 13 gate, Waves 14-19 promotions)

> Last updated: 2026-05-22 (Wave 18)
> Last updated: 2026-05-22 (Wave 19)
> Anchor: phi^2 + 1/phi^2 = 3
> CI workflow: [`.github/workflows/rings-rust.yml`](../.github/workflows/rings-rust.yml)
> Toolchain: pinned via [`Dockerfile.rust`](../Dockerfile.rust) -- `rust:1.83-bookworm`
Expand Down Expand Up @@ -112,20 +112,45 @@ green `rings-rust` workflow run this PR triggers.
|--------------------|-----------------------------------|-----:|------:|-------------------|
| `ring-091-rust` | Stochastic Rounding + SplitMix64 | 462 | 19 | `check` + `test` |

## Wave 11 -- ring-092..ring-099 (claimed-only, off-disk)
## Wave 19 import -- ring-092 (on disk, real)

Wave 19 (2026-05-22, Closes #725) imports the **fifth** Wave-11 crate for real.
Locally verified on Rust 1.83.0: `cargo check` green, `cargo test --lib`
reports **28 passed, 0 failed** on the first run. Ring-092 mirrors the
realizable subset of [`specs/nn/attention.t27`](../specs/nn/attention.t27)
(SacredAttention): sacred constants byte-for-byte (`NUM_HEADS=3`,
`HEAD_DIM=81`, `EMBED_DIM=243`, `CONTEXT_LEN=81`, `ROPE_PAIRS=40`,
`SACRED_GAMMA = phi^-3`, `SACRED_SCALE = 81^(-SACRED_GAMMA)`); `Trit`
enum; and the primitives `ternary_matmul`, `add_residual`, `apply_softmax`
(numerically stable max-subtract, per-head), `compute_scores` (Q.K^T with
causal mask + sacred scaling), `weighted_values`, `cache_kv`. A private
`exp_f64` (range-reduction + Taylor series) makes softmax viable in
`no_std` without libm. The crate's `attention_phi_identity_via_softmax_matmul`
is the **fourth cross-kernel anchor test** in the project (after
ring-088, ring-089, ring-091), routing `phi^2 + 1/phi^2 = 3` through
softmax-style normalization and ternary matmul. RoPE table init (cos/sin)
and the full `sacred_attention_kernel` orchestrator are explicitly out of
scope (R5-HONEST). Earlier Wave-11 narrative claimed 847 LOC; honest
Wave-19 measurement is **760 LOC**. Promotion will be re-confirmed by
the green `rings-rust` workflow run this PR triggers.

| Crate | Domain | LOC | Tests | Status |
|--------------------|-----------------------------------|-----:|------:|-------------------|
| `ring-092-rust` | Attention (Sacred primitives) | 760 | 28 | `check` + `test` |

## Wave 11 -- ring-093..ring-099 (claimed-only, off-disk)

Wave 11's narrative described 11 additional crates with ~ 8 969 LOC. Their
sources never reached this repository. Waves 15-18 acknowledge this
honestly and have promoted `ring-088`, `ring-089`, `ring-090`, and
`ring-091` out of this table. The rows below are **claimed-only**
sources never reached this repository. Waves 15-19 acknowledge this
honestly and have promoted `ring-088`, `ring-089`, `ring-090`, `ring-091`,
and `ring-092` out of this table. The rows below are **claimed-only**
placeholders, *not* deliverables. They will be promoted to `check` +
`test` one ring at a time, each via its own PR that carries real source
+ local verification (the Wave-15/16/17/18 template). LOC numbers below
+ local verification (the Wave-15/16/17/18/19 template). LOC numbers below
are quotes from past narrative, not measurements.

| Crate | Domain | LOC (claimed) | Status |
|--------------------|-----------------------------------|---------------:|----------------|
| `ring-092-rust` | Attention | 847 | `claimed-only` |
| `ring-093-rust` | Sparse MoE | 668 | `claimed-only` |
| `ring-094-rust` | AGI Runtime | 774 | `claimed-only` |
| `ring-095-rust` | phi-Adam | 659 | `claimed-only` |
Expand Down
1 change: 1 addition & 0 deletions rings/ring-092-rust/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cargo.lock
15 changes: 15 additions & 0 deletions rings/ring-092-rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[package]
name = "ring-092-rust"
version = "0.1.0"
edition = "2021"
description = "ring-092 Attention -- Sacred Attention primitives (TNN-backed, sacred scaling)"
license = "Apache-2.0"
publish = false

[lib]
path = "src/lib.rs"

[dependencies]

[features]
default = []
78 changes: 78 additions & 0 deletions rings/ring-092-rust/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# ring-092-rust -- Attention (Sacred Attention primitives)

Mirrors `specs/nn/attention.t27` (SacredAttention) -- the subset that is
realizable in pure `no_std` Rust without libm.

## What's exposed

- **Sacred constants** (byte-for-byte from spec):
`NUM_HEADS = 3`, `HEAD_DIM = 81`, `EMBED_DIM = 243`, `CONTEXT_LEN = 81`,
`ROPE_PAIRS = 40`, `SACRED_GAMMA = phi^-3 ~ 0.2360679...`,
`SACRED_SCALE = 81^(-SACRED_GAMMA) ~ 0.3543788557382518`.
- **`Trit`** -- balanced-ternary weight enum `{Neg, Zero, Pos}` with `value() -> i8`.
- **`ternary_matmul(input, weights, output, in_dim, out_dim)`** -- matrix-vector
product with ternary weights. Equivalent to spec's `ternary_matmul`.
- **`add_residual(output, input)`** -- in-place residual sum (clamped to
`min(output.len(), input.len())`).
- **`apply_softmax(scores, seq_len)`** -- per-head softmax with max-subtract
numerical stabilization. Operates on a `NUM_HEADS * CONTEXT_LEN` buffer.
- **`compute_scores(q, cache_k, position, seq_len, scores)`** -- Q.K^T per head,
multiplied by `SACRED_SCALE`, with a causal mask (positions `j > position`
forced to zero).
- **`weighted_values(scores, cache_v, seq_len, concat)`** -- softmax-weighted V sum.
- **`cache_kv(k_buffer, v_buffer, position, cache_k, cache_v)`** -- KV cache store
at offset `position * EMBED_DIM`.
- **`identity_witness() -> bool`** -- exercises `phi^2 + 1/phi^2 = 3`.

## no_std exp

Softmax requires `exp`, which is unavailable in `no_std` without libm. We
implement a private `exp_f64` using range reduction
(`exp(x) = (exp(x / 2^n))^(2^n)` with n=20) plus a 12-term Taylor series.
Verified against the standard library to better than 1e-9 across the
working range; underflows to 0.0 for `x < -700`.

## Cross-kernel anchor test (#4)

`attention_phi_identity_via_softmax_matmul` routes the constitutional
identity `phi^2 + 1/phi^2 = 3` through softmax-style normalization and a
ternary matmul. This is the fourth cross-kernel anchor in the ring set,
joining:

- ring-088 `mac_dot_phi_identity` (GF16 MAC)
- ring-089 `cpu_phi_identity_integer_projection` (TNN CPU)
- ring-091 `sr_quantize_phi_unbiased` (SR-quantization)
- **ring-092 `attention_phi_identity_via_softmax_matmul`** (softmax + ternary matmul)

## Out of scope (R5-HONEST)

- **RoPE table init** (`sacred_attention_init`) requires `cos`/`sin`; we don't
implement these without libm. Constants `ROPE_PAIRS` and shapes are still
exposed.
- **`sacred_attention_kernel` orchestrator** -- the full single-position
forward pass that composes project_qkv -> apply_rope_qk -> cache_kv ->
compute_scores -> apply_softmax -> weighted_values -> project_output ->
add_residual. The primitives this crate exposes are sufficient to
compose it externally.
- **`project_qkv` / `project_output`** -- thin wrappers in the spec; they
reduce to `ternary_matmul` calls which we provide directly.

## Local verification (Rust 1.83.0)

```
cargo check -p ring-092-rust # clean
cargo test -p ring-092-rust --lib # 28 passed
```

## Constitutional compliance

- L1 TRACEABILITY -- `Closes #725`
- L2 GENERATION -- no edits in protected dirs
- L3 PURITY -- ASCII source, English doc-comments
- L4 TESTABILITY -- 28 `#[test]` blocks, all green
- L5 IDENTITY -- `phi^2 + 1/phi^2 = 3` exercised in `identity_witness` and
in the cross-kernel anchor test
- L6 CEILING -- sacred constants mirrored byte-for-byte from spec
- L7 UNITY -- no `*.sh` files

`phi^2 + 1/phi^2 = 3`
Loading
Loading