Skip to content

wave-19: import ring-092 Attention -- Sacred Attention kernel (TNN-backed, 0-RoPE, sacred scaling) #725

Closed

Description

@gHashTag

Wave 19 — ring-092 Attention

Import ring-092-rust mirroring specs/nn/attention.t27 (SacredAttention).

Scope

  • ternary_matmul — TNN matmul (Trit = -1/0/+1)
  • add_residual — residual sum
  • apply_softmax — max-subtract + exp + normalize (per-head)
  • compute_scores — Q·K^T with causal mask, multiplied by SACRED_SCALE
  • weighted_values — softmax(scores) · V
  • cache_kv — KV cache store
  • Constants: NUM_HEADS=3, HEAD_DIM=81, EMBED_DIM=243, CONTEXT_LEN=81, ROPE_PAIRS=40
  • SACRED_GAMMA = phi^-3 ~ 0.2360679; SACRED_SCALE = 81^(-SACRED_GAMMA) ~ 0.354
  • identity_witness() exercising phi^2 + 1/phi^2 = 3
  • Cross-kernel anchor test ([SEED-1] Ring-1: All immediates — lex all 28 specs without errors #4): phi-identity routed through attention softmax+matmul

Out of scope (R5-HONEST)

  • RoPE table init via cos/sin (no_std without libm) -- skipped; ROPE constants still exposed
  • Full sacred_attention_kernel orchestrator -- primitives only

Constraints

L1 (Closes #N) -- L2 (no edits in protected dirs) -- L3 (ASCII, English) -- L4 (#[test]) -- L5 (anchor) -- L6 (mirror spec constants byte-for-byte) -- L7 (no .sh)

phi^2 + 1/phi^2 = 3

Activity

  1. added a commit that references this issue on May 22, 2026
    eea9804
  2. added 2 commits that reference this issue on Jun 1, 2026
    d31955d
    6ea2b70
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions