design: scope the SSM / hybrid / recurrent track (Mamba2, Jamba, RWKV7, Qwen3-Next, and more) #502

Part of #493

Context

SSM, hybrid, and recurrent architectures replace standard attention with state-space or linear-recurrent mixing. They need new graph primitives and explicit state handling that is not the KV cache. Target set: Mamba / Mamba2, Jamba, Falcon-H1, RWKV7, Qwen3-Next, Plamo2, Nemotron-H, Kimi-Linear, RecurrentGemma.

Scope

• Write a design doc covering the recurrent/state-space mixing primitives, state handling (allocation, per-slot carry through the batch engine), and how they coexist with the existing KV-cache attention layers in hybrid models.
• Implement a first reference architecture (for example Mamba2) to validate the design, then split the remaining work into a dedicated follow-up epic.

Acceptance criteria

• Design doc merged. Added at spike/openxla/SSM_HYBRID_DESIGN.md.
• One reference architecture (for example Mamba2) validated end to end, OR an explicit, recorded decision to defer with the follow-up epic created. Recorded decision: DEFER. Follow-up epic created: epic: OpenXLA SSM / hybrid / recurrent architecture support (Mamba2, Nemotron-H, Jamba, and more) #564.

Resolution

Deferred by explicit decision, per the acceptance criteria's second option. The design doc spike/openxla/SSM_HYBRID_DESIGN.md covers the recurrent/state-space mixing primitives, the per-slot state allocation and carry through the continuous-batching engine (distinct from the KV cache), and the hybrid coexistence with KV-cache attention layers, grounded in the current codebase. A first reference architecture (Mamba2) needs a new scan graph primitive, a new C-ABI state resource, a new per-layer mixer dispatch, and a new state-aware validation harness, comparable in size to the whole Stage 2 continuous-batching effort, so it is scoped into the follow-up epic #564 rather than landed here.

Dependencies

None hard. Conceptually follows the Window A foundation; state handling intersects the batch engine. Spun out into follow-up epic #564.