feat(models): add the Inkling text backbone (banded relative-position attention, short-conv residuals, sigmoid-routed MoE with shared experts) #1318

Summary

Add the Inkling decoder (model_type: "inkling_mm_model", text sub-config model_type: "inkling_text" or "inkling") as a text model: embedding RMSNorm, hybrid sliding/global layers whose attention has no RoPE but a learned banded relative-position bias with a log-position temperature, depthwise causal short convolutions on K, V and both residual branches (a 4-slot conv cache per layer), dense SwiGLU on the first dense_mlp_idx layers and a sigmoid-routed fine-grained MoE (256 routed, 6 active, 2 shared experts weighted by a logsigmoid softmax) elsewhere, a muP-scaled LM head and a padded vocabulary. Both the original model.llm.* checkpoint layout (bf16 and ModelOpt NVFP4 experts) and the pre-converted MLX 4-bit layout load. Image, audio, video and MTP are separate sub-issues of #1313; this issue must ignore model.visual.*, model.audio.* and model.mtp.* tensors without failing.

Current behavior

• grep -ri inkling src/ returns nothing; src/models/detection.rs:150 has no inkling arm.
• Nearest code to reuse: src/models/lfm2.rs:306 (ShortConv, depthwise causal conv with a per-layer conv state, Lfm2LayerCache::Conv) and src/models/conv_decode.rs (build_conv_decode_weight, the L = 1 broadcast-multiply decode step) for the short convolutions; src/models/deepseek_v3.rs for sigmoid routing with e_score_correction_bias; src/models/switch_layers.rs:754 SwitchGLU; src/models/mellum.rs for the sliding/global layer split. No module has a relative-position bias that replaces RoPE, a log-position temperature, or a shared-expert router weight derived from the same softmax as the routed weights.
• src/server/chat_template.rs renders with minijinja's builtin tojson, which rejects the sort_keys/separators kwargs the Inkling template uses (see fix(server): make the chat-template tojson filter match Python json.dumps (kwargs and separators) #1309).

Expected behavior

Detection

config.json top level: architectures: ["InklingForConditionalGeneration"], model_type: "inkling_mm_model" (also accept "inkling"), eos_token_id: 200006, image_token_id: 200054 (default), audio_token_id: 200053 (default), vocab_size: 201024, text_config: {...}, vision_config, audio_config, mtp_config, optional quantization / quantization_config. Route to ModelType::Inkling (text path; the VLM path is added by the tower sub-issues and must not change the detection string).

Text config (text_config)

Two spellings of the MLP sizes exist and both must load: native (intermediate_size = MoE expert width, dense_intermediate_size = dense width) and HF-transformers (intermediate_size = dense width, moe_intermediate_size = MoE width). Rule: if moe_intermediate_size is present, moe_width = moe_intermediate_size and dense_width = intermediate_size; otherwise moe_width = intermediate_size and dense_width = dense_intermediate_size (required in that case). Cross-check against weight shapes at load (w13_dn.weight is [2 * dense_width, hidden]).

Layer classification, i in 0..num_hidden_layers:

• sliding: layer_types[i] == "hybrid_sliding" if layer_types is set; else local_layer_ids.contains(i) if set; else (i + 1) % 6 != 0.
• dense MLP: mlp_layer_types[i] == "dense" if set; else i < dense_mlp_idx.

Per-layer forward

Shapes: x: [B, L, H], d = head_dim (128), n_h/n_kv from the global or sliding triple, K = sconv_kernel_size (4), W = sliding_window_size, R = rel_extent on global layers and R = W on sliding layers, eps = rms_norm_eps.

Short convolution sconv(u, state, residual) (u: [B, L, C], state: [B, K-1, C] f32, zeros on the first call):

uf = u.astype(f32)
xp = concat([state, uf], axis=1)                              # [B, K-1+L, C]
y  = depthwise_conv1d(xp, w[C, K, 1], no bias)                # [B, L, C]; computed in f32 (cast w to f32)
y  = (y + uf).astype(u.dtype)                                 # inner residual is part of the conv block
new_state = xp[:, -(K-1):, :]                                 # f32
return (residual + y if residual is given else y), new_state

Attention (h = RMSNorm(x; attn_norm)):

q = wq_du(h).reshape(B, L, n_h, d);   k = wk_dv(h);   v = wv_dv(h);   r = wr_du(h).reshape(B, L, n_h, d_rel)
k, conv[0] = sconv(k, conv[0]);       v, conv[1] = sconv(v, conv[1])          # over the flat n_kv*d channels
k = RMSNorm_d(k.reshape(B, L, n_kv, d); k_norm) -> [B, n_kv, L, d];   v -> [B, n_kv, L, d]
q = RMSNorm_d(q; q_norm) -> [B, n_h, L, d]
K, V = kv_cache.update_and_fetch(k, v);  S = K.shape[2]
sliding layers: if S > L + W - 1: drop the oldest S - (L + W - 1) keys (KVCache::trim_front) so S = L + W - 1
offset = S - L
bias[b, h, i, j] for i in 0..L, j in 0..S, dist = (i + offset) - j:
    dist < 0                       -> -1e30
    sliding and dist >= W          -> -1e30
    dist < R                       -> sum_d r[b, i, h, d] * P[d, dist]          # P = rel_logits_proj.proj [d_rel, R]
    otherwise                      -> 0
if global layer and log_scaling_n_floor is set:
    qpos = (arange(L) + offset + 1).astype(f32)                                   # 1-based absolute query position
    tau  = 1 + log_scaling_alpha * ln(max(qpos / n_floor, 1))                     # [L], >= 1, == 1 below 128k tokens
    q = q * tau[None, None, :, None];   bias = where(bias > -1e29, bias * tau[None, None, :, None], bias)
o = SDPA(q, K, V, scale = 1 / d, additive mask = bias.astype(q.dtype))          # scale is 1/head_dim, not 1/sqrt
o = o.transpose(0, 2, 1, 3).reshape(B, L, n_h * d);   a = wo_ud(o)

Graph-op form of the bias: rl = (r @ P).transpose(0, 2, 1, 3) [B, n_h, L, R]; g = clip(dist, 0, R - 1) broadcast to [B, n_h, L, S]; pb = take_along_axis(rl, g, -1); pb = where(dist >= R, 0, pb); bias = where(dist < 0 | (sliding & dist >= W), -1e30, pb).

Decoder layer:

a = attention(RMSNorm(x; attn_norm))
hmid, conv[2] = sconv(a, conv[2], residual = x)                  # hmid = x + a + conv(a)
m = mlp(RMSNorm(hmid; mlp_norm))
out, conv[3] = sconv(m, conv[3], residual = hmid)                # out = hmid + m + conv(m)

Dense MLP: down(silu(gate(n)) * up(n)) * global_scale (global_scale is a learned [1] tensor).

MoE (n: [N, H], Rn = n_routed_experts, SH = n_shared_experts, k = num_experts_per_tok, I = moe_width):

logits = n @ gate_weight.T                                        # gate_weight [Rn + SH, H]; the last SH rows are the shared experts
scores = sigmoid(logits[:, :Rn].astype(f32)) + e_score_correction_bias[Rn]
idx    = argpartition(-scores, k - 1)[:, :k]                       # selection uses scores + bias
tl     = concat([take_along_axis(logits[:, :Rn], idx), logits[:, Rn:]], -1).astype(f32)      # raw logits of the k routed + SH shared
lp     = -logaddexp(0, -tl)                                        # logsigmoid
w      = exp(lp - logsumexp(lp, -1, keepdims)) * route_scale * global_scale                   # softmax over k + SH entries
w_routed = w[:, :k].astype(n.dtype);   w_shared = w[:, k:].astype(n.dtype)                  # [N, k], [N, SH]
yr = sum_k w_routed[:, k] * Expert_{idx_k}(n)                      # SwiGLU experts, width I (SwitchGLU + moe_weighted_sum)
ys = shared_down( concat_s( silu(shared_gate_s(n)) * shared_up_s(n) * w_shared[:, s] ) )     # SH dense experts concatenated expert-major on the intermediate axis
out = (yr + ys).astype(n.dtype)

NVFP4 experts additionally carry per-expert scales: x_gate = gate_proj(n) * gate_scale[idx][..., None] before the activation and yr_k = down_proj(act) * out_scale[idx][..., None] after it (gate_scale = scale2_w13, out_scale = scale2_w13 * scale2_w2); for every other checkpoint both vectors are all ones and the multiplies are skipped.

Model: h = embed(ids); h = RMSNorm(h; embed_norm) if use_embed_norm; layers; h = RMSNorm(h; norm); logits = unembed(h / logits_mup_width_multiplier)[..., :unpadded_vocab_size].

Caches per layer: KVCache (global) or KVCache plus front trimming to L + W - 1 (sliding; a RotatingKVCache cannot be used because the bias needs true key distances, not rotated slots) and [Option<MlxArray>; 4] conv states: slot 0 [B, 3, n_kv * d], slot 1 [B, 3, n_kv * d], slot 2 [B, 3, H], slot 3 [B, 3, H], all f32.

Weight keys and sanitize

Original layout (thinkingmachines/Inkling-Small, bf16; -NVFP4; the 0.6B checkpoint, f32):

model.llm.embed.weight [V, H]        model.llm.embed_norm.weight [H]       model.llm.norm.weight [H]       model.llm.unembed.weight [V, H]
model.llm.layers.{i}.attn_norm.weight [H]     .mlp_norm.weight [H]     .attn_sconv.weight [H, 1, K]     .mlp_sconv.weight [H, 1, K]
model.llm.layers.{i}.attn.wq_du.weight [n_h*d, H]   .wk_dv.weight [n_kv*d, H]   .wv_dv.weight [n_kv*d, H]   .wr_du.weight [n_h*d_rel, H]   .wo_ud.weight [H, n_h*d]
model.llm.layers.{i}.attn.q_norm.weight [d]   .k_norm.weight [d]   .k_sconv.weight [n_kv*d, 1, K]   .v_sconv.weight [n_kv*d, 1, K]   .rel_logits_proj.proj [d_rel, R]
dense layers:  model.llm.layers.{i}.mlp.w13_dn.weight [2*dense_width, H]   .mlp.w2_md.weight [H, dense_width]   .mlp.global_scale [1]
MoE layers:    model.llm.layers.{i}.mlp.gate.weight [Rn+SH, H]   .mlp.gate.bias [Rn]   .mlp.gate.global_scale [1]
               .mlp.experts.w13_weight [Rn, 2I, H]   .mlp.experts.w2_weight [Rn, H, I]
               .mlp.shared_experts.shared_w13_weight [SH, 2I, H]   .mlp.shared_experts.shared_w2_weight [SH, H, I]
NVFP4 MoE:     .mlp.experts.w13_weight U8 [Rn, 2I, H/2]   .mlp.experts.w13_weight.scale F8_E4M3 [Rn, 2I, H/16]   .mlp.experts.w13_weight.scale2 F32 [Rn]
               .mlp.experts.w2_weight U8 [Rn, H, I/2]     .mlp.experts.w2_weight.scale F8_E4M3 [Rn, H, I/16]     .mlp.experts.w2_weight.scale2 F32 [Rn]
               .mlp.experts.w13_weight.input_amax, .original_shape (drop)
skip:          model.mtp.*   model.visual.*   model.audio.*   *training_args*

Pre-converted MLX layout (mlx-community/Inkling-Small-mlx-4bit, quantization: {group_size: 64, bits: 4, recipe: "experts_only"}): same model.llm.* names for attention and norms (bf16), dense layers as mlp.{gate_proj,up_proj,down_proj}.{weight,scales,biases} + mlp.global_scale, MoE layers as mlp.experts.{gate_proj,up_proj,down_proj}.{weight,scales,biases} ([Rn, out, in/8] uint32 plus [Rn, out, in/64] scales and biases), mlp.shared_experts.{gate_proj,up_proj,down_proj}.{weight,scales,biases} ([SH, ...]), mlp.gate.{weight,bias,global_scale}, and model.audio.encoder.{weight,scales,biases} (quantized embedding, ignored here). Conv weights are already [C, K, 1].

Sanitize rules (run before from_weights; must be idempotent):

1. Drop keys matching model.mtp., model.visual., model.audio., training_args, .input_amax, .original_shape.
2. Rename top level: model.llm.embed.weight -> model.embed_tokens.weight, model.llm.unembed.weight -> lm_head.weight, model.llm.embed_norm.weight -> model.embed_norm.weight, model.llm.norm.weight -> model.norm.weight, model.llm.layers.{i}. prefix -> model.layers.{i}..
3. Attention: attn.wq_du -> self_attn.q_proj, wk_dv -> k_proj, wv_dv -> v_proj, wr_du -> r_proj, wo_ud -> o_proj (.weight and any .scales/.biases); attn.q_norm/k_norm -> self_attn.q_norm/k_norm; attn.rel_logits_proj.proj -> self_attn.rel_proj; attn.k_sconv.weight -> self_attn.k_sconv.conv.weight, attn.v_sconv.weight -> self_attn.v_sconv.conv.weight; attn_norm -> input_layernorm, mlp_norm -> post_attention_layernorm, attn_sconv.weight -> attn_sconv.conv.weight, mlp_sconv.weight -> mlp_sconv.conv.weight. Conv weights: if the tensor is 3-D with last dim != 1, transpose [C, 1, K] -> [C, K, 1] (MLX depthwise layout); leave [C, K, 1] alone.
4. Dense MLP: mlp.w13_dn.weight [2I_d, H] is row-interleaved: reshape [I_d, 2, H]; gate = [:, 0, :], up = [:, 1, :] (make contiguous) -> mlp.gate_proj.weight, mlp.up_proj.weight; mlp.w2_md.weight -> mlp.down_proj.weight; mlp.global_scale unchanged.
5. MoE router: mlp.gate.weight -> mlp.gate_weight ([Rn+SH, H], keep bf16, never quantize), mlp.gate.bias -> mlp.e_score_correction_bias (f32), mlp.gate.global_scale or mlp.global_scale -> mlp.global_scale.
6. Routed experts:
   • bf16/f32 w13_weight [Rn, 2I, H]: de-interleave along axis 1 (reshape [Rn, I, 2, H]) into mlp.switch_mlp.gate_proj.weight / up_proj.weight [Rn, I, H]; w2_weight -> mlp.switch_mlp.down_proj.weight [Rn, H, I]; gate_scale = ones[Rn], out_scale = ones[Rn].
   • NVFP4 (w13_weight is U8): reinterpret the packed bytes as little-endian uint32 [Rn, 2I, H/8] (this is MLX's nvfp4 layout), de-interleave rows into gate/up [Rn, I, H/8], same de-interleave for .scale (uint8 [Rn, 2I, H/16] -> [Rn, I, H/16], raw E4M3 bytes; if the loader promoted F8_E4M3 to f16/f32, re-encode with f32_to_f8_e4m3 from src/models/sanitize.rs:127, which is lossless for E4M3-valued input); w2_weight -> down_proj.weight uint32 [Rn, H, I/8], .scale -> down_proj.scales [Rn, H, I/16]; gate_scale = scale2_w13.astype(f32) [Rn], out_scale = scale2_w13 * scale2_w2 [Rn]. Quantization for these layers is {group_size: 16, bits: 4, mode: "nvfp4"} (also promote hf_quant_config.json with quant_algo: "NVFP4" to that dict when config.json has no quantization). The first MoE layer (dense_mlp_idx) of the NVFP4 release is excluded from quantization and arrives bf16.
   • Pre-converted mlp.experts.{gate_proj,up_proj,down_proj}.* -> mlp.switch_mlp.* unchanged.
7. Shared experts to a dense concatenation: shared_w13_weight [SH, 2I, H] de-interleave to gate/up [SH, I, H] then reshape [SH*I, H] (expert-major rows) -> mlp.shared_experts.gate_proj.weight / up_proj.weight; shared_w2_weight [SH, H, I] -> transpose(1, 0, 2).reshape [H, SH*I] -> mlp.shared_experts.down_proj.weight. Pre-converted quantized triplets [SH, I, H/8] / [SH, I, H/g] use the same reshapes (gate/up: merge the leading two axes; down: transpose then merge the trailing two axes), which is exact because a row's quantization groups are preserved (I % g == 0).
8. Optional fusion (performance only, same numerics): concatenate q_proj, k_proj, v_proj, r_proj rows (and their .scales/.biases) into qkvr_proj and slice the output; a quantization dict keyed by ...self_attn.q_proj must then also apply to ...self_attn.qkvr_proj. Ship the unfused form first.

Chat template and tokens

Special tokens: <|message_user|> 200000, <|message_model|> 200001, <|message_system|> 200002, <|message_tool|> 200003, <|content_text|> 200004, <|content_image|> 200005, <|content_model_end_sampling|> 200006 (EOS), <|content_thinking|> 200008, <|end_message|> 200010, <|content_audio_input|> 200020, <|audio_end|> 200043, <|content_xml|> 200024, <|content_invoke_tool_json|> 200049, <|endoftext|> 199999 (pad), image placeholder <|unused_200054|> 200054, audio placeholder <|unused_200053|> 200053. The tokenizer declares no bos_token, so nothing is prepended. Stop on 200006. The template always prepends <|message_system|><|content_text|>Thinking effort level: <eff><|end_message|> where eff comes from the reasoning_effort chat-template kwarg (string none|minimal|low|medium|high|max mapped to 0|0.1|0.2|0.7|0.9|0.99, or a float in [0, 0.99], default 0.9); the generation prompt is <|message_model|>. The reasoning span is <|content_thinking|> ... <|end_message|> followed by <|message_model|><|content_text|> for the answer; register these as the reasoning markers for stream_filter.rs. Tool calls are rendered as <|message_model|><name><|content_invoke_tool_json|>{"name":...,"args":...}<|end_message|>; tool declarations use tojson(sort_keys=true, separators=(",", ":")) (needs #1309).

Implementation plan

1. src/models/inkling.rs (new) with InklingTextConfig (serde; #[serde(alias = "conv_kernel_size")] on sconv_kernel_size; the MoE/dense width resolution above), InklingConfig (top level: text_config, image_token_id, audio_token_id, eos_token_id, vocab_size, quantization, quantization_config), layer_is_sliding(i), layer_is_dense(i).
2. InklingShortConv { conv_weight [C, K, 1], decode_weight [1, K, C] } with forward(u, state, residual) -> (out, new_state): prefill through mlxcel_core::conv1d (groups = C) on the f32 padded input; L == 1 through crate::models::conv_decode::conv_decode_step with the precomputed decode weight. Store states f32.
3. InklingAttention { q_proj, k_proj, v_proj, r_proj, o_proj: UnifiedLinear, q_norm, k_norm: RMSNorm, k_sconv, v_sconv: InklingShortConv, rel_proj: MlxArray, is_sliding, n_heads, n_kv, head_dim, d_rel, rel_extent, window, log_floor: Option<f32>, log_alpha } implementing the attention block above with mlxcel_core::scaled_dot_product_attention (additive mask). Build dist once per call from offset, L, S.
4. InklingDenseMlp (SwiGLU + global_scale) and InklingSparseMoe { gate_weight, e_score_correction_bias, global_scale, switch_mlp: SwitchGLU, gate_scale, out_scale, shared: InklingSharedDense { gate_proj, up_proj, down_proj: UnifiedLinear } }. Router via mlxcel_core::{sigmoid, argpartition, take_along_axis, logaddexp, logsumexp_axis}; shared weights w_shared repeated over I per expert and multiplied into the concatenated intermediate before down_proj. Add a SwitchGLU::forward_with_expert_scales(x, idx, gate_scale, out_scale) in src/models/switch_layers.rs (update its // Used by: comment) that skips the multiplies when both vectors are None. Keep MLXCEL_FUSED_MOE=0 behavior: the fused kernel path is only taken when both scales are None.
5. InklingDecoderLayer, InklingLayerCache { kv: KVCache, conv: [Option<UniquePtr<MlxArray>>; 4] }, InklingModel (embed, embed_norm, layers, norm, lm_head) with forward_with_caches, make_caches, model-owned sequence state via src/models/model_owned.rs exactly as lfm2.rs:972 does, and LanguageModel (supports_batching = false initially; eos_token_ids = [200006]). Snapshot/restore of the 4 conv states plus KV for ModelStateSnapshot (the MTP sub-issue needs rollback_speculative_cache; provide snapshot_into/restore_from now, as Lfm2LayerCache does).
6. sanitize_weights implementing rules 1 to 7 (8 optional) in src/models/inkling.rs; quantized layers load via UnifiedLinear::from_weights_with_mode / SwitchLinear::from_weights_with_mode with the resolved mode (affine g64 b4 for the MLX checkpoint, nvfp4 g16 b4 for the NVFP4 release, none for bf16). Per-tensor bit inference from .scales shape (the experts_only recipe leaves attention bf16) follows the existing UnifiedLinear behavior.
7. Detection: "inkling_mm_model" | "inkling" => ModelType::Inkling in src/models/detection.rs, ModelType::Inkling in src/models/mod.rs (enum, description table, architectures list), loader arm in src/loading/mod.rs.
8. Chat: stream_filter.rs reasoning markers (<|content_thinking|> open, <|end_message|> close) keyed on ModelType::Inkling; reasoning_effort passes through chat_template_kwargs untouched.
9. docs/supported-models.md: add Inkling (text; towers and MTP follow).

Validation

(a) Unit tests, src/models/inkling_tests.rs:

• layer_classification_layer_types_then_local_ids_then_modulo: the three rules give the documented sliding sets for the 0.6B (layer_types), Inkling-Small (local_layer_ids) and default ((i+1) % 6 != 0) configs; mlp_layer_types overrides dense_mlp_idx.
• mlp_width_resolution_accepts_both_spellings: {intermediate_size: 4096, moe_intermediate_size: 512} gives dense 4096 / MoE 512; {intermediate_size: 2048, dense_intermediate_size: 16384} gives dense 16384 / MoE 2048.
• banded_bias_matches_reference_formula: random r, P, L = 5, S = 9, offset = 4, R = 6, sliding W = 4: compare the graph form against a scalar loop implementing the four-way rule.
• log_scaling_tau_is_identity_below_floor_and_scales_mask: with n_floor = 8, positions 1..8 give tau = 1; position 16 gives 1 + 0.1 ln 2; masked (-1e30) entries are unchanged.
• sconv_prefill_decode_parity_and_state: a 7-token prefill followed by 3 decode steps equals a 10-token prefill (conv output and final state), bf16 input, f32 state.
• router_weights_softmax_over_routed_and_shared_logsigmoid: hand-computed 4 routed + 2 shared case; selection uses bias, weights do not; sum(w) == route_scale * global_scale.
• shared_experts_dense_equals_per_expert_loop: random SH = 2 experts; the concatenated form equals sum_s w_s * Expert_s(x) within 1e-5 (f32).
• sanitize_original_layout_bf16 / sanitize_original_layout_nvfp4 / sanitize_preconverted_mlx_4bit: tiny synthetic maps for each layout produce the canonical key set, conv weights end [C, K, 1], w13 de-interleave is correct (row 0 -> gate 0, row 1 -> up 0), NVFP4 gate_scale/out_scale equal s13 and s13*s2, model.visual.*/model.audio.*/model.mtp.* are dropped, and sanitize is idempotent.
• prefill_is_causal_with_sliding_and_global_layers (96-token random prompt; logits at position t unchanged by later tokens) and decode_matches_prefill_logits (position-by-position parity between incremental decode and one-shot prefill, atol 3e-3 bf16).
• detection_inkling: both model_type spellings map to ModelType::Inkling (in src/models/detection_tests.rs).

(b) Real checkpoints:

• inference-optimization/Inkling-0.6B-A0.6B (2.6 GB f32, 12 layers, 8 experts, modelsize:small; same key layout as the large releases, HF-style MLP spelling) in models/Inkling-0.6B-A0.6B:

MLXCEL_FUSED_MOE=0 ./target/release/mlxcel generate -m models/Inkling-0.6B-A0.6B -p "The quick brown fox" -n 64

Acceptance: token-exact for 64 greedy tokens against a reference run of the same checkpoint on the same machine with the RAW prompt (this checkpoint is a small training run, so fluency is not required; id equality is the gate), and a layer-by-layer mean-abs trace within bf16 jitter.

• mlx-community/Inkling-Small-mlx-4bit (about 153 GB, experts-only 4-bit, modelsize:xlarge, needs a 192 GB machine) in models/Inkling-Small-mlx-4bit:

./target/release/mlxcel generate -m models/Inkling-Small-mlx-4bit -p "Explain why the sky is blue in two sentences." -n 64
./target/release/mlxcel-server -m models/Inkling-Small-mlx-4bit   # one chat request with reasoning_effort: "low"

Acceptance: finite logits, fluent answer, the chat request stops on id 200006 and the reasoning span is separated from the answer; decode throughput reported in the PR.

• NVFP4 path: thinkingmachines/Inkling-Small-NVFP4 (about 150 GB, modelsize:xlarge) loads and produces fluent text on Apple Silicon (native nvfp4 gather_qmm with the per-expert scale sidecars); CUDA runs when a node is reachable.

Acceptance criteria

• Inkling text loads from the original bf16 layout, the NVFP4 layout and the pre-converted MLX 4-bit layout
• Banded relative-position bias, log-position temperature, short-conv residuals with a 4-slot f32 conv cache and the logsigmoid-softmax router are implemented as specified and unit-tested against scalar references
• Token-exact on the 0.6B checkpoint; fluent on Inkling-Small 4-bit; chat template renders the thinking-effort header and stops on 200006
• docs/supported-models.md updated
• detection table in src/models/detection.rs updated with a test
• cargo test --workspace --profile test-fast --features metal,accelerate passes
• cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all -- --check pass

Out of scope

• Vision, audio, video towers and the MTP drafter (other sub-issues of feat(models): Inkling (Thinking Machines) omni model family, text + image + audio + video + MTP drafter #1313).
• Fused Metal kernels (mask, sconv decode, router, q4 down-combine) and the fused qkvr_proj.
• Batched prefill with left padding (conv-state masking); supports_batching = false until a follow-up.

Dependencies

• Blocked by fix(server): make the chat-template tojson filter match Python json.dumps (kwargs and separators) #1309 (only the tools branch of the template).

• Part of epic feat(models): Inkling (Thinking Machines) omni model family, text + image + audio + video + MTP drafter #1313