zaya: ZAYA1-74B sliding-window attention, legacy checkpoints (ZAYA1-base, reasoning-base), --remote chat templates

[bong-water-water-bong]

ZAYA1-74B-preview support plus the rest of the ZAYA1 family's checkpoints.

1. Sliding-window attention (ZAYA1-74B-preview). Its layers alternate between sliding-window attention (window 4097, rope theta 1e4) and full attention (theta 1e7). ZAYA1-8B has no sliding layers.

• conversion/zaya.py writes zaya.attention.sliding_window, zaya.attention.sliding_window_pattern and zaya.rope.freq_base_swa when the config has them
• src/models/zaya.cpp builds graph<iswa> and picks the rope base per layer
• the loader reads sliding_window_pattern the way cohere2moe does: a scalar first as a period, then an array per layer, and without the key a period of 2 (llama_model_saver does not write the key). The first version read a scalar through the array path, which fails test-llama-archs; fixed.

2. Legacy (Megatron-style) checkpoints. ZAYA1-base, ZAYA1-reasoning-base and the *-legacy repos keep Zyphra's original layout: separate attention and MoE half-layers, per-layer config lists, and experts stored one by one. transformers 5 cannot load them. The converter normalizes the config and tensors to the transformers layout.

3. --remote fetches chat_template.jinja. Without it, remote conversions had no chat template.

Verified on Strix Halo:

• test-llama-archs -a zaya: Vulkan and CPU OK, save/reload bit-exact (the synthetic model now takes the SWA path)
• ZAYA1-8B: F16 reconverted after the rebase picks the same top token as transformers FP32 at 95/96 teacher-forced positions, the same as before
• Legacy mapping: --remote Zyphra/ZAYA1-8B-legacy converts to a GGUF whose 1283 tensors are byte-identical to the one from Zyphra/ZAYA1-8B, with identical model metadata. Normalized configs equal the new ones for both the 8B and 74B pairs, including the 74B's window (legacy 4096 → 4097) and per-layer rope bases.
• ZAYA1-74B-preview (HF revision bc8eb466): Q8_0 79.7 GB with the SWA keys; Q4_K_M 42.6 GiB, llama-bench Vulkan0 pp512 505.8 tok/s, tg128 35.4 tok/s; chat coherent

Rebased onto #9.

🤖 Generated with Claude Code

[bong-water-water-bong]

Review from the commit poll. test-llama-archs fails with this change (fork CI: ubuntu x64/arm64, and the Windows jobs that run it):

26: | zaya | AMD EPYC 9V45 | MoE | E llama_model_load: error loading model: error loading model hyperparameters: key not found in model: zaya.attention.sliding_wi…

The generic fixture writes LLM_KV_ATTENTION_SLIDING_WINDOW for every arch, so the new get_key(LLM_KV_ATTENTION_SLIDING_WINDOW, …, false) finds one and turns on SWA. A companion key that is read as required is then missing. Two ways to fix it:

• key SWA off the pattern (or rope.freq_base_swa) being present, not the window alone. Only the converter writes the pattern, and only for ZAYA1-74B;
• or set the ZAYA fixture in tests/test-llama-archs.cpp explicitly (arch == LLM_ARCH_ZAYA, like its SSM_CONV_KERNEL = 2), and cover the SWA path with a fixture that carries all three keys.

The rest looks right to me:

• ZAYA1-8B is unchanged;
• LLAMA_SWA_TYPE_STANDARD (masked when q − k ≥ n_swa) matches HF's sliding-window mask;
• --rope-freq-base overrides still reach the non-SWA layers through get_rope_freq_base.

On "70/96": I measured the same with my harness on the Q4_K_M. The F16 GGUF scores 95/96, so the gap is quantisation.

[bong-water-water-bong]

Re-checked after d8c84b2: test-llama-archs passes on ubuntu x64/arm64 (the remaining Windows x64-openblas failure is test-thread-safety crashing at exit with 0xc0000374, which doesn't involve zaya). On strixhalo with this PR's zaya.cpp: test-llama-archs -a zaya OK on Vulkan and CPU, and ZAYA1-8B Q4_K_M perplexity on Vulkan is 21.5731, identical to before, so the 8B is unchanged. Merging.