Skip to content

chore(release): refresh Linux aarch64 CUDA artifact for specialized serving #1653

Open

Description

@inureyes

Problem

The official v0.6.0 Linux aarch64 CUDA 13 archive is older than the specialized-serving behavior now required by the public recipes catalog:

  • Text-generation recipes for Qwen, Gemma, and Llama reach the chat-completions boundary on an NVIDIA GB10.
  • The archive does not expose the current dedicated embedding-serving interface needed by the Qwen3 Embedding recipe.
  • A Whisper checkpoint loads and serves the audio transcription route, but /health continues to inspect the unavailable chat worker and remains at HTTP 503.

A current-main CUDA build compiled for SM 12.1 passes all five seed recipes through their task-specific HTTP endpoints on the same GB10. That proves the source behavior, but it must not be published as verification of the older v0.6.0 archive.

Proposed work

  • Publish a new versioned Linux aarch64 CUDA 13 release archive after the specialized embedding and audio-health changes reach the public repository.
  • Exercise the exact downloaded archive rather than an in-tree build.
  • Feed the resulting rendering hashes and logs into the downstream recipes verified-badge workflow.

Do not replace the existing v0.6.0 asset in place; a new version keeps download provenance and cached artifacts unambiguous.

Acceptance criteria

  • The new archive contains aarch64 ELF binaries dynamically linked to the expected CUDA 13 libraries with no missing shared objects on GB10.
  • mlxcel-server from that archive starts a Qwen3 embedding checkpoint in dedicated embedding mode and /v1/embeddings returns a non-empty vector.
  • mlxcel-server from that archive starts Whisper, /health reaches HTTP 200 after load, and /v1/audio/transcriptions returns non-empty text.
  • Qwen3 4B, Gemma 4 E4B, and Llama 3.1 8B chat recipes return a non-empty content or reasoning_content response.
  • The exact release version and binary checksum are preserved with the five-seed GB10 verification evidence.

Activity

  1. inureyes commented on Oct 15, 2026

    @inureyes
    MemberAuthor

    Status check against v0.7.0 (tagged 2026-09-09): this issue is still valid and stays open.

    Covered by v0.7.0:

    Not covered:

    • The audio-only /health fix (fix(server): report an audio-only server healthy once its worker is ready #1750, 5004dfcf) merged on 2026-09-10, one day after the tag. git merge-base --is-ancestor 5004dfcf v0.7.0 fails, so a Whisper server from the v0.7.0 archive still reports 503 and the /health acceptance criterion cannot pass on any published archive.
    • main is still at version 0.7.0 with 327 unreleased commits since the tag.

    This can close once the next release (0.7.1 or 0.8.0) is published and the five GB10 seed recipes pass against that exact downloaded archive, with its version and checksum recorded.

  2. self-assigned this
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

platform:linuxLinux (CUDA / packaging) specificpriority:highHigh prioritystatus:backlogIn the backlog, not yet readytype:choreMaintenance tasks (build, CI, etc.)

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions