Skip to content

fix(server): make audio synthesis robust to MLX FFI exceptions (cxx std::terminate bypasses the panic backstop) #382

Closed

Description

@inureyes

Problem / Background

The audio synthesis and transcription forward paths (src/server/kokoro_tts.rs, src/models/kokoro/*, src/server/whisper_stt.rs) build MLX graphs through mlxcel-core ops that are declared in the cxx bridge (src/lib/mlxcel-core/src/lib.rs) as -> UniquePtr<MlxArray>, i.e. NOT Result. Only a few boundaries (for example try_eval) return Result.

When the underlying MLX C++ throws (shape mismatch, allocation failure, etc.) inside a non-Result cxx function, cxx hard-aborts the process via std::terminate. This is NOT a Rust panic, so it is NOT intercepted by the catch_unwind backstop in src/server/audio_worker.rs (run_guarded), and it cannot be caught regardless of the release panic setting (abort or unwind).

Consequence: an MLX FFI exception on the audio forward path aborts the whole server process. This is the dominant residual fault vector for the audio path and is the one class that the Rust-panic-class remediations cannot cover. The existing input guards (4096-char input cap, Kokoro-vocab-restricted g2p output, 510-token truncation, finite/positive speed, per-token frame counts clamped to [1,100], whitelisted voice names) make it largely unreachable today, but it remains the gap.

Proposed Solution

Make the MLX ops used on the audio forward path fallible at the FFI boundary so a C++ exception becomes a recoverable Err instead of a std::terminate. Options to evaluate:

  • (a) Add Result-returning variants of the specific ops used on the audio forward path (cxx converts the thrown C++ exception into Err), and thread them through the Kokoro/Whisper forward code. Scoped to the audio path.
  • (b) A broader mlxcel-core policy of Result-returning graph ops. Larger, cross-cutting, affects every model. Likely out of scope for a first pass.

Feasibility and the exact op surface to convert need assessment. This is a mlxcel-core FFI change, intentionally separated from the panic-safety issue #375 (which addresses only Rust-panic-class containment).

Acceptance Criteria

  • An MLX C++ exception raised on the audio synthesis or transcription forward path results in a structured per-request error, not a process abort, in a release build.
  • A regression test (or documented manual verification) demonstrates the behavior.
  • No measurable regression to the audio decode/throughput path from the added fallibility.

Reference

Surfaced during the analysis of #375 (panic-safety for audio synthesis). Relates to the #374 catch_unwind backstop and the op declarations in src/lib/mlxcel-core/src/lib.rs. See src/server/audio_worker.rs (run_guarded).

Activity

  1. added
    type:securitySecurity vulnerability or fix
    area:modelsModel architectures, weights, loading, metadata
    area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)
    status:backlogIn the backlog, not yet ready
    on Jun 21, 2026
  2. added this to the 0.4 milestone on Jun 21, 2026
  3. added a commit that references this issue on Jun 21, 2026
  4. self-assigned this
    on Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)area:modelsModel architectures, weights, loading, metadatapriority:lowLow prioritystatus:doneCompletedtype:securitySecurity vulnerability or fix

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions