Skip to content

CI has no Metal build, so GPU-backend crashes like #998 cannot be caught before release #1002

Open

Description

@silverstein

Split out of #1001, which fixes a SIGABRT on interrupt that CI structurally cannot catch.

Nothing in the CI matrix builds with metal. ci.yml builds and tests minutes-core across diarize, streaming, parakeet, engine-sherpa and vad-ort, but never the GPU backends. So a crash that only exists when ggml's Metal backend is initialized cannot reproduce on any runner we have, and no test in a PR touching that path would fail if the fix regressed.

That is how #998 reached a user: the abort is deterministic on an Apple Silicon machine with --features metal, and invisible to us.

What would actually guard it: a small macOS job on the macos-latest runner that

  • builds the CLI with --features metal
  • runs minutes watch against a fixture long enough to still be decoding
  • sends SIGINT mid-decode
  • asserts the process exited normally rather than on a signal, i.e. exit status is not 134 and stderr carries no GGML_ASSERT

The assertion needs to be on the observable exit status, not on whether the code calls a particular exit helper. A test that checks we call exit_without_cxx_teardown would pass even if the teardown started aborting somewhere new.

Cost is a whisper tiny model download plus one short decode, so it can run on a path filter for changes under crates/cli, crates/core or the whisper dependency rather than on every PR.

Worth noting the same blind spot covers the other GPU backends: cuda, hipblas and vulkan are equally untested, and the residency-set teardown has analogues there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions