[inureyes]

Summary

make verify-test and make verify-clippy passed no package selector. The workspace root is itself the mlxcel package, so both resolved to -p mlxcel and neither built mlxcel-core, mlxcel-surgery or mlxcel-xla. Both targets now pass --workspace, verify-test also passes --no-fail-fast, and the lint debt that surfaces behind the wider scope is cleared.

Scope of the hole, measured

	test binaries	doctest targets	tests passed
root only (old gate)	76	1	5284
--workspace (new gate)	84	4	7100

The 8 skipped binaries are mlxcel-core's lib test, mlxcel-xla's lib test, and mlxcel-surgery's lib test plus its five tests/*_integration.rs. mlxcel-core alone carried 1354 of the missing tests, and it is the crate holding the MLX cxx bridge, layers.rs, the KV cache and the quantization loaders.

--workspace rather than default-members

default-members would have worked, and is rejected for two reasons.

It re-scopes every bare cargo invocation in the repository at once. release.yml runs cargo build --release --target aarch64-apple-darwin --locked with no selector, so a default-members covering all four would start compiling the default-off mlxcel-xla into every release build. scripts/run_quality_gate.sh has three more bare invocations.

It also moves the gate's scope into a manifest field that the gate's own command line does not mention. A scope you cannot read off the command is the defect this issue is about, so hiding it in Cargo.toml would be repeating the mistake somewhere new. The [workspace] stanza carries a comment recording the decision so the next person does not add one.

Feature resolution

cargo tree --workspace --features metal,accelerate -e features -i mlxcel-core resolves as follows, so this is not a complication:

member	resolved features
mlxcel (root)	default (surgery) + metal + accelerate
mlxcel-core	metal + accelerate, through the root's forwarding, one shared build
mlxcel-surgery	default (empty)
mlxcel-xla	default (empty); iree stays off

mlxcel-xla's iree staying off is what keeps the gate free of an IREE distribution: src/lib/mlxcel-xla/build.rs returns early on a missing CARGO_FEATURE_IREE rather than compiling the native shim. The code behind iree, diagnostics and micro-oracle therefore remains outside the gate, which the docs now say.

Concurrency, the #1008 question

--workspace puts the mlxcel-core suite and the root suite in one invocation, which is the arrangement that corrupts results in #1008. It does not reproduce, because cargo builds every test binary and then runs them one at a time.

Sampled every 0.5s across a full workspace run: 217 of 217 samples showed exactly one test binary running, across 75 distinct binaries, with mlxcel_core-cdfd4c46d62ff555 observed alone. Doctests are not a second path to it either: mlxcel-core executes 2 doctests and both are pure-CPU hardware::kv_cache_bytes*, and rustdoc merges them into one binary.

The no_other_mlxcel_core_test_binary_is_sharing_the_gpu guard would not have caught the root-suite case by itself, since it matches only a second mlxcel-core binary. The Makefile, docs/installation.md and CONTRIBUTING.md therefore record the serialization that a parallel test runner such as cargo nextest would have to re-establish.

--no-fail-fast

Only matters now that the run covers four members: without it the first failing test binary ends the run and hides the other three behind whatever failed first, which for a nightly that costs the better part of an hour is the wrong trade. Cargo still exits non-zero, so the gate is no weaker, and compile errors stop the run either way.

Lint debt cleared

Six clippy errors, all pre-existing, all in mlxcel-xla's lib-test target. mlxcel-core and mlxcel-surgery were already clean. The "7" in the issue thread reconciles as these six plus cargo's own error: could not compile mlxcel-xla (lib test) due to 6 previous errors summary line.

lint	site
unused_imports	emitter/mod.rs:157, the emit_vision_diagnostics re-export
clippy::manual_is_multiple_of	emitter/phi4_audio.rs:1003, emitter/qwen2_vl.rs:227, emitter/qwen2_vl.rs:458
clippy::identity_op	emitter/qwen2_vl.rs:827, a 48 * 64 + 0 index in a test
clippy::items_after_test_module	emitter/vision.rs:35

The unused import is allowed rather than deleted, and the #[cfg(any(test, feature = "diagnostics"))] gate kept, because that test arm is the only thing type-checking emit_vision_impl(config, true) in a build without diagnostics, and diagnostics implies iree, which needs an IREE dist and is therefore never on in CI. The four sibling re-export blocks above it already carry the same attribute for the same reason. The identity_op is fixed with a (row, column) accessor so the packed-bias assertions keep saying which cell they mean instead of degrading to a bare 48 * 64. The items_after_test_module fix moves the test module to the end of emitter/vision.rs with no change to its contents.

Timings, development machine

M1 Ultra, warm build, --profile test-fast --features metal,accelerate --no-fail-fast. These are development-machine numbers; the runner is slower and shared, and its measurement is a separate nightly dispatch that this PR does not trigger.

run	wall clock	result
cargo test --profile test-fast --features metal,accelerate --no-fail-fast	1m54s	5284 passed, 0 failed, 274 ignored
cargo test --profile test-fast --workspace --features metal,accelerate --no-fail-fast	2m21s	7100 passed, 0 failed, 290 ignored
delta	+27s (+23.7%)	+8 binaries, +1816 tests

Added build cost, isolated: deleting exactly the 8 member test executables and re-running --no-run rebuilt only the three members, root untouched, in 17.3s. A no-op --no-run is 0 to 1s at either scope.

So on a warm tree the whole change costs about 44s, split roughly 17s of build and 27s of execution. The runner's 48m54s is dominated by the root build, which is unchanged; the new members add 8 link steps out of 84. nightly-verify.yml's header records what to do if the budget ever stops fitting: split verify-test into per-member steps before dropping members from it.

Test plan

• make verify-fmt: pass (4s)
• make verify-clippy: pass (22s), workspace clean at -D warnings
• make verify-test: pass (217s), 84 binaries + 4 doctest targets, 7100 passed, 0 failed, 290 ignored
• Coverage demonstrated: the run lists mlxcel_core-…, mlxcel_surgery-…, mlxcel_xla-… and the five mlxcel-surgery integration binaries, plus Doc-tests for all four members
• Concurrency: 217/217 samples at one running test binary, mlxcel-core's observed alone
• Negative control, the criterion the issue asks for. A Debug-bound error planted in an mlxcel-core test, the fix(core): bound the declared quantization mode string, not just group_size and bits #973 defect class, then reverted:
  • old cargo clippy --all-targets --features metal,accelerate -- -D warnings: exit 0, passes with the error present
  • new make verify-clippy: exit 2, error[E0277]: NotDebug doesn't implement std::fmt::Debug at src/lib/mlxcel-core/src/streams.rs:289, could not compile mlxcel-core (lib test)
• python3 scripts/ci/check_cross_repo_refs.py clean; every #N in the diff verified against lablup/mlxcel
• No models/ path written, moved or read for write

Not applicable: no inference code changes, so no real-checkpoint validation. The 84-binary suite includes the model parity tests, which self-skip without weights.

Notes for reviewers

CLAUDE.md is a symlink to AGENTS.md and both are gitignored, so the equivalent update to the local agent contract is not in this diff.

The known-flaky granite4_vision_parity and ernie4_5_moe_vl_parity tests (#997) passed in all three full runs here.

Closes #1007

[inureyes]

Runner measurement

Run 30881430828, success, 88 binaries / 7100 passed / 0 failed.

step	before this PR (root only)	with --workspace
cargo fmt	4s	14s
cargo clippy	71s	2m28s
cargo test	48m54s	22m52s
job total	50m18s	25m49s
share of the 180-minute budget	27%	13%

Against the original --release baseline (run 30761020020, cargo test 2h54m34s at 97% of budget) the gate is now 7.6x faster while covering four workspace members instead of one, and 7100 tests instead of 5284.

These two columns are not a controlled A/B and should not be read as one. The before column is run 30871982545 on lablup-macmini01; this run landed on lablup-macmini13, a different machine with a different cache state. What the numbers do establish is the thing that matters for the acceptance criterion: the expanded gate finishes comfortably inside the budget, measured on the runner, on the machine that was least prepared for it.

Runner pool

This run is also the first successful one on lablup-macmini13. Three machines answer to self-hosted-macos-26-arm64 (macmini01, macmini12, macmini13) and until today only macmini01 was provisioned, which is why runs that looked like flaky infrastructure were really a coin flip over assignment. macmini12 and macmini13 have since had Homebrew, cmake, Xcode and the Metal toolchain installed. The Ensure build tools guard added in #1017 is what identified this: it printed xcode-select -p: /Library/Developer/CommandLineTools and an empty Xcode installs found: instead of letting cmake fail hundreds of lines later with a header error.

One caveat, tracked separately

A local full-workspace run hit drafter::tests::auto_detect_emits_info_log_for_default_fallback failing once under load (3 of 3 passing alone, 3 of 3 passing for the standalone core lib binary). Filed as #1023. It did not reproduce on this runner run. Together with #997 it means the widened gate has two known intermittent sources, and both should be resolved rather than tolerated: an intermittently red nightly gets ignored, and an ignored gate is the blind spot the nightly exists to close.