[davidtai]

Proposed changes

This PR adds C++ line coverage to the Fork Tests job (.github/workflows/fork-tests.yml). There is no coverage floor.

• Job-level CFLAGS and CXXFLAGS add -fprofile-instr-generate -fcoverage-mapping. Both builds in build-macos (the wheel and the C++ tests) run cmake from scratch, and cmake reads these variables. The flags cannot go through the cmake arguments, because setup.py splits CMAKE_ARGS at spaces.
• The test step sets LLVM_PROFILE_FILE to ${{ runner.temp }}/profiles/%p-%m.profraw. Each test process writes one profile for each instrumented binary. A job-level env cannot read the runner context (actionlint: context "runner" is not allowed here), so the test step sets the path. The build steps then write no profiles into the report.
• After the tests, xcrun llvm-profdata merge merges the profiles. xcrun llvm-cov report reads ./build/tests/tests, the installed extension (mlx/core.cpython-312-darwin.so) and the libraries in mlx/lib (libmlx.dylib, libjaccl.dylib). The Python tests run the library code in libmlx.dylib, not in the extension. -ignore-filename-regex drops /tests/, /_deps/, /python/, /.venv/ and /site-packages/.
• The job summary gets a small table with the totals for lines, functions and regions. The full report for each file is uploaded as the coverage-report artifact with actions/upload-artifact v7.0.1 (the SHA that build_and_test.yml already uses).
• The report steps also run after a test failure, so that a red run shows the coverage of the tests that ran. The summary shows the result of the test step.

Metal shaders cannot be covered. The .metal kernels run on the GPU, and clang coverage instruments only the host C and C++ code.

Runner probe result

Run on the head commit: https://github.com/Layr-Labs/mlx/actions/runs/36183840975. The job is red because of the C++ failure that #24 fixes.

• Coverage of the mlx sources from the C++ tests only: lines 43.21% (22915 of 53031), functions 51.12% (1932 of 3779), regions 40.05% (10424 of 26028), branches 37.61% (6014 of 15991). The Python tests did not run, because the C++ failure stops the test step.
• The artifact coverage-report has the full report for each file: https://github.com/Layr-Labs/mlx/actions/runs/36183840975/artifacts/10885213482
• llvm-cov warns that 185 functions have mismatched data. The C++ test build and the wheel compile these functions differently. llvm-cov uses the record that matches the profile.

Step (minutes)	Without instrumentation (runs 36172106158, 36174742752)	With instrumentation (runs 36181933249, 36183840975)
Build (wheel + C++)	14.61, 13.47	13.53, 12.95
C++ tests	0.63, 0.69	0.89, 0.82
Coverage report and upload	–	0.17, 0.12

Read: the build is not measurably slower, the C++ tests take about 0.2 minutes more, and the report steps take about 0.15 minutes. The cost for the Python tests shows in the first run after #22 is updated from main with #24.

Test plan

• actionlint .github/workflows/fork-tests.yml: no findings.
• actionlint on a test workflow with LLVM_PROFILE_FILE: ${{ runner.temp }}/... in a job-level env: context "runner" is not allowed here. This is why the test step sets the path.
• /usr/bin/python3 scripts/check_forkdiff.py --upstream-ref refs/remotes/upstream/main: 173 of 173 changed files described.
• Fork Tests on the first commit: https://github.com/Layr-Labs/mlx/actions/runs/36181933249. The report counted the generated files in the build folders and the uv Python headers. The second commit drops them.
• Fork Tests on the head commit: https://github.com/Layr-Labs/mlx/actions/runs/36183840975. The report step and the artifact upload pass. The job is red because of the C++ failure that fix(metal): build the GPT-OSS MXFP4 prefill tile into the metallib #24 fixes.
• Build and Test (https://github.com/Layr-Labs/mlx/actions/runs/36183841039) and Fork Diff Check (https://github.com/Layr-Labs/mlx/actions/runs/36183840920) pass on the head commit.
• gh api repos/Layr-Labs/mlx/commits/<sha> --jq .commit.verification.verified gives true for both commits.

CONTRIBUTING checklist

• Formatting: pre-commit ran on the head commit. The Check Lint job passed in run https://github.com/Layr-Labs/mlx/actions/runs/36183841039.
• Tests: this PR adds no library code. The workflow runs in the Test plan are the test.
• Benchmarks: this PR adds no library code. The instrumentation applies only to the CI build. No benchmark is needed.

Dependencies

The base branch is ci/fork-tests (#22). This PR does not include #24. #24 merges into main, and #22 is then updated from main. Until then, the C++ test gptoss mxfp4 prefill tiles preserve expert boundaries and partial rows fails and stops the test step, so the Python tests do not run. This PR runs again after that update. Merge order: #24 into main, #22 updated from main, this PR into ci/fork-tests, then #22 into main.

• ☐ I understand it is strictly prohibited to use AI to write PR description
• AI usage disclosure: this change and this description were written by Claude (Opus 5.5) under David Tai's direction and review.

🤖 Generated with Claude Code

[jonathan308]

Review follow-up for head 35ad1c9: before approval, please bring in the validated #22 stack including #24, update/retarget this PR as needed, and link a green instrumented Fork Tests run on the resulting head.

The current metal-build-and-test check is red. Please verify that all test stages complete and attach the coverage report from that successful run; a report produced after a partial failed run should not be used as the baseline.

The coverage-reporting approach itself is reasonable. For the later 80% initiative, keep the denominator explicit: this measures instrumented host code, not Metal shader coverage. I am not asking for an 80% threshold in this enabling PR.

[davidtai]

Reply written by Claude (agent) on David Tai's behalf.

Agreed. This PR stays stacked on ci/fork-tests and reruns after #22 is updated from main with #24 in; the coverage baseline is then taken from that complete run, not from the partial one. The denominator is instrumented host code (C++ and the Python extension); Metal shaders are outside it, and the body says so. No threshold is proposed here.