Skip to content

ci.yml's shard-file mapfile silently degrades to "run every test" if the shard-lookup one-liner throws #7767

Description

@JSONbored

Context

.github/workflows/ci.yml:951 reads a shard's file list via:

mapfile -t SHARD_FILES < <(node -e "console.log(JSON.parse(require('fs').readFileSync('shard-assignment.json','utf8'))['${{ matrix.shard }}'].join('\n'))")
npm run test:coverage -- --maxWorkers=4 "${SHARD_FILES[@]}" ...

This is the same subshell-swallow class as compose_file_args above, applied inside a GitHub Actions run: step (default bash -eo pipefail, confirmed this does not catch this pattern either). compute-test-shards.mjs:137 currently writes keys matching matrix.shard: [1,2,3], so this works today — but there is no test anywhere (checked test/unit/compute-test-shards.test.ts) asserting ci.yml's matrix.shard values stay in sync with this script's key format. If JSON.parse(...)['${{ matrix.shard }}'] is ever undefined (a shard-count change, a key-format change, or a corrupt/missing shard-assignment.json), .join('\n') throws inside node -e, node exits non-zero, and SHARD_FILES becomes an empty bash array — vitest then receives no explicit file filter, meaning "run every test file." Each of the 3 shard jobs would silently run the entire suite instead of its slice, defeating the load-bearing duration-aware sharding, with no error surfaced anywhere.

Requirements

Editing .github/workflows/ci.yml is a shared/sensitive file — keep the diff minimal and scoped to exactly this step, don't restructure other jobs. Replace the mapfile -t SHARD_FILES < <(node -e ...) idiom with one that propagates the node script's failure (checked command-substitution assignment, matching the fix pattern used for compose_file_args in the companion self-host-deploy issue), or add an explicit guard immediately after: [ "${#SHARD_FILES[@]}" -gt 0 ] || { echo "::error::empty shard file list for shard ${{ matrix.shard }}"; exit 1; }. Either approach is acceptable as long as a shard-lookup failure now fails the CI job loudly instead of silently running the full suite.

Deliverables

  • The shard-file lookup step in .github/workflows/ci.yml fails the job (not silently runs every test) when the node one-liner throws or returns an empty list.

Test Coverage Requirements

This is a workflow-YAML change, not covered by the Codecov patch gate — verify manually by simulating a bad matrix.shard value (or a temporarily-corrupted shard-assignment.json) locally and confirming the guard trips, per this repo's existing convention for workflow-level changes.

Expected Outcome

A shard-lookup failure in CI (bad matrix value, corrupt/missing shard-assignment file) fails that job explicitly instead of silently causing all 3 shards to redundantly run the entire test suite.

Links & Resources

.github/workflows/ci.yml:951, scripts/compute-test-shards.mjs:137 (the shard-key format this depends on staying in sync)

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions