Skip to content

test(benchmarks): host-run tests write fixture executables into the live ladder evidence root via a hardcoded path #1529

Description

@DecisionNerd

Problem

benchmarks/tests/test_progressive_host_run.py:31 hardcodes WORK_PARENT = Path("/home/ubuntu/graphforge-ladder"), the same directory the real progressive host ladder writes its retained evidence to (clean-<sha>-evidence/, clean-<sha>-work/, hand repro dirs). test_progressive_run.py:1308 uses the same literal path.

The tests create that directory if missing (WORK_PARENT.mkdir(parents=True, exist_ok=True), line 111), open tempfile.TemporaryDirectory(dir=WORK_PARENT) inside it, and then call write_host_bundle(output, …) with output equal to the temporary directory itself. host_run_fixture.executables(output.parent) therefore writes three fixture executables (gf, certify, generator, contents bounded executable fixture) into the live ladder root, and nothing removes them.

Observed 2026-09-21 on OVHC-AGENCY: /home/ubuntu/graphforge-ladder has birth time 18:48:27 UTC and contains only those three 26-byte fixture files. Every evidence directory that was there at 17:48 UTC (cited by #1526 and by #1478 §12) is gone. This issue does not claim the tests deleted them; the directory was absent when the tests ran and they recreated it. The defect is that a unit test suite shares a path with, and writes into, the host's retained benchmark evidence root at all.

Why the path is hardcoded

The host controller's native filesystem admission requires an ext4/xfs/btrfs volume, and /tmp on this host is tmpfs, so tmp_path cannot be used directly. That justifies an ext4-backed scratch root, not the evidence root.

Acceptance criteria

  • Host-run tests take their scratch root from an environment variable (for example GF_LADDER_TEST_ROOT) with a default that is not the evidence root and is documented next to the ext4 requirement; the default must be a dedicated directory such as /home/ubuntu/graphforge-ladder-test.
  • No test writes outside its own temporary directory: write_host_bundle/executables receive a path inside the temporary directory, and a test asserts the scratch root's top level is unchanged after the suite runs.
  • test_progressive_run.py:1308 uses the same scratch-root mechanism; no literal /home/ubuntu/graphforge-ladder remains under benchmarks/tests/.
  • docs/development/perf-g500-ladder.md names the evidence root as retained data that tests and cleanup tooling must not touch.

Non-goals

Recovering the deleted evidence (tracked in #1530) or changing the ladder controller's own output layout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions