Skip to content

compile_many_stage2_perf oracle: unrunnable since #800; with backend installed, stage-2 wall ~6.5s fails 50%-of-stage-1 bound — #337 seed regression or stale baseline #1346

Description

@zackees

Symptom

crates/fbuild-build/tests/compile_many_stage2_perf.rs (stage2_per_sketch_wall_is_a_fraction_of_stage1) has been unrunnable since #800: it builds through the process-wide compile backend, which only the daemon wires at startup, so any direct run hard-fails with:

BuildFailed(compile_backend not installed — fbuild-daemon must call
compile_backend::install_global at startup before any compile fires (FastLED/fbuild#800))

The sibling acceptance suites (teensylc/stm32/teensy30) install a test backend; this oracle never did, so the perf assertion has not actually executed since #800 landed.

What happens once the backend is installed

With a local install_test_compile_backend() helper added (same tokio::sync::OnceCell pattern as the acceptance suites — a second concurrent CompileBackend::start() cannot win the zccache cache-root writer slot), the oracle runs and fails the perf assertion:

stage 1 wall: 1.22s–1.36s
stage 2 wall: 6.48s–6.72s   [must be < ~0.61s–0.68s]
assertion: stage-2 sketch wall exceeded 50% of stage-1

Stage 2 is ~5× stage-1 wall — consistent with the framework archive not actually being seeded/reused, i.e. the regression the oracle was written to catch (the #337 seed not skipping the recompile), or a stale baseline (the machine/paths changed materially since the oracle last truly ran, pre-#800).

Why it can't be triaged as-is

  • Test binaries install no tracing subscriber, so the diagnostic compile-many stage 2 seed: linked N + copied N info line is invisible without extra wiring.
  • The panic message points at <sketch>/.fbuild/build/uno/release/compile_many.log, but the TempDir is dropped when the assert fires, deleting the log.

Suggested work

  1. Make the oracle diagnosable: preserve compile_many.log outside the tempdir on failure (e.g. copy to target/ or print tail into the panic message), and/or install a tracing subscriber in the test binary.
  2. Re-run and read the seed line: compile-many stage 2 seed: linked N + copied N. linked 0 + copied 0 (or missing) ⇒ seed regression; large N with still-slow stage 2 ⇒ the wall is going elsewhere (link/size/serialization).
  3. If the seed is genuinely broken, bisect against fbuild: seed stage-2 core/ from stage-1 to skip framework recompile (#335 architectural fix) #337; if the seed works but the wall regressed elsewhere, re-baseline the oracle's 50% bound with fresh numbers.

Context

Found while fixing the 7 integration-test files missing the #800 backend install (phase-8b PR for #1314). The other 6 files go green with the helper; this one is the only remaining red, and it is red for a real reason — the assertion is finally executing again.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions