Speed up scala CI job: drop redundant compile and reorder for early lint failure

[Yicong-Huang]

Task Summary

Trim redundant compile in the scala CI job and reorder steps so lint failures surface before the expensive dist + tests run.

Pain point

A successful scala CI run on main currently takes ~14 min 44 s. Step breakdown for run 25239784635:

step	duration
Install dependencies (pip)	1 m 21 s
Lint with scalafmt	45 s
Build distributable bundles (sbt 'clean; X/dist; ...')	3 m 4 s
Compile with sbt (sbt clean package)	1 m 26 s
Lint with scalafix	47 s
Run backend tests	6 m 36 s

Two concrete waste points and one ordering problem:

• Compile with sbt: sbt clean package runs after dist already compiled every module; it re-cleans the working tree and re-packages, and the resulting jars are not consumed by any subsequent step.
• Build distributable bundles prefixes its sbt invocation with clean, which is a no-op on the very first sbt build but discards earlier compile output if other sbt steps run first.
• scalafix lives at the bottom of the job. A bad scalafix lint costs the developer all 3 m 4 s of dist + a redundant 1 m 26 s compile before they see the error.

Proposal

Re-order the scala job's steps to put both linters in front so the cheap checks fail fast, then run dist (without clean) which reuses scalafix's compile output, then license/audit/test. Specifically:

1. Drop Compile with sbt: sbt clean package entirely.
2. Drop the leading clean; from Build distributable bundles.
3. Move Lint with scalafix to run before Build distributable bundles so its compile is reused by dist.
4. Move Install dependencies (pip) to run just before Run backend tests since only the tests need the python deps; lint and dist failures no longer pay the install cost.

Expected savings on a clean run: ~1 m 30 s (dropped redundant compile) plus a few seconds on the dropped clean and on the lint-failure path.

Priority

P2 – Medium

Task Type

• DevOps / Deployment