ci: cut Docker test job from 40m to 25m by building a lean distribution

[FrankChen021]

Summary

Cut the Docker test job from 39m41s to 25m06s by building only what the Docker image needs, and move the release-only validation it used to perform into the packaging-check job, which is off the CI critical path.

Observed Docker baseline

Baseline is the master Docker job in workflow run 34300307586, job 102305613946, at commit cc8b586. The job completed successfully in 39m41s:

Step	Duration
Build the Druid distribution	18m20s
Build the Docker image	1m29s
Run Docker tests	19m08s

Attributing the Maven goals in the distribution-build log to CPU time (45 CPU-minutes on a 4-vCPU runner):

Goal	CPU time	Needed for the Docker image?
cyclonedx:makeBom (118 modules)	13.1 min	No
javadoc:jar (79 modules, from apache-release)	7.7 min	No
exec:exec (of which generate-licenses-report in distribution: 229s)	6.5 min	No, source release only
checkstyle:check	5.0 min	No
web-console npm install + build	4.7 min	Yes
assembly:single (binary 36s, source release 42s)	2.0 min	Binary only
compile + testCompile	2.7 min	Yes
apache-rat:check	1.5 min	No

The distribution module also ran alone for the last 5m36s of the build (three of four cores idle), almost entirely in the apache-release-only generate-licenses-report execution and the source-release assembly.

In the test step, -pl embedded-tests -am re-ran 38 upstream modules (including web-console, ~2 minutes) before the first test started.

All of the removed work is already validated on the same commit by the Static Checks CI workflow: packaging-check builds the distribution with -Prat -Pdist -Pbundle-contrib-exts and runs checkstyle, pmd, forbiddenapis, animal-sniffer, enforcer and cyclonedx; static-checks-maven runs the same static checks plus spotbugs. The only checks that had no other CI run were the apache-release-only ones: javadoc generation, source jars, the source-release assembly and the license dependency reports.

Changes

1. .github/workflows/docker-tests.yml: build the distribution inline with mvn -B -T1C clean install -pl '!benchmarks' -Pdist,bundle-contrib-exts,skip-static-checks,skip-tests -Dweb.console.skip=true, the same profiles the Dockerfile builder stage uses. benchmarks is excluded since nothing in the distribution or the Docker tests depends on it (no measurable time saving; it ran on an idle thread). The web console assets are skipped: no Docker test uses the console, the router tolerates the missing assets (the unit-test shards already run embedded routers with web.console.skip=true), and packaging-check still builds and packages the console with -Dweb.console.skip=false on every commit. web-console was the critical path of the lean build (3m49s of 5m04s), so this should bring the step to roughly 4 minutes. The image under test therefore differs from the shipped image only in lacking the console assets. Drops apache-release, rat and all static checks from the Docker job. .github/scripts/build-dist is deleted: the Docker workflow was its only caller and the name suggested a general-purpose release build.
2. .github/scripts/packaging-check.sh: add -Papache-release -Dgpg.skip -Ddependency-check.skip to both Maven invocations, so javadoc, source jars, the source-release assembly and the license dependency reports keep a CI run. This job is not on the critical path of its workflow.
3. .github/scripts/static-checks-maven.sh: replace license_checks_script.sh with a standalone repo-wide apache-rat:check -Prat. RAT stays here because packaging-check excludes benchmarks from its reactor; the license dependency report generation and check-licenses.py are now covered by packaging-check via apache-release. Also remove the duplicate distribution install.
4. .github/scripts/run_docker-tests: drop -am. The distribution build step installs every reactor module on the fresh runner and no Maven build cache is persisted across jobs, so embedded-tests resolves its dependencies (including the test-scoped web-console) from ~/.m2/repository. If a persisted build cache is ever reintroduced, -am must come back or maven-install-plugin must be forced via runAlways.
5. .github/scripts/openrewrite.sh: remove the duplicate distribution install.

The only workflow change is the inlined build step; no caching or job-graph changes.

Distribution-install cleanup rationale

The removed commands in static-checks-maven.sh and openrewrite.sh are:

mvn -B install -q -ff -pl distribution -P skip-tests -Dweb.console.skip=true ...

They only install the default distribution project without the packaging profiles, and neither script consumes the result. packaging-check.sh remains the job that builds and validates the distribution.

Measured outcome

PR head 05d007f, Unit & Integration run 34428720842 and Static Checks run 34428720600:

Job / step	Baseline	This PR	Δ
Docker: Build the Druid distribution	18m20s	2m46s	−15m34s
Docker: Build the Docker image	1m29s	1m05s	−24s
Docker: Run Docker tests	19m08s	17m15s	−1m53s (Maven overhead before first test 2m00s → 13s)
Docker job total	39m41s	21m47s	−17m54s
packaging-check (now runs apache-release)	13m27s	19m00s	+5m33s
static-checks-maven (license reports removed, RAT kept)	32m06s	27m27s	−4m39s
Static Checks CI workflow (longest job)	32m06s	27m27s	−4m39s

All 107 Docker tests pass (19 skipped, as on master). The Docker job is no longer the longest job in the Unit & Integration workflow; the unit-test shards (~35–37 min) are now the critical path.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Reduces Docker CI job time by introducing a dedicated Maven build-output cache and avoiding redundant Maven reactor rebuild work during Docker test runs.

Changes:

• Restore/save Maven build-cache outputs (~/.m2/build-cache) in the Docker tests workflow, saving only on trusted master pushes.
• Remove -am from the Docker test Maven invocation to avoid rebuilding upstream reactor modules after build-dist.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/docker-tests.yml	Adds restore/save steps for a separate Maven build-output cache around build-dist.
.github/scripts/run_docker-tests	Removes -am from the Maven invocation and documents reliance on artifacts installed by build-dist.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[FrankChen021]

Severity	Findings
P0	0
P1	1
P2	0
P3	0
Total	1

Reviewed 2 of 2 changed files.

Static new-review mode. One high-confidence cache/lifecycle finding is attached inline.

---

This is an automated review by Codex GPT-5.6-Luna(max)

[FrankChen021]

Severity	Findings
P0	0
P1	1
P2	1
P3	0
Total	2

Severity	Findings
P0	0
P1	1
P2	1
P3	0
Total	2

Reviewed 6 of 6 changed files.

Updated-since-review mode: the current head was checked from the incremental diff, then against the full diff and surrounding Maven/CI configuration. Two concrete findings are attached inline.

---

This is an automated review by Codex GPT-5.6-Luna(max)

[FrankChen021]

I have reviewed the updated code for correctness, edge cases, concurrency, security, and integration risks; no new issues found.

The incremental changes remove the persisted Docker build cache, keep the Docker test invocation dependent on the preceding full reactor install, and restore repo-wide RAT coverage including benchmarks. The prior review findings are addressed.

Reviewed 5 of 5 changed files.

---

This is an automated review by Codex GPT-5.6-Luna(max)

[FrankChen021]

🟢 Approval recommended

No actionable issues found in this review. The lean Docker build still installs the reactor artifacts required by embedded-tests, and the skipped console and benchmarks are not consumed by the Docker image or its tagged tests. The release-only and license checks remain covered by the packaging-check and static-checks jobs.

Reviewed 6 of 6 changed files.

Validation: gh pr checks 20297 --repo apache/druid passed for head 05d007f0e174855cd32e9af907199dc3d569d938; no local build or broad test run was performed.

---

This is an automated review by Codex GPT-5.6-Luna(max)