Skip to content

ci(nightly): reduce ARM test matrix to representative subset - #246

Merged
rm3l merged 1 commit into
redhat-developer:mainfrom
rm3l:chore/reduce-nightly-arm-matrix
Jun 19, 2026
Merged

rm3l merged 1 commit into
redhat-developer:mainfrom
rm3l:chore/reduce-nightly-arm-matrix

Conversation

@rm3l

@rm3l rm3l commented Jun 19, 2026 •

Copy link
Copy Markdown
Member

Description

ARM tests verify architecture compatibility, not functional differences that vary by compose config or container tool. Running the full matrix on ARM (~76 jobs) is unnecessary.

This reduces the ARM nightly matrix to a representative subset per branch:

  • Container tool: docker only
  • Compose configs: default + orchestrator-workflow only
  • User config: false only

This brings ARM jobs down from ~76 to ~8 (2 per branch). The full x86 matrix is unchanged.

Which issue(s) does this PR fix or relate to

N/A

PR acceptance criteria

  • Tests updated and passing
  • Documentation updated
  • Built-in TechDocs updated if needed. Note that TechDocs changes may need to be reviewed by a Product Manager and/or Architect to ensure content accuracy, clarity, and alignment with user needs.

How to test changes / Special notes to the reviewer

ARM tests verify architecture compatibility, not functional differences
that vary by compose config or container tool. Keep only docker +
default/orchestrator + no user config per branch (~8 jobs instead of ~76).

Assisted-by: Claude
@rm3l
rm3l force-pushed the chore/reduce-nightly-arm-matrix branch from f6bdcbe to 347a973 Compare June 19, 2026 10:42
@rm3l rm3l changed the title chore(nightly): reduce ARM test matrix to representative subset ci(nightly): reduce ARM test matrix to representative subset Jun 19, 2026
@rm3l
rm3l marked this pull request as ready for review June 19, 2026 10:44
@rhdh-qodo-merge

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Reduce nightly ARM CI matrix to a representative subset
⚙️ Configuration changes ✨ Enhancement 🕐 10-20 Minutes

Grey Divider

Description

• Cut nightly ARM CI jobs by excluding non-essential matrix combinations.
• Keep ARM coverage focused on architecture compatibility (docker + key compose configs).
• Leave the full x86 nightly matrix unchanged.
Diagram

graph TD
  A["Nightly workflow"] --> B["Matrix strategy"] --> C["Nightly test job"]
  C --> D["x86 matrix (unchanged)"]
  C --> E["ARM matrix (reduced)"] --> F["Exclude: podman + userConfig=true + extra compose"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Split ARM into a dedicated job with an explicit include-only matrix
  • ➕ Clearer intent: ARM runs only the explicitly enumerated combos
  • ➕ Avoids maintaining a growing list of excludes as x86 matrix evolves
  • ➖ More YAML and duplication unless extracted into a reusable workflow/template
  • ➖ Requires restructuring job(s), slightly larger change footprint
2. Drive matrices from JSON and compute ARM subset programmatically
  • ➕ Centralizes matrix definition and reduces manual exclude churn
  • ➕ Easier to enforce invariants (e.g., docker-only on ARM)
  • ➖ Adds indirection and complexity to the workflow
  • ➖ Harder to read/debug than plain YAML matrices

Recommendation: The current approach (adding ARM-focused excludes) is a good minimal change that achieves the job-count reduction without restructuring the workflow. If the x86 matrix is expected to keep expanding, consider the dedicated ARM include-only job later to make the ARM subset self-documenting and less coupled to future matrix growth.

Files changed (1) +19 / -0

Other (1) +19 / -0
nightly.yamlExclude non-representative ARM matrix combinations in nightly CI +19/-0

Exclude non-representative ARM matrix combinations in nightly CI

• Adds explicit matrix exclusions for ubuntu-24.04-arm to keep ARM coverage to docker-only, no user config, and a limited compose-config subset. Includes inline comments documenting that ARM runs are for architecture compatibility rather than full configuration coverage.

.github/workflows/nightly.yaml

@rhdh-qodo-merge

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Informational

1. Brittle ARM exclusions 🐞 Bug ⚙ Maintainability
Description
The ARM matrix.exclude rules match on the full composeConfig object (including cliArgs), so
future changes to cliArgs can silently cause exclusions to stop applying and unexpectedly
re-expand the nightly ARM job count. This is a drift hazard because the same config values are
duplicated in both matrix.composeConfig and matrix.exclude.
Code

.github/workflows/nightly.yaml[R80-91]

+          - os: ubuntu-24.04-arm
+            composeConfig:
+              name: "corporate-proxy"
+              cliArgs: "-f compose.yaml -f compose-with-corporate-proxy.yaml"
+          - os: ubuntu-24.04-arm
+            composeConfig:
+              name: "dynamic-plugins-root"
+              cliArgs: "-f compose.yaml -f compose-dynamic-plugins-root.yaml"
+          - os: ubuntu-24.04-arm
+            composeConfig:
+              name: "developer-lightspeed"
+              cliArgs: "-f compose.yaml -f developer-lightspeed/compose.yaml"
Relevance

⭐ Low

PR #229 accepted identical full-object composeConfig in matrix.exclude; no history of rejecting
this brittleness/drift concern.

PR-#229

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The workflow defines composeConfig as an object (name + cliArgs) and then repeats those full
objects in the ARM exclude rules; this duplication is what creates the drift hazard if cliArgs
changes in one place but not the other.

.github/workflows/nightly.yaml[59-66]
.github/workflows/nightly.yaml[72-91]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The workflow excludes ARM combinations by duplicating full `composeConfig` objects (both `name` and `cliArgs`) inside `matrix.exclude`. This makes the exclusion rules fragile: if a compose config’s `cliArgs` is updated in `matrix.composeConfig` but not updated in the corresponding `exclude` entry, the exclusion will no longer match and ARM coverage will unintentionally grow.

### Issue Context
`matrix.composeConfig` is defined as an object list with `name` + `cliArgs`, and the new ARM reduction excludes unwanted configs by repeating those objects under `matrix.exclude`.

### Fix Focus Areas
- .github/workflows/nightly.yaml[59-91]

### Suggested fix approaches (pick one)
1) **Use YAML anchors/aliases** to define each compose config object once and reference it from both `composeConfig:` and `exclude:` (prevents drift).
2) **Split the matrix into scalar fields** (e.g., `compose_config_name` and `compose_cli_args`) so `exclude` can match only on `compose_config_name` and doesn’t need to duplicate the args.
3) If keeping the current structure, **add a strong comment** near `composeConfig` / `exclude` indicating that `cliArgs` changes must be mirrored in the ARM exclude entries (least robust).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@rhdh-qodo-merge rhdh-qodo-merge Bot added enhancement New feature or request Other labels Jun 19, 2026
@rm3l
rm3l merged commit d58ebe2 into redhat-developer:main Jun 19, 2026
19 checks passed
@rm3l
rm3l deleted the chore/reduce-nightly-arm-matrix branch June 19, 2026 10:52
@sonarqubecloud

Copy link
Copy Markdown

@rm3l

rm3l commented Jun 19, 2026

Copy link
Copy Markdown
Member Author

Merging to trigger the nightly workflow manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Other

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant