MCO-1961: Allow multiple machine-os versions by sdodson · Pull Request #2157 · openshift/oc

sdodson · 2025-12-01T18:02:30Z

No description provided.

openshift-ci · 2025-12-01T18:02:34Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

coderabbitai · 2025-12-01T18:02:36Z

Walkthrough

Allow the "machine-os" component to differ across tags: loadImageStreamTransforms now ignores version conflicts for the machine-os component when merging component version annotations, and readComponentVersions excludes machine-os from the "multiple versions" tracking in both construction and enrichment passes.

Changes

Cohort / File(s)	Summary
machine-os exception (merge + read) `pkg/cli/admin/release/image_mapper.go`, `pkg/cli/admin/release/info.go`	`image_mapper.go:loadImageStreamTransforms` skips the single-version conflict error for component key `"machine-os"` when merging annotations. `info.go:readComponentVersions` omits `"machine-os"` from the `multiples` set in both the initial build and enrichment passes, preventing it from triggering "multiple versions/display names" handling.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

sdodson · 2025-12-02T20:44:26Z

/test images unit

openshift-ci-robot · 2025-12-03T03:36:48Z

@sdodson: This pull request references MCO-1961 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ardaguclu · 2025-12-03T04:14:46Z

 			if ok {
-				if existing.Version != v.Version {
+				// we allow multiple machine-os versions due to dual stream efforts
+				if existing.Version != v.Version && k != "machine-os" {


This disables component version skew check entirely for machine-os component. Would it be better if we only allow 2 different machine-os versions and fail if there are 3 (to prevent any divergence) as we did #1662 (and #1656).

Additionally, same changes need to go Hypershift https://github.com/openshift/hypershift/blob/02f528cd5364b03168500ada33c9a0d9d0593025/support/releaseinfo/releaseinfo.go#L185

Thanks for pointing out the hypershift use case. For the time being we're simply dropping the machine-os version from the rhel-coreos-10 images but I think we need to figure out how to effectively re-introduce this. Ideally for 4.22 but perhaps deferred to the next release after 4.22. I'm going to close this for the time being but open up a ticket for the RHCOS team to dive into this across the problem space.

Limiting to two values here will probably backfire later because we may have more streams coming up : the nvidia kernel, confidential clusters.. Now that this can of worms is open, more may even be added

tchap · 2025-12-03T12:09:36Z

We should also add some tests BTW.

sdodson · 2025-12-05T17:12:22Z

/close
for now, RHCOS team to take a ticket to chase this across the problem space to ensure that we can have multiple unique versions for RHCOS in the release payload.

openshift-ci · 2025-12-05T17:12:57Z

@sdodson: Closed this PR.

Details

In response to this:

/close
for now, RHCOS team to take a ticket to chase this across the problem space to ensure that we can have multiple unique versions for RHCOS in the release payload.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

jbtrystram · 2026-03-11T20:51:08Z

Picking that up.
So i did build a payload with two machine-os images, and with this patch, oc does not complain.

oc adm release info quay.io:443/jbtrystramtestimages/rhel-coreos:4.22.0-ec.3-full-payload                                
Name:           4.22.0-ec.3
Digest:         sha256:d6514d0ec05b7e09e8f0dedcc68fe23d3afdd3a899dba391258210cc5399efa2
Created:        2026-03-11T20:08:31Z
OS/Arch:        linux/amd64
Manifests:      859
Metadata files: 2

Pull From: quay.io:443/jbtrystramtestimages/rhel-coreos@sha256:d6514d0ec05b7e09e8f0dedcc68fe23d3afdd3a899dba391258210cc5399efa2

Release Metadata:
  Version:  4.22.0-ec.3
  Upgrades: 4.21.0, 4.21.1, 4.21.2, 4.21.3, 4.21.4, 4.22.0-ec.0, 4.22.0-ec.1, 4.22.0-ec.2
  Metadata:

Component Versions:
  kubectl          1.34.1        
  kubernetes       1.34.2        
  kubernetes-tests 1.34.1        
  machine-os       10.2.20260311 Red Hat Enterprise Linux CoreOS

Images:
  NAME                                           DIGEST
  agent-installer-api-server               
<...> 
  rhel-coreos                                    sha256:42e2e4d11674c603398469770700ef5a30432294ffa245ed1c67852e93fc380e
  rhel-coreos-10                                 quay.io:443/jbtrystramtestimages/rhel-coreos:4.22-10.2-tagged
  rhel-coreos-10-extensions                      sha256:ee90288a9dfc801da76f890438957d38702c09c94ecaaaeef0f56b5cd2eb19a9
  rhel-coreos-extensions                         sha256:b7d9bb1dee4383e0e64baaabb915c194deebf9cbe19726aaea7e3498eaa2c59e
<...>

Warnings:
* multiple versions or display names reported for the following component(s): machine-os

sdodson · 2026-03-17T19:04:04Z

/reopen
@ardaguclu We're now looking at potentially having multiple streams which means we'll have N number of images labeled with machine-os version. Can you think of any issues that will arise in product if we allow that?

Thanks @jbtrystram for testing this out, I was never able to get to that point.

openshift-ci · 2026-03-17T19:04:27Z

@sdodson: Reopened this PR.

Details

In response to this:

/reopen
@ardaguclu We're now looking at potentially having multiple streams which means we'll have N number of images labeled with machine-os version. Can you think of any issues that will arise in product if we allow that?

Thanks @jbtrystram for testing this out, I was never able to get to that point.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2026-03-17T19:04:30Z

@sdodson: This pull request references MCO-1961 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ardaguclu · 2026-03-18T07:06:40Z

/reopen @ardaguclu We're now looking at potentially having multiple streams which means we'll have N number of images labeled with machine-os version. Can you think of any issues that will arise in product if we allow that?

Thanks @jbtrystram for testing this out, I was never able to get to that point.

It is difficult to see the implications of this change (maybe there isn't any). But machine-os is critical parts of this releasing process

oc/pkg/cli/admin/release/new.go

Lines 55 to 58 in 05fa7bb

    
           // We strongly control the set of allowed component versions to prevent confusion 
        
           // about what component versions may be used for. Changing this list requires 
        
           // approval from the release architects. 
        
           AllowedComponents: []string{"kubernetes", "machine-os", "kernel", "crio", "kubectl", "kubernetes-tests"},

,

oc/pkg/cli/admin/release/info.go

Line 2113 in 05fa7bb

    
           func (o *InfoOptions) describeRpmDiff(releaseInfo *ReleaseInfo, diff *ReleaseDiff, cacheDir, format, targetImage string) error {

In my opinion, by allowing any arbitrary stream we would permanently lose the control. Would it be too difficult to embed all possible streams in oc and do not allow the others?. That would bring about some maintenance burden, but at least we intentionally update the list.

ardaguclu · 2026-03-18T07:08:23Z

Also cc'ing @wking @bradmwilliams (please ignore if you are not related to this topic)

sdodson · 2026-03-18T15:44:29Z

In my opinion, by allowing any arbitrary stream we would permanently lose the control. Would it be too difficult to embed all possible streams in oc and do not allow the others?. That would bring about some maintenance burden, but at least we intentionally update the list.

In the past few months its become clear that we're likely to see a proliferation of streams. It's probably right on the tipping point as to whether or not it will be manageable to do this with a lot of cross repo coordination, right now the MCO, Installer, and RHCOS are all oriented toward dynamic discovery so that we don't have to coordinate.

jbtrystram · 2026-03-24T10:30:02Z

@sdodson here is a patch to remove the warning when creating the payload :

From 81fa5a71b3bbfb1eda6c001df6634f90d4743afc Mon Sep 17 00:00:00 2001
From: jbtrystram <jbtrystram@redhat.com>
Date: Tue, 24 Mar 2026 11:26:34 +0100
Subject: [PATCH] release/info: allow multiple machine-os versions

Rhcos will now come in two flavors : rhel-9 base and rhel-10. We remove
the warning when more than one version is reported.
Let's not put a limit here as  we're likely to see a proliferation of
streams with nvidia and confidential clusters comming up.

Assisted-by: OpenCode.ai <Opus 4.6>
---
 pkg/cli/admin/release/info.go | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/pkg/cli/admin/release/info.go b/pkg/cli/admin/release/info.go
index 2367751d1..ff9b60718 100644
--- a/pkg/cli/admin/release/info.go
+++ b/pkg/cli/admin/release/info.go
@@ -1058,7 +1058,8 @@ func readComponentVersions(is *imageapi.ImageStream, errOut io.Writer) (Componen
        sort.Strings(keys)
        for _, k := range keys {
                v := combined[k]
-               if v.Len() > 1 {
+               // we allow multiple machine-os versions due to dual stream efforts
+               if v.Len() > 1 && k != "machine-os" {
                        multiples = multiples.Insert(k)
                }
                if _, ok := out[k]; ok {
@@ -1075,7 +1076,8 @@ func readComponentVersions(is *imageapi.ImageStream, errOut io.Writer) (Componen
                if !ok {
                        continue
                }
-               if v.Len() > 1 {
+               // we allow multiple machine-os display names due to dual stream efforts
+               if v.Len() > 1 && k != "machine-os" {
                        multiples = multiples.Insert(k)
                }
                version, ok := out[k]
-- 
2.53.0

ardaguclu · 2026-03-24T11:34:07Z

I'll defer the decision to @sdodson about relaxing the validation

sdodson · 2026-03-24T13:34:44Z

@sdodson here is a patch to remove the warning when creating the payload :

Thanks, I wasn't able to git apply the patch, not sure why. But I've applied the changes. PTAL

coderabbitai

🧹 Nitpick comments (1)

pkg/cli/admin/release/info.go (1)
1062-1062: Consider extracting the "machine-os" string to a constant.

The literal "machine-os" is used in multiple locations across this file and image_mapper.go. A shared constant would reduce the risk of typos and make future updates easier.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/cli/admin/release/info.go` at line 1062, The literal "machine-os" is
duplicated; define a shared constant (e.g., MachineOSKey or MACHINE_OS) and
replace all occurrences of the string in this file and in image_mapper.go
(locations like the conditional using k == "machine-os" and any map lookups or
key comparisons) to use that constant; update imports/visibility as needed so
both files can reference the constant (place it in a common package or exported
const) and run tests to ensure no typos remain.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/cli/admin/release/info.go`:
- Line 1062: The literal "machine-os" is duplicated; define a shared constant
(e.g., MachineOSKey or MACHINE_OS) and replace all occurrences of the string in
this file and in image_mapper.go (locations like the conditional using k ==
"machine-os" and any map lookups or key comparisons) to use that constant;
update imports/visibility as needed so both files can reference the constant
(place it in a common package or exported const) and run tests to ensure no
typos remain.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 29b7d6bd-fc19-48f8-8a64-638452406180

📥 Commits

Reviewing files that changed from the base of the PR and between 0ef0058 and 70c6016.

📒 Files selected for processing (2)

pkg/cli/admin/release/image_mapper.go
pkg/cli/admin/release/info.go

sdodson · 2026-03-24T13:52:03Z

I'd like to move forward with this so we can make sure that this works properly in time for branch cut.

ardaguclu · 2026-03-24T13:55:05Z

/lgtm

jbtrystram

/lgtm

openshift-ci · 2026-03-24T14:30:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ardaguclu, jbtrystram, sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~pkg/cli/admin/release/OWNERS~~ [ardaguclu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ardaguclu · 2026-03-24T19:24:43Z

/retest

openshift-ci · 2026-03-24T21:37:16Z

@sdodson: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

sdodson · 2026-03-24T21:56:43Z

/verified later

openshift-ci-robot · 2026-03-24T21:56:47Z

@sdodson: /verified later <@username> requires at least one GitHub @username to be specified (it can be a comma delimited list). It indicates the engineer(s) that will be performing the verification. See https://docs.ci.openshift.org/docs/architecture/jira/#premerge-verification for more information.

Details

In response to this:

/verified later

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

sdodson · 2026-03-24T21:56:56Z

/verified later @sdodson

openshift-ci-robot · 2026-03-24T21:57:08Z

@sdodson: This PR has been marked to be verified later by @sdodson.

Details

In response to this:

/verified later @sdodson

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Since we merged openshift/oc#2157 `oc` supports multiple `machine-os` images, so we can write those labels. See openshift/oc#2157

sdodson · 2026-03-28T14:23:48Z

/cherry-pick release-4.21

openshift-cherrypick-robot · 2026-03-28T14:24:30Z

@sdodson: new pull request created: #2244

Details

In response to this:

/cherry-pick release-4.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Allow multiple machine-os versions in component version checks to support dual stream efforts. This mirrors the same change in openshift/oc#2157. Refs: openshift/oc#2157 Signed-off-by: Bryan Cox <brcox@redhat.com> Commit-Message-Assisted-by: Claude (via Claude Code) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 1, 2025

sdodson mentioned this pull request Dec 1, 2025

NO-ISSUE: Add rhel-10-coreos{,-extensions} to image-references openshift/machine-config-operator#5448

Merged

sdodson force-pushed the main branch from 3f7907c to 3a73120 Compare December 2, 2025 20:31

sdodson force-pushed the main branch from 3a73120 to 0ef0058 Compare December 3, 2025 03:08

sdodson changed the title ~~Allow multiple machine-os versions~~ MCO-1961: Allow multiple machine-os versions Dec 3, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Dec 3, 2025

sdodson marked this pull request as ready for review December 3, 2025 03:36

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 3, 2025

openshift-ci Bot requested review from ardaguclu and tchap December 3, 2025 03:37

ardaguclu reviewed Dec 3, 2025

View reviewed changes

openshift-ci Bot closed this Dec 5, 2025

openshift-ci Bot reopened this Mar 17, 2026

MCO-1961: Allow multiple machine-os image versions

70c6016

coderabbitai Bot reviewed Mar 24, 2026

View reviewed changes

openshift-ci Bot assigned ardaguclu Mar 24, 2026

openshift-ci Bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 24, 2026

jbtrystram approved these changes Mar 24, 2026

View reviewed changes

openshift-ci Bot assigned jbtrystram Mar 24, 2026

openshift-ci-robot added verified-later verified Signifies that the PR passed pre-merge verification criteria labels Mar 24, 2026

sdodson merged commit 8d7f53e into openshift:main Mar 24, 2026
15 of 17 checks passed

jbtrystram added a commit to jbtrystram/rhel-coreos-config that referenced this pull request Mar 26, 2026

rhel-10.2: inject openshift labels

59bc948

Since we merged openshift/oc#2157 `oc` supports multiple `machine-os` images, so we can write those labels. See openshift/oc#2157

jbtrystram mentioned this pull request Mar 26, 2026

NO-JIRA: rhel-10.2: inject openshift labels coreos/rhel-coreos-config#225

Merged

openshift-cherrypick-robot mentioned this pull request Mar 28, 2026

[release-4.21] MCO-1961: Allow multiple machine-os versions #2244

Merged

bryan-cox mentioned this pull request Mar 28, 2026

OCPBUGS-81284: fix(release): allow multiple machine-os component versions openshift/hypershift#8101

Merged

4 tasks

neisw mentioned this pull request Mar 29, 2026

TRT-2599: Revert #7774 "CNTRLPLANE-2551: Hypershift integration for API Auth Config changes" openshift/hypershift#8099

Closed

Conversation

sdodson commented Dec 1, 2025

Uh oh!

openshift-ci Bot commented Dec 1, 2025

Uh oh!

coderabbitai Bot commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Uh oh!

sdodson commented Dec 2, 2025

Uh oh!

openshift-ci-robot commented Dec 3, 2025 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ardaguclu Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sdodson Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

jbtrystram Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

tchap commented Dec 3, 2025

Uh oh!

sdodson commented Dec 5, 2025

Uh oh!

openshift-ci Bot commented Dec 5, 2025

Uh oh!

jbtrystram commented Mar 11, 2026

Uh oh!

sdodson commented Mar 17, 2026

Uh oh!

openshift-ci Bot commented Mar 17, 2026

Uh oh!

openshift-ci-robot commented Mar 17, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ardaguclu commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ardaguclu commented Mar 18, 2026

Uh oh!

sdodson commented Mar 18, 2026

Uh oh!

jbtrystram commented Mar 24, 2026

Uh oh!

ardaguclu commented Mar 24, 2026

Uh oh!

sdodson commented Mar 24, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

sdodson commented Mar 24, 2026

Uh oh!

ardaguclu commented Mar 24, 2026

Uh oh!

jbtrystram left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Mar 24, 2026

Uh oh!

ardaguclu commented Mar 24, 2026

Uh oh!

openshift-ci Bot commented Mar 24, 2026

Uh oh!

sdodson commented Mar 24, 2026

Uh oh!

openshift-ci-robot commented Mar 24, 2026

Uh oh!

sdodson commented Mar 24, 2026

Uh oh!

openshift-ci-robot commented Mar 24, 2026

Uh oh!

Uh oh!

sdodson commented Mar 28, 2026

coderabbitai Bot commented Dec 1, 2025 •

edited

Loading

openshift-ci-robot commented Dec 3, 2025 •

edited by openshift-ci Bot

Loading

ardaguclu Dec 3, 2025 •

edited

Loading

openshift-ci-robot commented Mar 17, 2026 •

edited by openshift-ci Bot

Loading

ardaguclu commented Mar 18, 2026 •

edited

Loading