Skip to content

CNTRLPLANE-2914: Backport controlPlaneVersion status to release-4.21 - #2

Open
devguyio wants to merge 5 commits into
release-4.21from
backport-controlplaneversion-release-4.21
Open

CNTRLPLANE-2914: Backport controlPlaneVersion status to release-4.21#2
devguyio wants to merge 5 commits into
release-4.21from
backport-controlplaneversion-release-4.21

Conversation

@devguyio

@devguyio devguyio commented Apr 7, 2026

Copy link
Copy Markdown

Summary

Cherry-pick of openshift#7887 (CNTRLPLANE-2914: Add controlPlaneVersion status to HostedCluster and HostedControlPlane) to release-4.21.

This backport adds the controlPlaneVersion status field to HostedCluster and HostedControlPlane, providing a management-side view of control plane component rollout status independent of the CVO.

Changes

  • Add ControlPlaneVersionStatus and ControlPlaneUpdateHistory API types
  • Implement reconcileControlPlaneVersion aggregation logic in CPO
  • Implement pruneHistory with CVO weighted ranking algorithm
  • Propagate controlPlaneVersion from HCP to HC status
  • Add WaitForControlPlaneRollout e2e helper (gated to 4.21+)
  • Add WaitForImageRollout deprecated alias for WaitForDataPlaneRollout
  • Comprehensive unit test coverage for API types, reconciliation, pruning, and propagation

Conflict Resolution

  • CRD manifest files deleted in release-4.21 (GCPPlatform, HCPEtcdBackup, ClusterUpdateAcceptRisks, ExternalOIDCWithUpstreamParity) were excluded
  • Configuration field in applyconfiguration types preserved (exists in 4.21, not in original feature branch base)
  • WaitForControlPlaneComponentRollout (existing 4.21 function) preserved alongside new WaitForControlPlaneRollout
  • Version gate adjusted from Version422 to Version421 for backport context

Note: This PR targets the fork's release-4.21 branch. A corresponding PR should be opened against openshift/hypershift release-4.21 once this is reviewed.

Ref: openshift#7887

@devguyio devguyio mentioned this pull request Apr 7, 2026
Closed
@devguyio
devguyio force-pushed the backport-controlplaneversion-release-4.21 branch from 683ebcb to 60f4e16 Compare April 7, 2026 14:05
@devguyio

devguyio commented Apr 13, 2026

Copy link
Copy Markdown
Author

Manual Testing Report — controlPlaneVersion status (release-4.21)

Environment:

  • Management cluster: OCP 4.22.0-0.nightly-multi-2026-02-16-045147 (Kubernetes v1.34.2)
  • Region: us-east-1
  • HO image: quay.io/abdalla/hypershift:backport-cpv-421 (built from backport-controlplaneversion-release-4.21 branch)
  • CPO image: quay.io/abdalla/hypershift:backport-cpv-421 (same dev all-in-one image, applied via hypershift.openshift.io/control-plane-operator-image annotation)

Test 1: Initial Cluster Creation (4.21.0)

Steps:

  1. Installed HO from PR branch
  2. Created HostedCluster cpv-test-421 with --release-image quay.io/openshift-release-dev/ocp-release:4.21.0-multi and --control-plane-operator-image quay.io/abdalla/hypershift:backport-cpv-421
  3. Waited for control plane availability

Result: PASS

  • HC reached Available=True, Progress=Completed
  • controlPlaneVersion status populated on both HC and HCP: 
    {
  "desired": {"image": "quay.io/openshift-release-dev/ocp-release@sha256:02c5f4...", "version": "4.21.0"},
  "history": [{"state": "Completed", "version": "4.21.0", "startedTime": "12:21:55Z", "completionTime": "12:28:41Z"}],
  "observedGeneration": 1
}
  • CP Version column correctly shows 4.21.0 via new CRD printer column

Test 2: CRD Printer Columns

Steps:

  1. Verified new printer columns from PR are present in live CRD

Result: PASS (with caveat)

  • New columns present: CP Version, CP Progress (priority=1), DP Progress (priority=1)
  • Caveat: oc apply (strategic merge) during hypershift install did NOT fully update printer columns from old CRDs. Required oc replace with the CRD manifest from the PR branch to get correct display. Without this, VERSION showed empty and PROGRESS showed Partial despite actual status being Completed.
  • After CRD replacement, display was correct.

Test 3: Upgrade with No Nodes (Partial State)

Steps:

  1. Scaled nodepool to 0 replicas
  2. Upgraded HC from 4.21.0 → 4.21.1

Result: PASS

  • HCP controlPlaneVersion.history correctly showed: 
    4.21.1 Partial   (rolling, no nodes to complete)
4.21.0 Completed
  • desired updated to 4.21.1
  • observedGeneration incremented

Test 4: Stacked Upgrade (4.21.1 → 4.21.2, Still No Nodes)

Steps:

  1. While 4.21.1 was still Partial, upgraded HC to 4.21.2

Result: PASS

  • History correctly accumulated 3 entries: 
    4.21.2 Partial
4.21.1 Partial
4.21.0 Completed
  • Pruning did not discard intermediate Partial entries

Test 5: Scale Up and Completion

Steps:

  1. Scaled nodepool back to 2 replicas
  2. Waited for nodes to join and CVO rollout to complete

Result: PASS

  • Nodes joined successfully (2/2)
  • 4.21.2 rollout completed
  • Final controlPlaneVersion.history: 
    4.21.2 Completed
4.21.1 Partial    (preserved — skipped upgrade)
4.21.0 Completed
  • HC display: 
    VERSION=4.21.2  CP VERSION=4.21.2  PROGRESS=Completed  AVAILABLE=True

Test 6: HC ↔ HCP Propagation

Result: PASS

  • controlPlaneVersion on HCP matched HC throughout all tests
  • desired, history, and observedGeneration propagated correctly

Summary

Test Result
Initial creation (4.21.0) ✅ PASS
CRD printer columns ✅ PASS (needs oc replace for existing clusters)
Upgrade with no nodes (Partial) ✅ PASS
Stacked upgrade (3 history entries) ✅ PASS
Scale up and completion ✅ PASS
HC ↔ HCP propagation ✅ PASS

Note on CRD update: When upgrading an existing HyperShift installation, hypershift install uses apply which does strategic merge on CRDs. The new printer columns may not take effect until the CRDs are fully replaced (e.g., via oc replace -f <crd-manifest>). This is a general HyperShift upgrade consideration, not specific to this PR.

@devguyio

devguyio commented Apr 13, 2026

Copy link
Copy Markdown
Author

🧪 qe — 2026-04-13T19:45:30Z

Manual Testing Report — controlPlaneVersion status (release-4.21)

Environment:

  • Management cluster: OCP 4.22.0-0.nightly-multi-2026-02-16-045147 (Kubernetes v1.34.2)
  • Region: us-east-1
  • HO image: quay.io/abdalla/hypershift:backport-cpv-421 (built from backport-controlplaneversion-release-4.21 branch)
  • CPO image: quay.io/abdalla/hypershift:backport-cpv-421 (same dev all-in-one image, applied via hypershift.openshift.io/control-plane-operator-image annotation)

Test 1: Initial Cluster Creation (4.21.0)

Steps:

  1. Installed HO from PR branch
  2. Created HostedCluster cpv-test-421 with --release-image quay.io/openshift-release-dev/ocp-release:4.21.0-multi and --control-plane-operator-image quay.io/abdalla/hypershift:backport-cpv-421
  3. Waited for control plane availability

Result: PASS

  • HC reached Available=True, Progress=Completed
  • controlPlaneVersion status populated on both HC and HCP: 
    {
  "desired": {"image": "quay.io/openshift-release-dev/ocp-release@sha256:02c5f4...", "version": "4.21.0"},
  "history": [{"state": "Completed", "version": "4.21.0", "startedTime": "12:21:55Z", "completionTime": "12:28:41Z"}],
  "observedGeneration": 1
}
  • CP Version column correctly shows 4.21.0 via new CRD printer column

Test 2: CRD Printer Columns

Steps:

  1. Verified new printer columns from PR are present in live CRD

Result: PASS (with caveat)

  • New columns present: CP Version, CP Progress (priority=1), DP Progress (priority=1)
  • Caveat: oc apply (strategic merge) during hypershift install did NOT fully update printer columns from old CRDs. Required oc replace with the CRD manifest from the PR branch to get correct display. Without this, VERSION showed empty and PROGRESS showed Partial despite actual status being Completed.
  • After CRD replacement, display was correct.

Test 3: Upgrade with No Nodes (Partial State)

Steps:

  1. Scaled nodepool to 0 replicas
  2. Upgraded HC from 4.21.0 → 4.21.1

Result: PASS

  • HCP controlPlaneVersion.history correctly showed: 
    4.21.1 Partial   (rolling, no nodes to complete)
4.21.0 Completed
  • desired updated to 4.21.1
  • observedGeneration incremented

Test 4: Stacked Upgrade (4.21.1 → 4.21.2, Still No Nodes)

Steps:

  1. While 4.21.1 was still Partial, upgraded HC to 4.21.2

Result: PASS

  • History correctly accumulated 3 entries: 
    4.21.2 Partial
4.21.1 Partial
4.21.0 Completed
  • Pruning did not discard intermediate Partial entries

Test 5: Scale Up and Completion

Steps:

  1. Scaled nodepool back to 2 replicas
  2. Waited for nodes to join and CVO rollout to complete

Result: PASS

  • Nodes joined successfully (2/2)
  • 4.21.2 rollout completed
  • Final controlPlaneVersion.history: 
    4.21.2 Completed
4.21.1 Partial    (preserved — skipped upgrade)
4.21.0 Completed
  • HC display: 
    VERSION=4.21.2  CP VERSION=4.21.2  PROGRESS=Completed  AVAILABLE=True

Test 6: HC ↔ HCP Propagation

Result: PASS

  • controlPlaneVersion on HCP matched HC throughout all tests
  • desired, history, and observedGeneration propagated correctly

Summary

Test Result
Initial creation (4.21.0) ✅ PASS
CRD printer columns ✅ PASS (needs oc replace for existing clusters)
Upgrade with no nodes (Partial) ✅ PASS
Stacked upgrade (3 history entries) ✅ PASS
Scale up and completion ✅ PASS
HC ↔ HCP propagation ✅ PASS

Note on CRD update: When upgrading an existing HyperShift installation, hypershift install uses apply which does strategic merge on CRDs. The new printer columns may not take effect until the CRDs are fully replaced (e.g., via oc replace -f <crd-manifest>). This is a general HyperShift upgrade consideration, not specific to this PR.

@devguyio-agent-smith
devguyio-agent-smith Bot force-pushed the backport-controlplaneversion-release-4.21 branch from 60f4e16 to d4cc7e5 Compare July 16, 2026 11:59
devguyio and others added 5 commits July 21, 2026 09:23
@devguyio @claude
feat(api): add ControlPlaneVersionStatus to track control plane compo…
149ba05 
…nent versions

Add ControlPlaneVersionStatus and ControlPlaneUpdateHistory types to
HostedClusterStatus and HostedControlPlaneStatus. These track
management-side control plane component versions independently from CVO,
enabling service providers to detect completed control plane upgrades
and compute NodePool version skew without waiting for data-plane rollout.

Ref: CNTRLPLANE-2914

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>
@devguyio @claude
chore: run make update to regenerate CRDs, deepcopy, clients, vendor,…
2fb5bd9 
… and docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@devguyio @claude
feat(cpo): reconcile controlPlaneVersion from ControlPlaneComponent s…
0502504 
…tatus

Implement reconcileControlPlaneVersion() that aggregates
ControlPlaneComponent status into hcp.Status.ControlPlaneVersion with
version transition logic, image-only change detection, and
observedGeneration tracking.

Include pruneHistory() ported from CVO's weighted ranking algorithm
with exact constants to cap history at 100 entries.

Ref: CNTRLPLANE-2914

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>
@devguyio @claude
feat(hc): propagate controlPlaneVersion from HCP to HC status
60675d4 
Deep-copy controlPlaneVersion from HostedControlPlane status to
HostedCluster status, following the existing Platform field propagation
pattern. Handles nil gracefully for version skew safety during rolling
upgrades.

Ref: CNTRLPLANE-2914

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>
@devguyio @claude
chore(make): extract api-lint and api-lint-fix targets
74d1d3a 
Extract the api/ module lint commands into standalone api-lint and
api-lint-fix targets, reused by lint and lint-fix as dependencies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>
@devguyio-agent-smith
devguyio-agent-smith Bot force-pushed the backport-controlplaneversion-release-4.21 branch from d4cc7e5 to 74d1d3a Compare July 21, 2026 07:23
@devguyio-agent-smith devguyio-agent-smith Bot mentioned this pull request Jul 21, 2026
Open
@devguyio-agent-smith

devguyio-agent-smith Bot commented Jul 21, 2026

Copy link
Copy Markdown

Exploratory QE Validation Report: controlPlaneVersion Backport to release-4.21

Test Environment

  • Management cluster: aabdelre-mgmt (OCP 4.22.0-0.nightly-multi-2026-02-16-045147, K8s 1.34, us-east-1)
  • HostedCluster: cpv-test-421 (namespace: clusters)
  • HCP namespace: clusters-cpv-test-421
  • Release image: quay.io/openshift-release-dev/ocp-release:4.21.2-multi (upgraded to 4.21.3-multi)
  • Custom HO image: quay.io/abdalla/hypershift:backport-4.21
  • Custom CPO image: quay.io/abdalla/control-plane-operator:backport-4.21
  • Node pool replicas: 0 (control-plane-only validation)

Scenario 1: controlPlaneVersion field presence on HCP - PASS

Commands:

oc get hostedcontrolplane -n clusters-cpv-test-421 -o jsonpath='{.items[0].status.controlPlaneVersion.desired}'
oc get hostedcontrolplane -n clusters-cpv-test-421 -o jsonpath='{.items[0].status.controlPlaneVersion.history}'
oc get hostedcontrolplane -n clusters-cpv-test-421 -o jsonpath='{.items[0].status.controlPlaneVersion.observedGeneration}'

Observed output:

  • desired: {"image":"quay.io/openshift-release-dev/ocp-release@sha256:e01656a...","version":"4.21.2"}
  • history: 1 entry (Partial, see Scenario 2)
  • observedGeneration: 1

Result: All three fields (desired, history[], observedGeneration) present and populated within ~1 minute of CPO startup. PASS

Scenario 2: Initial deployment - Partial to Completed transition - PASS

Commands:

oc get controlplanecomponent -n clusters-cpv-test-421
oc get hostedcontrolplane -n clusters-cpv-test-421 -o jsonpath='{.items[0].status.controlPlaneVersion.history[0]}'

Observed output:

  • 39 ControlPlaneComponent resources created
  • Observed Partial -> Completed transition during initial deployment:
    • At 22:40:41Z: state=Partial, startedTime=2026-07-20T22:40:41Z, no completionTime
    • At 23:00:44Z: All components RolloutComplete=True, state=Completed
  • Final: history[0].state = Completed, completionTime = 2026-07-20T23:00:44Z

Result: All ControlPlaneComponents reached target version. history[0].state transitioned from Partial to Completed. Transition took ~20 minutes. PASS

Scenario 3: HCP-to-HC status propagation - PASS

Commands:

oc get hostedcontrolplane -n clusters-cpv-test-421 -o jsonpath='{.items[0].status.controlPlaneVersion}'
oc get hostedcluster cpv-test-421 -n clusters -o jsonpath='{.status.controlPlaneVersion}'

Observed output:

  • HCP controlPlaneVersion and HC controlPlaneVersion are identical (deep copy verified)
  • Both show: desired version 4.21.2, history[0].state=Completed, completionTime=2026-07-20T23:00:44Z, observedGeneration=1

Result: HC controlPlaneVersion matches HCP values exactly. PASS

Scenario 4: Upgrade - Partial to Completed transition - PASS

Commands:

oc patch hostedcluster cpv-test-421 -n clusters --type=merge \
  -p '{"spec":{"release":{"image":"quay.io/openshift-release-dev/ocp-release:4.21.3-multi"}}}'

oc get hostedcontrolplane -n clusters-cpv-test-421 \
  -o jsonpath='{.items[0].status.controlPlaneVersion}'

Observed output:

  • Within 30 seconds: new Partial entry prepended at history[0]:
    • version: 4.21.3, startedTime: 2026-07-21T06:51:53Z, no completionTime
    • Previous 4.21.2 Completed entry preserved at history[1]
    • desired updated to 4.21.3 with new image digest
    • observedGeneration bumped to 2
  • After ~5 minutes: history[0] transitioned to Completed:
    • completionTime: 2026-07-21T06:56:45Z
    • All 39 CPCs at RolloutComplete=True with version 4.21.3

Final state:

{
  "desired": {"version": "4.21.3", "image": "...sha256:f25a1e9a..."},
  "history": [
    {"state": "Completed", "version": "4.21.3", "startedTime": "2026-07-21T06:51:53Z", "completionTime": "2026-07-21T06:56:45Z"},
    {"state": "Completed", "version": "4.21.2", "startedTime": "2026-07-20T22:40:41Z", "completionTime": "2026-07-20T23:00:44Z"}
  ],
  "observedGeneration": 2
}

Result: z-stream upgrade correctly prepends a new Partial entry that transitions to Completed. Previous entries preserved. PASS

Scenario 5: History pruning - PASS

Approach: Lowered maxHistory from 100 to 3 in controlplaneversion.go, built a custom CPO image, deployed via annotation override.

Setup: Patched HCP status with 7 history entries (4.21.3, 4.21.2, 4.21.1, 4.20.5, 4.20.4, 4.20.3, 4.20.2).

Observed output after CPO reconcile:

History count: 3
  0: 4.21.3 Completed
  1: 4.21.2 Completed
  2: 4.21.1 Completed

Result: 7 entries pruned to 3. Kept the most recent entries, removed older z-stream entries, consistent with CVO weighted ranking algorithm. Code change reverted after testing. PASS

Summary

Scenario Description Result
1 controlPlaneVersion field presence on HCP PASS
2 Initial deployment - Partial to Completed PASS
3 HCP-to-HC status propagation PASS
4 Upgrade - Partial to Completed (4.21.2 -> 4.21.3) PASS
5 History pruning (maxHistory=3) PASS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@devguyio