CNTRLPLANE-2914: Backport controlPlaneVersion status to release-4.20 by devguyio · Pull Request #3 · devguyio-agentic/hypershift

devguyio · 2026-04-07T12:34:44Z

Summary

Cherry-pick of openshift#7887 (CNTRLPLANE-2914: Add controlPlaneVersion status to HostedCluster and HostedControlPlane) to release-4.20.

This backport adds the controlPlaneVersion status field to HostedCluster and HostedControlPlane, providing a management-side view of control plane component rollout status independent of the CVO.

Changes

Add ControlPlaneVersionStatus and ControlPlaneUpdateHistory API types
Implement reconcileControlPlaneVersion aggregation logic in CPO
Implement pruneHistory with CVO weighted ranking algorithm
Propagate controlPlaneVersion from HCP to HC status
Add WaitForControlPlaneRollout e2e helper (gated to 4.20+)
Add WaitForImageRollout deprecated alias for WaitForDataPlaneRollout
Comprehensive unit test coverage for API types, reconciliation, pruning, and propagation

Conflict Resolution

CRD manifest files deleted in release-4.20 (GCPPlatform, Hypershift-CustomNoUpgrade, Hypershift-TechPreviewNoUpgrade) were excluded
aggregated-docs.md excluded (deleted in release-4.20)
Functions from main not present on release-4.20 (reconcileCAPIFinalizers, computeGCPPSCCondition) were excluded
Version gate adjusted from Version422 to Version420 for backport context
Version421 and Version422 constants added for forward compatibility

Note: This PR targets the fork's release-4.20 branch. A corresponding PR should be opened against openshift/hypershift release-4.20 once this is reviewed.

Ref: openshift#7887

devguyio · 2026-04-13T20:18:00Z

🧪 qe — 2026-04-13T20:17:11Z

Comprehensive Testing Report: `controlPlaneVersion` Backport to `release-4.20`

Environment

Parameter	Value
Management cluster	OCP 4.22.0-0.nightly-multi (Kubernetes v1.34.2)
HO image	`quay.io/abdalla/hypershift:backport-cpv-420` (all-in-one dev image)
CPO image	`quay.io/abdalla/hypershift:backport-cpv-420` (same all-in-one image)
Initial release	`quay.io/openshift-release-dev/ocp-release:4.20.0-multi`
HostedCluster	`cpv-test-420` (namespace: `clusters`, infra-id: `cpv-test-420-bfl4s`)
Region	`us-east-1`
Node pool	2 replicas (scaled to 0 during upgrade test, then back to 2)

Build & Deployment

Checked out backport-controlplaneversion-release-4.20 branch
Built all binaries locally with make build (Go 1.26.2)
Built all-in-one dev image using UBI9 base with all binaries + CPO labels
Pushed to quay.io/abdalla/hypershift:backport-cpv-420
Deleted existing CRDs for clean install (ensures printer columns are correct)
Installed HO with --enable-conversion-webhook=true (default)
Created HC with --control-plane-operator-image quay.io/abdalla/hypershift:backport-cpv-420

Test Scenario: Multi-Upgrade with History Accumulation

Objective: Validate that controlPlaneVersion status field correctly tracks CP component rollout across multiple upgrades, accumulating history entries with proper state transitions.

Step 1: Initial Install (4.20.0)

Created HC with release image 4.20.0-multi
controlPlaneVersion immediately populated with:
- desired.version: 4.20.0
- history[0]: version 4.20.0, state Partial
Result: ✅ PASS — Status field populated on initial creation

Step 2: Scale NodePool to 0

Scaled cpv-test-420-us-east-1a to 0 replicas
CP remains Partial (no nodes to complete data-plane rollout)
Result: ✅ PASS — Nodepool scaled successfully

Step 3: Upgrade to 4.20.1

Patched HC spec with 4.20.1-multi release image
After reconciliation:
- desired.version: 4.20.1
- history[0]: version 4.20.1, state Partial (in progress)
- history[1]: version 4.20.0, state Partial, completionTime set (superseded)
Result: ✅ PASS — History accumulated, previous entry marked with completionTime

Step 4: Upgrade to 4.20.2

Patched HC spec with 4.20.2-multi release image
After reconciliation:
- desired.version: 4.20.2
- history[0]: version 4.20.2, state Partial (in progress)
- history[1]: version 4.20.1, state Partial, completionTime set
- history[2]: version 4.20.0, state Partial, completionTime set
Result: ✅ PASS — 3 history entries, correct ordering (newest first)

Step 5: Scale NodePool Back to 2

Scaled nodepool back to 2 replicas
Waited for CP components to reconcile at 4.20.2
Result: ✅ PASS — Nodes provisioned, CP components became available

Step 6: Verify Completed State

After CP availability:
- desired.version: 4.20.2
- history[0]: version 4.20.2, state Completed, completionTime: 2026-04-13T20:13:51Z
- history[1]: version 4.20.1, state Partial, completionTime: 2026-04-13T20:07:28Z
- history[2]: version 4.20.0, state Partial, completionTime: 2026-04-13T20:06:07Z
Result: ✅ PASS — Latest entry transitioned to Completed

Printer Column Verification

NAME           VERSION   CP VERSION   KUBECONFIG                      PROGRESS   AVAILABLE   CP PROGRESS   DP PROGRESS
cpv-test-420             4.20.2       cpv-test-420-admin-kubeconfig   Partial    True        Completed     Partial

CP Version column: ✅ Displays 4.20.2 (from status.controlPlaneVersion.history[?(@.state=="Completed")].version)
CP Progress column: ✅ Displays Completed (from status.controlPlaneVersion.history[0].state)
DP Progress column: ✅ Displays Partial (from status.version.history[0].state)
Note: VERSION (CVO-based) is empty as expected — CVO version depends on data-plane completion

HCP ↔ HC Propagation

Both HCP and HC controlPlaneVersion status are identical — confirming correct propagation from HCP to HC:

Same desired, history, and observedGeneration values
Result: ✅ PASS

Final controlPlaneVersion Status (raw)

{
    "desired": {
        "image": "quay.io/openshift-release-dev/ocp-release@sha256:40cdd399e6243207a2d5bb6f2eaf8b57d01495c937c882eb6431a86f18439b70",
        "version": "4.20.2"
    },
    "history": [
        {
            "completionTime": "2026-04-13T20:13:51Z",
            "image": "quay.io/openshift-release-dev/ocp-release@sha256:40cdd399e6243207a2d5bb6f2eaf8b57d01495c937c882eb6431a86f18439b70",
            "startedTime": "2026-04-13T20:07:28Z",
            "state": "Completed",
            "version": "4.20.2"
        },
        {
            "completionTime": "2026-04-13T20:07:28Z",
            "image": "quay.io/openshift-release-dev/ocp-release@sha256:4e903dead99fc02fc64c5f599aafe066f99c33a5b375d2fe85e85f0ca64f3aaa",
            "startedTime": "2026-04-13T20:06:07Z",
            "state": "Partial",
            "version": "4.20.1"
        },
        {
            "completionTime": "2026-04-13T20:06:07Z",
            "image": "quay.io/openshift-release-dev/ocp-release@sha256:4f1e772349a20f2eb69e8cf70d73b4fcc299c15cb6e4f027696eb469e66d4080",
            "startedTime": "2026-04-13T20:03:01Z",
            "state": "Partial",
            "version": "4.20.0"
        }
    ],
    "observedGeneration": 3
}

Summary

Test	Result
controlPlaneVersion populated on initial creation	✅ PASS
History accumulates across upgrades	✅ PASS
Superseded entries get completionTime	✅ PASS
State transitions: Partial → Completed	✅ PASS
3 history entries (4.20.0 → 4.20.1 → 4.20.2)	✅ PASS
HCP → HC status propagation	✅ PASS
CRD printer columns (CP Version, CP Progress, DP Progress)	✅ PASS
Custom CPO image used	✅ PASS

Overall: ✅ ALL TESTS PASSED — The controlPlaneVersion backport to release-4.20 is working correctly.

…nent versions Add ControlPlaneVersionStatus and ControlPlaneUpdateHistory types to HostedClusterStatus and HostedControlPlaneStatus. These track management-side control plane component versions independently from CVO, enabling service providers to detect completed control plane upgrades and compute NodePool version skew without waiting for data-plane rollout. Ref: CNTRLPLANE-2914 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>

… and docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…tatus Implement reconcileControlPlaneVersion() that aggregates ControlPlaneComponent status into hcp.Status.ControlPlaneVersion with version transition logic, image-only change detection, and observedGeneration tracking. Include pruneHistory() ported from CVO's weighted ranking algorithm with exact constants to cap history at 100 entries. Ref: CNTRLPLANE-2914 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>

Deep-copy controlPlaneVersion from HostedControlPlane status to HostedCluster status, following the existing Platform field propagation pattern. Handles nil gracefully for version skew safety during rolling upgrades. Ref: CNTRLPLANE-2914 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>

Extract the api/ module lint commands into standalone api-lint and api-lint-fix targets, reused by lint and lint-fix as dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>

devguyio-agent-smith · 2026-07-21T10:49:13Z

Exploratory QE Validation Report: controlPlaneVersion Backport to release-4.20

Test Environment

Management cluster: aabdelre-mgmt (OCP 4.22.0-0.nightly-multi-2026-02-16-045147, K8s 1.34, us-east-1)
HostedCluster: cpv-test-420 (namespace: clusters)
HCP namespace: clusters-cpv-test-420
Release image: quay.io/openshift-release-dev/ocp-release:4.20.14-multi (upgraded to 4.20.15-multi)
Custom HO image: quay.io/abdalla/hypershift:backport-4.20
Custom CPO image: quay.io/abdalla/control-plane-operator:backport-4.20
Node pool replicas: 0 (control-plane-only validation)

Scenario 1: controlPlaneVersion field presence on HCP - PASS

Commands:

oc get hostedcontrolplane cpv-test-420 -n clusters-cpv-test-420 \
  -o jsonpath='{.status.controlPlaneVersion}'

Observed output:

desired: {"image":"quay.io/openshift-release-dev/ocp-release@sha256:08a2dc23...","version":"4.20.14"}
history: 1 entry (see Scenario 2)
observedGeneration: 1

Result: All three fields (desired, history[], observedGeneration) present and populated. PASS

Scenario 2: Initial deployment - Partial to Completed transition - PASS

Commands:

oc get hostedcontrolplane cpv-test-420 -n clusters-cpv-test-420 \
  -o jsonpath='{.status.controlPlaneVersion.history[0]}'

Observed output:

Partial -> Completed transition:
- startedTime: 2026-07-21T07:43:47Z
- completionTime: 2026-07-21T07:52:04Z
- All ControlPlaneComponents reached target version 4.20.14
Transition took ~8 minutes

Result: history[0].state transitioned from Partial to Completed with valid timestamps. PASS

Scenario 3: HCP-to-HC status propagation - PASS

Commands:

oc get hostedcontrolplane cpv-test-420 -n clusters-cpv-test-420 \
  -o jsonpath='{.status.controlPlaneVersion}'
oc get hostedcluster cpv-test-420 -n clusters \
  -o jsonpath='{.status.controlPlaneVersion}'

Observed output:

HCP and HC controlPlaneVersion are identical (deep copy verified)
Both show: desired version 4.20.14, history[0].state=Completed, completionTime=2026-07-21T07:52:04Z, observedGeneration=1

Result: HC controlPlaneVersion matches HCP values exactly. PASS

Scenario 4: Upgrade - Partial entry creation on version change - PASS

Commands:

oc patch hostedcluster cpv-test-420 -n clusters --type=merge \
  -p '{"spec":{"release":{"image":"quay.io/openshift-release-dev/ocp-release:4.20.15-multi"}}}'

oc get hostedcontrolplane cpv-test-420 -n clusters-cpv-test-420 \
  -o jsonpath='{.status.controlPlaneVersion}'

Observed output:

Within 30 seconds: new Partial entry prepended at history[0]:
- version: 4.20.15, startedTime: 2026-07-21T07:59:46Z, no completionTime
- Previous 4.20.14 Completed entry preserved at history[1]
- desired updated to 4.20.15 with new image digest
- observedGeneration bumped to 2

State at observation:

{
  "desired": {"version": "4.20.15", "image": "...sha256:7574c1d0..."},
  "history": [
    {"state": "Partial", "version": "4.20.15", "startedTime": "2026-07-21T07:59:46Z"},
    {"state": "Completed", "version": "4.20.14", "startedTime": "2026-07-21T07:43:47Z", "completionTime": "2026-07-21T07:52:04Z"}
  ],
  "observedGeneration": 2
}

Note: The cluster-network-operator operand (ovnkube-control-plane) is still rolling out the new image on the zero-worker cluster, keeping state at Partial. This is expected behavior -- some operands depend on node readiness signals. The full Partial->Completed upgrade transition was validated on the 4.21 backport (PR #2).

Result: Upgrade correctly prepends a new Partial entry with startedTime, preserves previous Completed entry, bumps observedGeneration. PASS

Scenario 5: History pruning - PASS

Approach: Lowered maxHistory from 100 to 3 in controlplaneversion.go, built a custom CPO image, deployed via annotation override.

Setup: Patched HCP status with 7 history entries (4.20.15 Partial, 4.20.14, 4.20.13, 4.20.12, 4.20.11, 4.19.5, 4.19.4).

Observed output after CPO reconcile:

History count: 3
  0: 4.20.15 Partial
  1: 4.20.14 Completed
  2: 4.20.13 Completed

Result: 7 entries pruned to 3. Kept the current Partial entry and most recent Completed entries, removed older z-stream and minor version entries, consistent with CVO weighted ranking algorithm. Code change reverted after testing. PASS

Summary

Scenario	Description	Result
1	controlPlaneVersion field presence on HCP	PASS
2	Initial deployment - Partial to Completed	PASS
3	HCP-to-HC status propagation	PASS
4	Upgrade - Partial entry creation (4.20.14 -> 4.20.15)	PASS
5	History pruning (maxHistory=3)	PASS

devguyio mentioned this pull request Apr 7, 2026

Backport controlPlaneVersion status to release-4.21 and release-4.20 devguyio-bot-squad/devguyio-agentic-team#122

Closed

devguyio force-pushed the backport-controlplaneversion-release-4.20 branch from eb9b470 to d092c03 Compare April 7, 2026 14:05

devguyio-agent-smith Bot force-pushed the backport-controlplaneversion-release-4.20 branch from d092c03 to 115d511 Compare July 16, 2026 11:59

devguyio and others added 5 commits July 21, 2026 09:26

chore: run make update to regenerate CRDs, deepcopy, clients, vendor,…

40637da

… and docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

devguyio-agent-smith Bot force-pushed the backport-controlplaneversion-release-4.20 branch from 115d511 to 1f789a3 Compare July 21, 2026 07:26

Navigation Menu

Repository navigation

CNTRLPLANE-2914: Backport controlPlaneVersion status to release-4.20 - #3

CNTRLPLANE-2914: Backport controlPlaneVersion status to release-4.20#3
devguyio wants to merge 5 commits into
release-4.20from
backport-controlplaneversion-release-4.20

devguyio commented Apr 7, 2026

Uh oh!

devguyio commented Apr 13, 2026

Uh oh!

devguyio-agent-smith Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

devguyio commented Apr 7, 2026

Summary

Changes

Conflict Resolution

Uh oh!

devguyio commented Apr 13, 2026

🧪 qe — 2026-04-13T20:17:11Z

Comprehensive Testing Report: controlPlaneVersion Backport to release-4.20

Environment

Build & Deployment

Test Scenario: Multi-Upgrade with History Accumulation

Step 1: Initial Install (4.20.0)

Step 2: Scale NodePool to 0

Step 3: Upgrade to 4.20.1

Step 4: Upgrade to 4.20.2

Step 5: Scale NodePool Back to 2

Step 6: Verify Completed State

Printer Column Verification

HCP ↔ HC Propagation

Final controlPlaneVersion Status (raw)

Summary

Uh oh!

devguyio-agent-smith Bot commented Jul 21, 2026

Exploratory QE Validation Report: controlPlaneVersion Backport to release-4.20

Test Environment

Scenario 1: controlPlaneVersion field presence on HCP - PASS

Scenario 2: Initial deployment - Partial to Completed transition - PASS

Scenario 3: HCP-to-HC status propagation - PASS

Scenario 4: Upgrade - Partial entry creation on version change - PASS

Scenario 5: History pruning - PASS

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comprehensive Testing Report: `controlPlaneVersion` Backport to `release-4.20`