OCPBUGS-99183: Fix NetworkDiagnosticsConfig serial test race condition

[mkowalski]

Summary

The AfterEach handler in the NetworkDiagnosticsConfig serial test suite resets the network diagnostics config but does not wait for the openshift-network-diagnostics namespace to be fully recreated. After the "Should remove all network diagnostics pods when disabled" test, the namespace enters Terminating state. The next test starts immediately and cannot find diagnostics pods, polling indefinitely until the 5h job timeout is hit.

Root Cause

1. Should remove all network diagnostics pods when disabled disables diagnostics → namespace gets deleted
2. AfterEach resets config to nil (default/enabled) but returns immediately
3. Next test starts before CNO recreates the namespace
4. CNO tries to apply NetworkPolicy but namespace is still Terminating → ApplyOperatorConfig Degraded
5. Job times out at 5h

Fix

Add a wait in AfterEach that ensures:

1. The openshift-network-diagnostics namespace is Active (not Terminating)
2. network-check-source pods exist
3. network-check-target pods exist
4. NetworkDiagnosticsAvailable condition is True

5 minute timeout with 5s polling — matching the existing test timeouts in the suite.

Evidence

4 consecutive failures on 5.0 metal techpreview serial jobs (ipv4, ipv6, dualstack), all with identical pattern. Sippy regression ID 43316.

Example: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-main-nightly-5.0-e2e-metal-ipi-serial-ovn-ipv4-techpreview-1of2/2078329267703255040

Bug

https://issues.redhat.com/browse/OCPBUGS-99183

---

🤖 This PR was generated by AI on behalf of @mkowalski, who has reviewed it.

Summary by CodeRabbit

• Tests
  • Improved network diagnostics test cleanup by waiting for the diagnostics environment to fully become active before subsequent tests run.
  • Added verification that required namespaces, pods, and availability status are ready, with diagnostic logging when readiness is delayed.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci-robot]

@mkowalski: This pull request references Jira Issue OCPBUGS-99183, which is invalid:

• expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

The AfterEach handler in the NetworkDiagnosticsConfig serial test suite resets the network diagnostics config but does not wait for the openshift-network-diagnostics namespace to be fully recreated. After the "Should remove all network diagnostics pods when disabled" test, the namespace enters Terminating state. The next test starts immediately and cannot find diagnostics pods, polling indefinitely until the 5h job timeout is hit.

Root Cause

1. Should remove all network diagnostics pods when disabled disables diagnostics → namespace gets deleted
2. AfterEach resets config to nil (default/enabled) but returns immediately
3. Next test starts before CNO recreates the namespace
4. CNO tries to apply NetworkPolicy but namespace is still Terminating → ApplyOperatorConfig Degraded
5. Job times out at 5h

Fix

Add a wait in AfterEach that ensures:

1. The openshift-network-diagnostics namespace is Active (not Terminating)
2. network-check-source pods exist
3. network-check-target pods exist
4. NetworkDiagnosticsAvailable condition is True

5 minute timeout with 5s polling — matching the existing test timeouts in the suite.

Evidence

4 consecutive failures on 5.0 metal techpreview serial jobs (ipv4, ipv6, dualstack), all with identical pattern. Sippy regression ID 43316.

Example: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-main-nightly-5.0-e2e-metal-ipi-serial-ovn-ipv4-techpreview-1of2/2078329267703255040

Bug

https://issues.redhat.com/browse/OCPBUGS-99183

---

🤖 This PR was generated by AI on behalf of @mkowalski, who has reviewed it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Walkthrough

The networking test cleanup now waits after resetting diagnostics configuration until the diagnostics namespace is active, source and target pods exist, and the availability condition is true.

Changes

Network diagnostics synchronization

Layer / File(s)	Summary
Wait for diagnostics convergence test/extended/networking/network_diagnostics.go	AfterEach polls for the active diagnostics namespace, source and target pods, and a true NetworkDiagnosticsAvailable condition, logging unsuccessful checks.раз

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR only adds an AfterEach wait; the Ginkgo titles in the file are static and contain no dynamic or unstable values.
Test Structure And Quality	✅ Passed	The suite keeps focused It blocks, uses AfterEach cleanup, and the new wait is bounded with a 5m Eventually plus diagnostic logs.
Microshift Test Compatibility	✅ Passed	The patch only adds an AfterEach wait using core K8s namespace/pod checks in an existing Serial suite; it adds no new MicroShift-only incompatibility or skip change.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR only adds an AfterEach convergence wait for namespace/pods/condition and introduces no new multi-node or HA assumptions.
Topology-Aware Scheduling Compatibility	✅ Passed	Only a test AfterEach wait was added; no manifests, operators, or controllers with scheduling constraints were changed.
Ote Binary Stdout Contract	✅ Passed	The diff only adds logging inside AfterEach; no process-level stdout writes or suite/main setup output were introduced.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	No new Ginkgo tests were added; the change only adds an AfterEach wait using cluster-internal K8s APIs, with no IPv4 literals or external connectivity.
No-Weak-Crypto	✅ Passed	PR only adds an Eventually wait in a networking test; no weak-crypto APIs, custom crypto, or secret/token comparisons were introduced.
Container-Privileges	✅ Passed	Only a Go test file changed; no container/K8s manifests or privilege-related fields were added in the diff.
No-Sensitive-Data-In-Logs	✅ Passed	New AfterEach logs are generic status/polling messages and do not print secrets, tokens, PII, or customer data.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: fixing a NetworkDiagnosticsConfig serial test race condition.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mkowalski

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• test/extended/networking/OWNERS [mkowalski]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mkowalski]

/jira refresh

[openshift-ci-robot]

@mkowalski: This pull request references Jira Issue OCPBUGS-99183, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (core-networking-bot@redhat.com), skipping review request.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-merge-bot]

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

[openshift-ci]

@mkowalski: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-serial-2of2	47c9b61	link	true	/test e2e-aws-ovn-serial-2of2
ci/prow/e2e-vsphere-ovn	47c9b61	link	true	/test e2e-vsphere-ovn
ci/prow/e2e-gcp-ovn	47c9b61	link	true	/test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-microshift	47c9b61	link	true	/test e2e-aws-ovn-microshift

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.