rosa-e2e-hcp: add rosa-gather-hcp-diagnostics chain with RHOBS/RMO/AVO log collection

[redhat-chai-bot]

Summary

Expand RHOBS diagnostic log collection for ROSA HCP e2e and FVT jobs. Follows up on #83475 (merged) which wired the bare rosa-gather-rhobs-logs ref into the workflow.

Changes

1. Extended log collection (rosa-gather-rhobs-logs-commands.sh)

• Added RMO (Route Monitor Operator) log collection from openshift-route-monitor-operator namespace
• Added AVO (AWS VPCE Operator) log collection from openshift-aws-vpce-operator namespace
• Both follow the existing collect_logs pattern with --limit 5000
• Updated ref YAML documentation to list the new log types

2. New chain: rosa-gather-hcp-diagnostics

• Created rosa-gather-hcp-diagnostics-chain.yaml wrapping rosa-gather-rhobs-logs
• Provides a single entry point for both workflow and inline FVT job post steps
• Generated metadata JSON via make registry-metadata

3. Workflow update (rosa-e2e-hcp-workflow.yaml)

• Replaced ref: rosa-gather-rhobs-logs (from rosa-e2e-hcp: add rosa-gather-rhobs-logs to post steps #83475) with chain: rosa-gather-hcp-diagnostics

4. FVT job updates

• Staging (ocm-fvt-rosa-hcp-staging): Added chain to all 18 test entries' post: blocks after rosa-e2e-collect-cs-telemetry
• Integration (ocm-fvt-rosa-hcp-integration): Added chain to the 2 steps-based entries (autonode-integration, ad-integration-regional). The 4 inline-command backup jobs were left unchanged.

Motivation

Supports diagnostics for recurring RHOBS Synthetic Monitoring test failures (e.g. sre:hcp:probe_active recording rule timeout tracked in ROSAENG-60340). RMO and AVO logs are critical for debugging false-positive API health alerts and VPCE connectivity issues on HCP clusters.

Supersedes #83476 and #83477 (closed due to rebase issues on this fast-moving repo).

Jira: ROSAENG-65395

---

AI-generated. Review for accuracy.

@dustman9000 requested in Slack thread

[coderabbitai]

Warning

Review limit reached

@redhat-chai-bot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 19 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 88d2f4a1-14c1-40a4-abf8-5c43bc09efd8

📥 Commits

Reviewing files that changed from the base of the PR and between 1ef5a7f and b51891c.

📒 Files selected for processing (8)

• ci-operator/config/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main__ocm-fvt-rosa-hcp-integration.yaml
• ci-operator/config/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main__ocm-fvt-rosa-hcp-staging.yaml
• ci-operator/step-registry/rosa/e2e/hcp/rosa-e2e-hcp-workflow.yaml
• ci-operator/step-registry/rosa/gather/hcp-diagnostics/OWNERS
• ci-operator/step-registry/rosa/gather/hcp-diagnostics/rosa-gather-hcp-diagnostics-chain.metadata.json
• ci-operator/step-registry/rosa/gather/hcp-diagnostics/rosa-gather-hcp-diagnostics-chain.yaml
• ci-operator/step-registry/rosa/gather/rhobs-logs/rosa-gather-rhobs-logs-commands.sh
• ci-operator/step-registry/rosa/gather/rhobs-logs/rosa-gather-rhobs-logs-ref.yaml

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-merge-bot]

[REHEARSALNOTIFIER]
@redhat-chai-bot: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-online-rosa-e2e-main-e2e-rosa-hcp-smoke	openshift-online/rosa-e2e	presubmit	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-ad-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-sanity-jira-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-periodics-rosa-hcp-e2e-staging-stable-4-21	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-zero-egress-upgrade-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-periodics-rosa-hcp-e2e-staging-stable-4-22	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-upgrade-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-integration-ocm-fvt-periodic-cs-rosa-hcp-ad-integration-regional-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-periodics-rosa-hcp-e2e-staging-stable-4-19	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-pl-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-sanity-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-nx-y-upgrade-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-upgrade-rosa-hcp-upgrade-staging-y-minus-1	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-amd64-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-access-request-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-ad-ocp5-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-zero-egress-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-amd64-upgrade-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-autonode-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-periodics-rosa-hcp-e2e-staging-nightly-5-0	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-periodics-rosa-hcp-e2e-staging-stable-4-20	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-adobe-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-upgrade-rosa-hcp-upgrade-staging-y-minus-3	N/A	periodic	Registry content changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-arm-staging-main	N/A	periodic	Ci-operator config changed
periodic-ci-openshift-online-rosa-e2e-main-ocm-fvt-rosa-hcp-staging-ocm-fvt-periodic-cs-rosa-hcp-y-upgrade-staging-main	N/A	periodic	Ci-operator config changed

A total of 29 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[dustman9000]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dustman9000, redhat-chai-bot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• ci-operator/config/openshift-online/rosa-e2e/OWNERS [dustman9000]
• ci-operator/step-registry/rosa/e2e/hcp/OWNERS [dustman9000]
• ci-operator/step-registry/rosa/gather/OWNERS [dustman9000]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dustman9000]

/pj-rehearse ack

[openshift-merge-bot]

@dustman9000: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[openshift-ci]

@redhat-chai-bot: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.