CNTRLPLANE-3037: Introduce envtest for integration tests - #8034
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
openshift-ci-robot
added
the
jira/valid-reference
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/test unit |
|
Important Review skippedAuto reviews are limited based on label configuration. 🚫 Review skipped — only excluded labels are configured. (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds an envtest-based, YAML-driven API validation test framework and Makefile support to build and use setup-envtest and manage KUBEBUILDER_ASSETS for multiple OCP and upstream Kubernetes versions. It introduces envtest package types, a Ginkgo suite bootstrap, a generator that converts .testsuite.yaml files into table-driven tests, a NodePool testsuite manifest, and README documentation. hack/tools is updated to include setup-envtest. Several legacy E2E tests and YAML assets were removed. New Makefile variables and phony targets were added to compile/setup Sequence Diagram(s)sequenceDiagram
participant Makefile
participant Tools as "hack/tools (go build)"
participant SetupEnv as setup-envtest
participant TestRunner as "go test / Ginkgo"
participant EnvTest as "envtest.Environment"
participant SuiteLoader as "LoadTestSuiteSpecs"
participant Generator as "GenerateTestSuite"
participant APIServer as "API Server (with CRDs)"
participant K8sClient as "controller-runtime Client"
Makefile->>Tools: build setup-envtest binary (SETUP_ENVTEST_VER)
Tools-->>SetupEnv: produce setup-envtest executable
Makefile->>SetupEnv: invoke setup-envtest use (--index for OCP) to resolve assets
SetupEnv-->>Makefile: return KUBEBUILDER_ASSETS path(s)
Makefile->>TestRunner: invoke TestAPIs with KUBEBUILDER_ASSETS (loop per configured version)
TestRunner->>EnvTest: start envtest.Environment and install CRDs (allCRDs(featureSet))
EnvTest->>APIServer: launch API server with installed CRDs
TestRunner->>SuiteLoader: Load .testsuite.yaml files from test/envtest
SuiteLoader-->>TestRunner: []SuiteSpec
TestRunner->>Generator: GenerateTestSuite for each SuiteSpec
Generator->>K8sClient: create/update/delete unstructured test resources
K8sClient->>APIServer: send create/update requests
APIServer-->>K8sClient: validation responses / persisted objects
K8sClient-->>Generator: return results
Generator->>TestRunner: assert expected errors/state
TestRunner->>EnvTest: stop envtest.Environment
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
follow up for #8016 |
openshift-ci
Bot
added
the
do-not-merge/hold
openshift-ci
Bot
added
area/ci-tooling
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: enxebre The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
approved
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
test/envtest/generator.go (1)
58-67: Validate case GVKs against the suite header.Lines 63-65 only check that
crdNameis present, andversionis never enforced. Line 73 then derives the cleanup GVK from the first case's YAML instead. That leaves the suite header as inert metadata: a copiedinitial/updateddocument can silently exercise the wrong resource/version while the suite still reads as valid.Also applies to: 72-85
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/envtest/generator.go` around lines 58 - 67, The suite validation currently only checks s.CRDName and never enforces s.Version nor validates that each case's GVK matches the suite header; update the parsing/validation (around SuiteSpec unmarshalling and the code that derives cleanup GVK from the first case) to: ensure SuiteSpec has both CRDName and Version present (e.g., s.CRDName and s.Version), iterate over the parsed cases (e.g., cases or CaseSpec entries) and verify each case's Group/Kind/Version matches the suite header CRDName and Version, returning an error if any mismatch is found, and stop deriving the cleanup GVK from the first case YAML—use the suite header GVK (CRDName+Version) as the canonical cleanup GVK instead.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Makefile`:
- Around line 316-318: The Makefile test target currently runs the prerequisite
test-envtest-api-all and then runs go test ./... which re-runs the envtest
packages; update the test rule so it does not re-run envtest: either remove
test-envtest-api-all from the test target prerequisites and keep the existing go
test ./... invocation, or (preferably) change the go test invocation in the test
target to exclude the envtest packages (e.g., exclude packages under
./test/envtest) so the explicit test-envtest-api-all prerequisite handles
envtest bootstrapping and coverage only; adjust the KUBEBUILDER_ASSETS/GO
invocation accordingly to ensure coverage remains collected once.
In `@test/envtest/generator.go`:
- Around line 88-93: The AfterEach cleanup currently calls
k8sClient.DeleteAllOf(ctx, u, client.InNamespace("default")) but constructs an
Expect without applying a matcher, so failures are ignored; update the AfterEach
to assert the delete call succeeds by wrapping the call with
Expect(...).To(Succeed()) (or Expect(...).ToNot(HaveOccurred())) using the same
cleanupGVK, u.SetGroupVersionKind and k8sClient.DeleteAllOf symbols so any
deletion error will fail the test and prevent leaked objects.
In `@test/envtest/nodepools.testsuite.yaml`:
- Around line 412-435: The "when image is empty it should fail" test duplicates
the bad-format case because release.image is still set to "@"; change the
NodePool fixture's spec.release.image in that test to an empty string (or remove
the image field) so it truly tests the empty-image branch. Similarly, for the
"defaults to amd64" test remove the spec.arch: amd64 entry from that NodePool
fixture so the code exercises the omitted-arch/default path instead of always
using amd64.
---
Nitpick comments:
In `@test/envtest/generator.go`:
- Around line 58-67: The suite validation currently only checks s.CRDName and
never enforces s.Version nor validates that each case's GVK matches the suite
header; update the parsing/validation (around SuiteSpec unmarshalling and the
code that derives cleanup GVK from the first case) to: ensure SuiteSpec has both
CRDName and Version present (e.g., s.CRDName and s.Version), iterate over the
parsed cases (e.g., cases or CaseSpec entries) and verify each case's
Group/Kind/Version matches the suite header CRDName and Version, returning an
error if any mismatch is found, and stop deriving the cleanup GVK from the first
case YAML—use the suite header GVK (CRDName+Version) as the canonical cleanup
GVK instead.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 4227ca0f-0294-4036-b192-890e26dd5bb5
⛔ Files ignored due to path filters (49)
hack/tools/go.sumis excluded by!**/*.sumhack/tools/vendor/github.com/go-logr/logr/slogr/slogr.gois excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/.gitignoreis excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/.golangci.yamlis excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/LICENSEis excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/README.mdis excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/slogzapr.gois excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/zapr.gois excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/zapr_noslog.gois excluded by!**/vendor/**hack/tools/vendor/github.com/go-logr/zapr/zapr_slog.gois excluded by!**/vendor/**hack/tools/vendor/modules.txtis excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/LICENSEis excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/README.mdis excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/env/env.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/env/exit.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/env/helpers.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/main.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/remote/client.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/remote/http_client.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/remote/read_body.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/store/helpers.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/store/store.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/version/version.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/versions/parse.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/versions/platform.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/versions/version.gois excluded by!**/vendor/**hack/tools/vendor/sigs.k8s.io/controller-runtime/tools/setup-envtest/workflows/workflows.gois excluded by!**/vendor/**vendor/modules.txtis excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/envtest/crd.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/envtest/doc.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/envtest/helper.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/envtest/webhook.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/flock/doc.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/flock/errors.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/flock/flock_other.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/flock/flock_unix.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/addr/manager.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/certs/tinyca.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane/apiserver.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane/auth.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane/etcd.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane/kubectl.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane/plane.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/process/arguments.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/process/bin_path_finder.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/process/procattr_other.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/process/procattr_unix.gois excluded by!vendor/**,!**/vendor/**vendor/sigs.k8s.io/controller-runtime/pkg/internal/testing/process/process.gois excluded by!vendor/**,!**/vendor/**
📒 Files selected for processing (13)
Makefilehack/tools/go.modhack/tools/tools.gotest/e2e/assets/hostedcluster-base.yamltest/e2e/assets/nodepool-base.yamltest/e2e/create_cluster_test.gotest/e2e/v2/tests/api_ux_validation_test.gotest/envtest/README.mdtest/envtest/generator.gotest/envtest/hostedclusters.testsuite.yamltest/envtest/nodepools.testsuite.yamltest/envtest/suite_test.gotest/envtest/types.go
💤 Files with no reviewable changes (4)
- test/e2e/assets/hostedcluster-base.yaml
- test/e2e/create_cluster_test.go
- test/e2e/v2/tests/api_ux_validation_test.go
- test/e2e/assets/nodepool-base.yaml
| - name: when image is empty it should fail | ||
| initial: | | ||
| apiVersion: hypershift.openshift.io/v1beta1 | ||
| kind: NodePool | ||
| spec: | ||
| arch: amd64 | ||
| clusterName: some-cluster | ||
| management: | ||
| autoRepair: false | ||
| upgradeType: Replace | ||
| release: | ||
| image: "@" | ||
| replicas: 0 | ||
| platform: | ||
| aws: | ||
| instanceProfile: a-profile | ||
| instanceType: m6a.2xlarge | ||
| rootVolume: | ||
| size: 120 | ||
| type: gp3 | ||
| subnet: | ||
| id: "subnet-any" | ||
| type: AWS | ||
| expectedError: "Image must start with a word character (letters, digits, or underscores) and contain no white spaces" |
There was a problem hiding this comment.
These two cases don't hit the branch named in the test.
Line 423 still uses @, so the “image is empty” case just duplicates the bad-format case. Line 1297 still sets arch: amd64, so the “defaults to amd64” case never exercises the omitted-arch path.
Suggested fixture correction
- name: when image is empty it should fail
initial: |
apiVersion: hypershift.openshift.io/v1beta1
kind: NodePool
spec:
arch: amd64
clusterName: some-cluster
management:
autoRepair: false
upgradeType: Replace
release:
- image: "@"
+ image: ""
replicas: 0
platform:
aws:
instanceProfile: a-profile
instanceType: m6a.2xlarge
- name: should pass when imageType is Windows without arch (defaults to amd64)
initial: |
apiVersion: hypershift.openshift.io/v1beta1
kind: NodePool
spec:
- arch: amd64
clusterName: some-cluster
management:
autoRepair: false
upgradeType: Replace
release:
image: quay.io/openshift-release-dev/ocp-release:4.17.0-rc.0-x86_64Also applies to: 1292-1315
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/envtest/nodepools.testsuite.yaml` around lines 412 - 435, The "when
image is empty it should fail" test duplicates the bad-format case because
release.image is still set to "@"; change the NodePool fixture's
spec.release.image in that test to an empty string (or remove the image field)
so it truly tests the empty-image branch. Similarly, for the "defaults to amd64"
test remove the spec.arch: amd64 entry from that NodePool fixture so the code
exercises the omitted-arch/default path instead of always using amd64.
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
test/envtest/generator.go (1)
91-97:⚠️ Potential issue | 🟠 MajorStill missing a matcher on cleanup.
Line 96 wraps
DeleteAllOfinExpect(...)but never calls.To(...), so cleanup errors are ignored and leaked objects can contaminate later table entries.Run the following script to verify the current assertion:
#!/bin/bash set -euo pipefail nl -ba test/envtest/generator.go | sed -n '91,97p'Expected result: the cleanup block contains
Expect(k8sClient.DeleteAllOf(...))without a matcher.Suggested fix
AfterEach(func() { if cleanupGVK.Kind != "" { u := &unstructured.Unstructured{} u.SetGroupVersionKind(cleanupGVK) - Expect(k8sClient.DeleteAllOf(ctx, u, client.InNamespace("default"))) + Expect(k8sClient.DeleteAllOf(ctx, u, client.InNamespace("default"))).To(Succeed()) } })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/envtest/generator.go` around lines 91 - 97, The AfterEach cleanup calls Expect(k8sClient.DeleteAllOf(...)) without a matcher, so deletion errors are ignored; update the assertion around k8sClient.DeleteAllOf(ctx, u, client.InNamespace("default")) (used with cleanupGVK and unstructured.Unstructured) to include a matcher such as .To(Succeed()) or .ToNot(HaveOccurred()) so failures surface (e.g., Expect(k8sClient.DeleteAllOf(...)).To(Succeed())).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Makefile`:
- Around line 320-324: The ENVTEST_OCP_INDEX currently points to the moving
master branch (ENVTEST_OCP_INDEX :=
https://raw.githubusercontent.com/openshift/api/master/envtest-releases.yaml),
which makes test matrix non-reproducible; change it to a commit-pinned URL or
vendor the envtest index into the repo and point ENVTEST_OCP_INDEX at that
static file. Update the Makefile to replace the master raw URL with a specific
commit SHA raw URL (or a local vendored path) and ensure
ENVTEST_OCP_K8S_VERSIONS remains compatible with the pinned index.
- Around line 316-318: The default Makefile target "test" currently depends on
the full envtest matrix "test-envtest-api-all", making everyday runs slow and
network-dependent; change "test" to run only a single representative envtest
target (or no envtest) instead of depending on "test-envtest-api-all", and keep
"test-envtest-api-all" as a separate CI-only target; update any other targets
that inherit this dependency (notably "pre-commit") so they do not depend on the
full matrix, and ensure the cover output behavior (writing cover.out) remains
correct for the representative local run while the matrix target remains solely
for CI.
In `@test/envtest/suite_test.go`:
- Around line 59-71: The package-level Describe block is executed at package
init time but suites is populated in TestAPIs, so the generated specs register
zero tests; move the call to LoadTestSuiteSpecs so suites is populated before
any Describe evaluation (e.g., call LoadTestSuiteSpecs during init() or in
TestMain) and remove or adjust the LoadTestSuiteSpecs invocation inside TestAPIs
to avoid double-loading; ensure the global variable suites is set by the earlier
initialization so the Describe block (and its loop over suites) sees the loaded
specs.
---
Duplicate comments:
In `@test/envtest/generator.go`:
- Around line 91-97: The AfterEach cleanup calls
Expect(k8sClient.DeleteAllOf(...)) without a matcher, so deletion errors are
ignored; update the assertion around k8sClient.DeleteAllOf(ctx, u,
client.InNamespace("default")) (used with cleanupGVK and
unstructured.Unstructured) to include a matcher such as .To(Succeed()) or
.ToNot(HaveOccurred()) so failures surface (e.g.,
Expect(k8sClient.DeleteAllOf(...)).To(Succeed())).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: ec95d69f-bd2a-48a6-95e9-07eb7a24650f
📒 Files selected for processing (12)
.golangci.ymlMakefiletest/e2e/assets/hostedcluster-base.yamltest/e2e/assets/nodepool-base.yamltest/e2e/create_cluster_test.gotest/e2e/v2/tests/api_ux_validation_test.gotest/envtest/README.mdtest/envtest/generator.gotest/envtest/hostedclusters.testsuite.yamltest/envtest/nodepools.testsuite.yamltest/envtest/suite_test.gotest/envtest/types.go
💤 Files with no reviewable changes (4)
- test/e2e/assets/nodepool-base.yaml
- test/e2e/create_cluster_test.go
- test/e2e/assets/hostedcluster-base.yaml
- test/e2e/v2/tests/api_ux_validation_test.go
✅ Files skipped from review due to trivial changes (3)
- test/envtest/README.md
- test/envtest/types.go
- test/envtest/nodepools.testsuite.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- .golangci.yml
enxebre
changed the title
|
@enxebre: This pull request references CNTRLPLANE-3037 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. This pull request references Jira Issue OCPBUGS-78837, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/invalid-bug
|
/test e2e-aws |
Test Resultse2e-aws
|
|
|
install/unistall per suite has considerable impact on timing. |
openshift-ci
Bot
added
the
needs-rebase
1ae4450 to
d1a360d
Compare
- Add an envtest setup goal on the Makefile - Add a new unit test using envtest to cover the same as the create cluster e2e test. Signed-off-by: Borja Clemente <bclement@redhat.com>
openshift-ci
Bot
removed
the
needs-rebase
Introducing envtest requires adding new dependencies, which are being vendored in a separate commit to ease review. Signed-off-by: Borja Clemente <bclement@redhat.com>
Replace the Go-based envtest test with a YAML-driven test framework following the openshift/api tests pattern. The framework: - Loads .testsuite.yaml files defining onCreate/onUpdate test cases - Per suite, installs/uninstalls the CRD under test for each feature set variant (derived from stable./techpreview. file prefix) - Validates all CRDs install via dedicated GenerateCRDInstallTest - Generates Ginkgo test tables from YAML specs - Supports expectedError, expectedStatusError for validation tests - Supports initialCRDPatches for ratcheting validation via yaml-patch - Uses //go:build envtest tag so tests are excluded from go test ./... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
done PTAL @JoelSpeed |
|
@JoelSpeed would it make sense to try to align the assumptions that the CRD filtering does and then this repo can just vendor o/api/test logic? |
This turned impractical because it would require openshift/api#2637 which relies on write-available-featuresets using features.AllFeatureSets() which is consuming the gates from openshift/api. I created an alternative though that's closer to o/api but use spec.featureGate instead of the annotation for CRD filtering. |
|
@enxebre: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
openshift-ci
Bot
added
the
needs-rebase
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/close An alternative PR merged #8089 |
|
@JoelSpeed: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
5 participants
What this PR does / why we need it:
Drops existing api ux test which requires real infra in favour of YAML-driven integration tests that validate HyperShift CRD schemas
(including CEL validation rules) using envtest.
Tests run against multiple Kubernetes and OCP API server versions to catch compatibility issues
across releases.
The YAML format is compatible with
openshift/api tests.
Which issue(s) this PR fixes:
Fixes
Special notes for your reviewer:
Checklist:
Summary by CodeRabbit
New Features
Tests
Documentation
Chores