OCPBUGS-89330: Fix cache path to avoid /var/cache/dnf conflict

[ericahinkleRH]

Problem

On OpenShift 4.19.32, operator-controller pods crash on startup with:

chmod /var/cache/dnf: operation not permitted

This happens because:

1. Default cache path is /var/cache
2. RHEL base image contains /var/cache/dnf owned by root
3. Pod runs as non-root (UID 1001)
4. EnsureEmptyDirectory() tries to chmod root-owned directories, which fails

Solution

Change default cache path from /var/cache to /var/cache/operator-controller to avoid conflicts with system directories.

Testing

• Build and deploy with new default
• Verify pod starts successfully
• Verify cache operations work correctly

Fixes: https://redhat.atlassian.net/browse/OCPBUGS-89330

Summary by CodeRabbit

• Bug Fixes
  • Updated the default filesystem cache location for the operator-controller by changing the --cache-path default from /var/cache to /var/cache/operator-controller, keeping cache files in a dedicated directory for improved organization.

[openshift-ci-robot]

@ericahinkleRH: This pull request references Jira Issue OCPBUGS-89330, which is invalid:

• expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Problem

On OpenShift 4.19.32, operator-controller pods crash on startup with:

chmod /var/cache/dnf: operation not permitted

This happens because:

1. Default cache path is /var/cache
2. RHEL base image contains /var/cache/dnf owned by root
3. Pod runs as non-root (UID 1001)
4. EnsureEmptyDirectory() tries to chmod root-owned directories, which fails

Solution

Change default cache path from /var/cache to /var/cache/operator-controller to avoid conflicts with system directories.

Testing

• Build and deploy with new default
• Verify pod starts successfully
• Verify cache operations work correctly

Fixes: https://redhat.atlassian.net/browse/OCPBUGS-89330

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ericahinkleRH
Once this PR has been reviewed and has the lgtm label, please assign joelanford for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c16708c4-1aa1-4bb3-9c45-ae0a7405fd7c

📥 Commits

Reviewing files that changed from the base of the PR and between 155a229 and 9ff63e9.

📒 Files selected for processing (1)

• cmd/operator-controller/main.go

🚧 Files skipped from review as they are similar to previous changes (1)

• cmd/operator-controller/main.go

---

Walkthrough

The --cache-path flag now defaults to /var/cache/operator-controller instead of /var/cache.

Changes

Cache Path Configuration

Layer / File(s)	Summary
Controller-specific cache directory cmd/operator-controller/main.go	Updates the default --cache-path value to /var/cache/operator-controller.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: joelanford, rashmigottipati

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: moving the default cache path to avoid the /var/cache/dnf conflict.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PR only changes a cache-path default in main.go; no Ginkgo test titles were added or modified.
Test Structure And Quality	✅ Passed	No Ginkgo test code was changed; the PR only updates the cache-path default in main.go.
Microshift Test Compatibility	✅ Passed	No new Ginkgo tests were added; the PR only changes a default cache-path flag in main.go.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	Only a cache-path default changed in cmd/operator-controller/main.go; no new e2e tests or SNO-sensitive cluster assumptions were added.
Topology-Aware Scheduling Compatibility	✅ Passed	Only the cache-path default changed in cmd/operator-controller/main.go; no replicas, node selectors, affinity, PDBs, or topology-sensitive scheduling logic were added.
Ote Binary Stdout Contract	✅ Passed	The PR only changes a cache-path default in init(); it adds no new stdout writes in main/init or top-level initializers, and main still logs errors to stderr.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	No new Ginkgo e2e tests were added; only the cache-path default in cmd/operator-controller/main.go changed.
No-Weak-Crypto	✅ Passed	Only a cache-path default changed; no weak crypto, custom crypto, or insecure secret comparisons appear in the modified code.
Container-Privileges	✅ Passed	Only cmd/operator-controller/main.go changed, updating --cache-path; no manifest privilege settings were introduced.
No-Sensitive-Data-In-Logs	✅ Passed	Diff only changes the default cache-path flag; no new log statements or sensitive values were added.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[rashmigottipati]

Hi @ericahinkleRH, thank you for your contribution.

I noticed the default --cache-path is defined upstream as /var/cache, and this change updates that default. Should this fix be made in the upstream operator-controller project first and then be pulled here through the sync? Or is this issue specific to the downstream OpenShift deployment? I'd like to understand whether there's a reason for carrying this as a downstream-only change.

[ericahinkleRH]

Hi @rashmigottipati! This is OpenShift-specific because of the different base images:

Upstream uses gcr.io/distroless/static:nonroot which doesn't have /var/cache/dnf, while OpenShift uses registry.ci.openshift.org/ocp/5.0:base-rhel9 which contains /var/cache/dnf owned by root.

When the pod runs as UID 1001, EnsureEmptyDirectory() tries to chmod /var/cache/dnf and fails with "operation not permitted" because non-root can't chmod root-owned directories.

Changing the default to /var/cache/operator-controller avoids this conflict with system directories in the RHEL base image while still working fine on distroless.

[rashmigottipati]

Thanks for the context @ericahinkleRH . That makes sense -the issue is tied to the OpenShift RHEL base image layout rather than the upstream default itself. Given that upstream's distroless image does not contain the conflicting /var/cache/dnf path, a downstream change here seems appropriate. lgtm from that perspective.

[ericahinkleRH]

/jira refresh

[openshift-ci-robot]

@ericahinkleRH: This pull request references Jira Issue OCPBUGS-89330, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[ericahinkleRH]

/test e2e-aws-techpreview-olmv1-ext

[tmshort]

/hold
This can't be fixed here in main; as it's the upstream code.
If you want to fix it here in openshift, you can change the value of -cache-path in the manifest.

[tmshort]

Given that we don't provide a mechanism in the upstream manifests to modify the arguments to the deployments; this should likely be done upstream.

[ericahinkleRH]

Thanks for the explanation! You're absolutely right - I should have made this upstream first.

I've created the upstream PR here: operator-framework/operator-controller#2835

I included the OCPBUGS-89330 reference in the description so it will automatically attach to the downstreaming PR when it gets merged and pulled downstream.

Closing this PR in favor of the upstream fix. Once upstream merges and syncs down, the bug will be automatically attached to that PR.

Thanks for catching this!

[openshift-ci-robot]

@ericahinkleRH: This pull request references Jira Issue OCPBUGS-89330. The bug has been updated to no longer refer to the pull request using the external bug tracker.

Details

In response to this:

Problem

On OpenShift 4.19.32, operator-controller pods crash on startup with:

chmod /var/cache/dnf: operation not permitted

This happens because:

1. Default cache path is /var/cache
2. RHEL base image contains /var/cache/dnf owned by root
3. Pod runs as non-root (UID 1001)
4. EnsureEmptyDirectory() tries to chmod root-owned directories, which fails

Solution

Change default cache path from /var/cache to /var/cache/operator-controller to avoid conflicts with system directories.

Testing

• Build and deploy with new default
• Verify pod starts successfully
• Verify cache operations work correctly

Fixes: https://redhat.atlassian.net/browse/OCPBUGS-89330

Summary by CodeRabbit

• Bug Fixes
• Updated the default filesystem cache location for the operator-controller by changing the --cache-path default from /var/cache to /var/cache/operator-controller, keeping cache files in a dedicated directory for improved organization.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.