Skip to content

fix(agent): refuse Secret access through the agent and make Secrets RBAC opt-in - #714

Merged
blue4209211 merged 2 commits into
mainfrom
fix/agent-secret-lockdown
Nov 8, 2026
Merged

blue4209211 merged 2 commits into
mainfrom
fix/agent-secret-lockdown

Conversation

@mayankpande88

Copy link
Copy Markdown
Contributor

Summary

The runner's ClusterRole granted get/list/watch on every Secret in the cluster, and kubectl_command_executor accepted kubectl get secret as a read. Anything that could send the agent an action could read every credential in the cluster. This PR closes that in the agent itself and removes the cluster-wide grant from the chart.

  • Executor refuses Secret access in every mode, read-only and write:
    • Secret-type kinds in every spelling, including short names: secret, sealedsecret, externalsecret/es, secretstore/ss, clustersecretstore/css, secretproviderclass/spc.
    • --raw paths that reach them, including percent-encoded ones.
    • File-backed output formats (-o jsonpath-file=, go-template-file, custom-columns-file, templatefile). These read a file from the agent pod and echo it on a parse error.
    • Mounted credential paths (/var/run/secrets…) and create token.
  • get_resource / get_resource_yaml blank Secret values. Key names are kept, and last-applied-configuration is dropped because it carries the values in plain text.
  • New list_helm_releases action. It reads Helm release Secrets inside the agent and returns metadata only: name, namespace, revision, status, chart, version, appVersion, kubeVersion and last deployed. Release values and manifests stay in the cluster.
  • Chart: no Secrets rules by default. The default, write-mode and read-only renders grant no Secrets verb. Write mode also loses its cluster-wide secrets update/patch/delete, since nothing can use them now. Two opt-ins remain:
    • runner.scannerAutoCopyPullSecrets now defaults to false. When on, it grants get on secrets in every namespace (no list or watch) and create/update/patch/delete in the release namespace only.
    • runner.helmReleaseDiscovery, new, default false. When on, it grants cluster-wide list on secrets for list_helm_releases. RBAC can't narrow this to Helm's Secrets.
  • CI: helm-dev-lint fails if any render grants a Secrets verb outside those two opt-ins.

Type of change

  • Breaking change (chart upgrade requires user action)

Upgrading changes behavior in three ways:

  • Private-image scans no longer copy the workload's pull secrets unless runner.scannerAutoCopyPullSecrets=true is set.
  • get_resource on Secrets fails with Forbidden unless an opt-in grants list.
  • Callers that read Helm releases with kubectl get secrets -l owner=helm must move to list_helm_releases and set runner.helmReleaseDiscovery=true.

Chart version

  • No version bump needed (the release stamps it)

Test plan

  • make validate in runner/ (fmt, vet, go test -race ./...)
  • helm lint charts/nudgebee-agent passes
  • helm template reviewed for default, write mode, read-only and each opt-in. The new CI step passes locally.
  • Installed the rendered RBAC on a kind cluster and verified as the runner's ServiceAccount
Engineering detail
  • New tests:
    • secretpolicy_test.go: 34 refused commands, run in both modes; secret mutations in write mode; names that only contain "secret" are still allowed; redaction through get_resource and the YAML path.
    • helm_list_test.go: latest revision only, release values never returned, Forbidden mapped to the opt-in error.
  • The Secret check runs before the write-mode early return in validateSegment, so runner.enableWritePermissions does not lift it.
  • Flag values are scanned for kind names too. A namespace literally named secrets, es or ss is refused. That is accepted: a false refusal is a clear error, a miss is a leak.
  • list_helm_releases pages at 50 Secrets, so a large cluster's release blobs are never held at once. It returns a clear "not enabled" error when the list is forbidden.
  • Not covered here: the default ClusterRole still grants pods/exec cluster-wide, so exec-based actions can still read a pod's environment or mounted files.

Local e2e: on a kind cluster, the chart's rendered RBAC was applied and the agent's code ran with the runner ServiceAccount's token, using real kubectl and the real API server. Default render: auth can-i get/list/create secrets all returned no. The executor refused all 7 Secret-reading commands in both modes (get secrets, get secret … -o jsonpath={.data…}, --raw …/secrets, --raw …/%73ecrets, -o jsonpath-file=…/token, create token), and get pods ran. get_resource on secrets returned Forbidden, and list_helm_releases returned the "not enabled" error. With helmReleaseDiscovery=true: list allowed, get and watch denied. list_helm_releases returned revision 3 of a test release (name, chart 1.2.3, kubeVersion), and get_resource returned blank values. None of the planted Secret or Helm values appeared in any output. With scannerAutoCopyPullSecrets=true: get allowed in the workload namespace, list denied, and create only in the release namespace. A real schedule_k8s_job copied only the dockerconfigjson pull secret (an Opaque Secret listed on the same pod was skipped). The copy was owned by the Job, and the Job resumed.

…BAC opt-in

The runner's ClusterRole granted get/list/watch on every Secret in the
cluster, and kubectl_command_executor accepted `get secret` as a read, so
anything able to reach the agent could read every credential in the cluster.

Agent:
- kubectl_command_executor refuses secret-bearing kinds (secrets, sealed,
  external, store and provider-class kinds, all spellings), --raw paths to
  them, file-backed output formats (-o *-file, templatefile), mounted
  credential paths and `create token`. Enforced in read-only and write mode.
- get_resource / get_resource_yaml blank Secret values and drop the
  last-applied-configuration annotation; key names are kept.
- New list_helm_releases action: decodes Helm release Secrets in the agent
  and returns chart metadata only (name, revision, status, chart, version,
  appVersion, kubeVersion). Release values never leave the cluster.

Chart:
- No Secrets rules in the default, write-mode or read-only render.
- runner.scannerAutoCopyPullSecrets now defaults to false. When on it grants
  get (no list/watch) on secrets cluster-wide plus create/update/patch/delete
  in the release namespace.
- New runner.helmReleaseDiscovery (default false) grants cluster-wide list
  on secrets for list_helm_releases.
- CI asserts no Secrets grant exists outside the two opt-ins.
@mayankpande88
mayankpande88 requested a review from a team as a code owner November 7, 2026 19:52

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request restricts the agent's default Kubernetes Secret access by making secret-copying opt-in, removing cluster-wide Secret permissions from the base service account, and introducing a strict secret-access policy for kubectl commands and resource retrieval. It also adds an opt-in Helm release discovery feature that lists Helm release metadata without leaking release values. The review feedback highlights a path traversal bypass vulnerability in the --raw command validation, a potential nil pointer dereference in Helm release decoding, and opportunities to improve path validation robustness and test coverage.

Comment thread runner/pkg/kube/secretpolicy.go Resolved
Comment thread runner/pkg/kube/secretpolicy.go Resolved
Comment thread runner/pkg/discovery/helm_list.go Resolved
Comment thread runner/pkg/kube/secretpolicy_test.go Resolved
- --raw paths are checked as written and after path.Clean, so dot segments
  and doubled slashes (serviceaccounts/sa/./token) cannot dodge the check.
- Mounted-secret patterns drop the leading slash so relative spellings
  (../var/run/secrets/...) match.
- list_helm_releases skips a release blob that decodes to no release name.
@blue4209211
blue4209211 added this pull request to the merge queue Nov 8, 2026
Merged via the queue into main with commit a7657f2 Nov 8, 2026
@blue4209211
blue4209211 deleted the fix/agent-secret-lockdown branch November 8, 2026 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants