Enable secret injection by sidecar

[tkarger]

Summary

Enable the injection of secrets by mounting an emptyDir volume and use the secret to setup a user.

Problem

The current setup is limited to secrets that exist on installation.

{{- if .Values.auth.usersExistingSecret }}
  - name: valkey-users-secret
    mountPath: /valkey-users-secret
    readOnly: true
{{- end }}

{{- if .Values.auth.usersExistingSecret }}
  - name: valkey-users-secret
    secret:
      secretName: {{ .Values.auth.usersExistingSecret }}
      defaultMode: 0400
{{- end }}

If you want to inject a secret on runtime by a sidecar, it'll result in an error cause the secret is not available on installation.
In our case we want to setup an admin user with the provided credentials in the secret.
The secret needs to be injected before running the init_config.yaml ConfigMap.

For e.g. the Vault agent injector an enhancement is required.

Proposed Solution

Create a volume with emptyDir

{{- if .Values.auth.injectSecret.enabled }}
- name: inject-secret
  emptyDir: {}
{{- end }}

Add a volumeMount to the containers

{{- if and .Values.auth.injectSecret.enabled }}
  {{ $dir := regexReplaceAll "/[^/]+$" .Values.auth.injectSecret.mountPath "/" }}
- name: inject-secret
  mountPath: "{{ $dir }}"
{{- end }}

This solution enables the injection of a secret under the auth.injectSecret.mountPath.

The ConfigMap needs to be adapted like e.g.

# Set user from injected secret with permissions
{{- if and .Values.auth.injectSecret.enabled .Values.auth.injectSecret.mountPath }}
INJECTED_PASSWORD=$(cat {{ .Values.auth.injectSecret.mountPath }})
PASSHASH=$(echo -n "$INJECTED_PASSWORD" | sha256sum | cut -f 1 -d " ")
echo "user {{ .Values.auth.injectSecret.username }} on #$PASSHASH {{ .Values.auth.injectSecret.permissions }}" >> /etc/valkey/users.acl
log "Set user {{ .Values.auth.injectSecret.username }} by injected secret"
{{- end }}

The replication configuration needs to be adapted like e.g.

{{- if .Values.auth.enabled }}
# Get the password for the replication user
{{- $replUsername := ""}}
{{- if .Values.auth.injectSecret.enabled }}
{{- $replUsername = .Values.auth.injectSecret.username }}
REPL_PASSWORD=$(cat {{ .Values.auth.injectSecret.mountPath }})
{{- else }}
{{- $replUsername = .Values.replica.replicationUser }}
{{- $replUser := index .Values.auth.aclUsers $replUsername }}
{{- $replPasswordKey := $replUser.passwordKey | default $replUsername }}
REPL_PASSWORD=$(get_user_password "{{ $replUsername }}" "{{ $replPasswordKey }}") || exit 1
{{- end }}

Happy to discuss!
Cheers,
Tim

[sgissi]

Thanks @tkarger. If the injector Pod takes long to run, won't the script at init_config.yaml will see empty data? It looks like a racing condition and also makes it very hard to follow the logic which is very important for security-related settings. Would it be possible to configure Vault to create a secret instead? That would force the init pod to wait for the secret, initialize the users and the secret can be removed afterwards.