ConfigMaps and Secrets
Getting configuration into a pod as environment variables or mounted files, and the honest truth about how much protection a Secret does and does not give you.
Getting configuration into a pod as environment variables or mounted files, and the honest truth about how much protection a Secret does and does not give you.
Your image is the same in every environment — that was the whole point of the Docker course. So the database URL, the log level and the API credentials have to arrive from somewhere else, and in Kubernetes that somewhere has two names: ConfigMap for ordinary configuration and Secret for credentials.
By the end of this lesson you will know both, the two ways each can reach a pod, why one of those ways updates live and the other does not, and — plainly, without hand-waving — how much protection a Secret actually gives you.
A ConfigMap is a named bag of key-value pairs, stored in the cluster:
apiVersion: v1
kind: ConfigMap
metadata:
name: notes-config
data:
LOG_LEVEL: debug
DATABASE_HOST: postgres
DATABASE_PORT: '5432'
FEATURE_DARK_MODE: 'true'A Secret is the same object with a different name and slightly different handling:
apiVersion: v1
kind: Secret
metadata:
name: notes-secrets
type: Opaque
stringData:
DATABASE_PASSWORD: devpass
STRIPE_API_KEY: sk_test_51H8xQ2eZvKYlo2Ckubectl apply -f config.yaml
kubectl get configmaps
kubectl get secretsTwo details about the YAML itself. Values must be strings —
'5432' and 'true' need their quotes, because YAML would otherwise
give you a number and a boolean and the API server would reject them.
And stringData lets you write a Secret in plain text; Kubernetes
base64-encodes it on the way in. The data field is the encoded form,
which is what you get back when you read the object.
Creating them imperatively is often easier, and the --dry-run trick
turns that into a manifest:
kubectl create configmap notes-config \
--from-literal=LOG_LEVEL=debug \
--from-literal=DATABASE_HOST=postgres
kubectl create configmap notes-config \
--from-env-file=.env.development
kubectl create secret generic notes-secrets \
--from-literal=DATABASE_PASSWORD=devpass
kubectl create configmap nginx-conf \
--from-file=nginx.confThe first of two mechanisms, and the one to reach for by default.
Pull in individual keys, which lets you rename as you go:
spec:
containers:
- name: api
image: notes-api:0.1.0
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: notes-config
key: LOG_LEVEL
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: notes-secrets
key: DATABASE_PASSWORDOr pull in everything at once, which is much shorter:
spec:
containers:
- name: api
image: notes-api:0.1.0
envFrom:
- configMapRef:
name: notes-config
- secretRef:
name: notes-secretsenvFrom turns every key into an environment variable of the same
name. It is the right choice when your ConfigMap exists for this
application; the explicit form is better when you are borrowing one
key from a shared ConfigMap, or when the names do not line up.
Confirm what arrived:
kubectl exec deploy/api -- env | sortThe second mechanism projects a ConfigMap or Secret into the filesystem, one file per key:
spec:
volumes:
- name: config
configMap:
name: nginx-conf
- name: credentials
secret:
secretName: notes-secrets
containers:
- name: api
image: notes-api:0.1.0
volumeMounts:
- name: config
mountPath: /etc/nginx/conf.d
readOnly: true
- name: credentials
mountPath: /etc/secrets
readOnly: truekubectl exec deploy/api -- ls /etc/secrets
# DATABASE_PASSWORD STRIPE_API_KEY
kubectl exec deploy/api -- cat /etc/secrets/DATABASE_PASSWORDThe choice between the two mechanisms comes down to one property.
Read once, at start-up. Never updated.
A process reads its environment when it starts and cannot be given a new one.
Right for the common case — a handful of scalar settings the application reads once.
Refreshed within about a minute.
The kubelet re-syncs them, so the file on disk changes under a running pod. Your application still has to re-read or watch it — but the possibility exists.
Also the only sensible way to deliver a whole file: an nginx config, a TLS certificate, a JSON settings blob.
You can also project selected keys to specific filenames:
volumes:
- name: config
configMap:
name: nginx-conf
items:
- key: nginx.conf
path: default.confBad — the ConfigMap is updated and the running pods keep the old values:
kubectl apply -f configmap.yaml # LOG_LEVEL: debug -> info
kubectl get pods # unchanged, still on debugGood — the pods are replaced, so they read the new values on start-up:
kubectl apply -f configmap.yaml
kubectl rollout restart deployment/api
kubectl rollout status deployment/apiNothing in Kubernetes restarts pods when a ConfigMap changes. The
apply succeeds, kubectl get configmap shows the new value, and the
application behaves exactly as before — which is the worst kind of
failure, because every piece of evidence says the change landed.
kubectl rollout restart is the deliberate fix. The durable fix is to
make the configuration part of the pod template so a change triggers a
rollout on its own:
spec:
template:
metadata:
annotations:
checksum/config: '4f2a91c4e8b3d7'Any change to the template — including an annotation — starts a normal rolling update. Helm and Kustomize both generate this checksum for you; by hand, changing the ConfigMap's name on each release has the same effect and leaves an audit trail.
The honest answer, because half-knowing this is dangerous.
Secrets are base64-encoded, not encrypted. Base64 is an encoding, reversible by anyone:
kubectl get secret notes-secrets -o jsonpath='{.data.STRIPE_API_KEY}' \
| base64 -dReal, and limited.
A separate object type, so RBAC can grant ConfigMaps and withhold Secrets.
Values hidden from get and describe, so they stay out of
screen shares. Held in memory rather than written to a node's
disk. Distributed only to nodes running a pod that needs them.
And encrypted in etcd — if encryption at rest is configured. Off by default on a self-managed cluster, on by default on most managed ones.
Anyone with read access has the value.
Base64 is an encoding, not encryption. One command decodes it.
And anyone who can exec into a pod that mounts the Secret
can simply read the file.
type: Opaque is the general case. Two others exist because
Kubernetes itself consumes them.
TLS certificates, which is what the Ingress lesson used:
kubectl create secret tls web-tls --cert=tls.crt --key=tls.keyRegistry credentials, for pulling from a private registry:
kubectl create secret docker-registry regcred \
--docker-server=ghcr.io \
--docker-username=antonii-devfoxlabs \
--docker-password="$GITHUB_TOKEN"spec:
imagePullSecrets:
- name: regcred
containers:
- name: api
image: ghcr.io/antonii-devfoxlabs/notes-api:0.1.0Without that, a private image produces ImagePullBackOff with
unauthorized in the pod's events — a failure that looks like a bad
image name and is not.
Four things worth doing from the start.
Split configuration by how it changes. One ConfigMap per application for its own settings; a shared one only for values that are genuinely shared. Secrets separate from ConfigMaps always, because the access rules differ.
Mount read-only. readOnly: true on every config and secret
volume. Nothing should be writing to its own configuration.
Fail at start-up on a missing value. The same rule as in Docker: a
required variable read with bracket access, so the pod crashes with a
clear message rather than serving errors. CrashLoopBackOff with
KeyError: 'DATABASE_URL' in the logs is a five-second diagnosis.
Keep a committed example. A configmap.example.yaml listing every
key with a fake value tells the next person what the application needs,
which is exactly what the real Secret cannot.
apiVersion: v1
kind: ConfigMap
metadata:
name: notes-config
data:
LOG_LEVEL: debug
DATABASE_PORT: '5432' # values must be STRINGS
---
apiVersion: v1
kind: Secret
metadata:
name: notes-secrets
type: Opaque # or kubernetes.io/tls, dockerconfigjson
stringData: # plain text in; encoded at rest
DATABASE_PASSWORD: devpass# Into a pod as environment variables (no live updates)
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef: { name: notes-config, key: LOG_LEVEL }
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef: { name: notes-secrets, key: DATABASE_PASSWORD }
envFrom: # ...or every key at once
- configMapRef: { name: notes-config }
- secretRef: { name: notes-secrets }
# Into a pod as files (refreshed while running)
volumes:
- name: config
configMap: { name: nginx-conf }
- name: credentials
secret: { secretName: notes-secrets }
containers:
- name: api
volumeMounts:
- { name: config, mountPath: /etc/nginx/conf.d, readOnly: true }
- { name: credentials, mountPath: /etc/secrets, readOnly: true }# Creating
kubectl create configmap notes-config --from-literal=LOG_LEVEL=debug
kubectl create configmap notes-config --from-env-file=.env.development
kubectl create configmap nginx-conf --from-file=nginx.conf
kubectl create secret generic notes-secrets \
--from-literal=DATABASE_PASSWORD=devpass
kubectl create secret tls web-tls --cert=tls.crt --key=tls.key
kubectl create secret docker-registry regcred \
--docker-server=ghcr.io --docker-username=u --docker-password="$T"
# ...or generate the manifest instead of applying
kubectl create configmap notes-config --from-env-file=.env \
--dry-run=client -o yaml > configmap.yaml
# Inspecting
kubectl get configmap notes-config -o yaml
kubectl describe configmap notes-config
kubectl exec deploy/api -- env | sort # what the pod got
kubectl get secret notes-secrets \
-o jsonpath='{.data.DATABASE_PASSWORD}' | base64 -d
# After changing a ConfigMap or Secret
kubectl rollout restart deployment/api # env vars need this
kubectl rollout status deployment/api
# mounted FILES refresh on their own within ~a minute
# ENVIRONMENT variables never do
# What a Secret is
# base64-encoded, not encrypted; decode with `base64 -d`
# separate RBAC, hidden from get/describe, in-memory when mounted
# encrypted in etcd only if encryption at rest is configured
# never commit one: use Sealed Secrets, SOPS, or External SecretsConfiguration is in place. The remaining gap for a real application is storage: a pod's filesystem dies with it, exactly as a container's did in Docker, so a database needs something that outlives the pod. That is PersistentVolumes and claims, which is next.
Before moving on, do the experiment that makes the update rule stick:
change a ConfigMap value, confirm with kubectl get configmap that it
changed, then run kubectl exec deploy/api -- env and watch the pod
report the old one.