Labels, Selectors and Namespaces
How Kubernetes groups things without a hierarchy: labels as the glue every controller uses, selectors as the query, and namespaces as the boundary around a whole environment.
How Kubernetes groups things without a hierarchy: labels as the glue every controller uses, selectors as the query, and namespaces as the boundary around a whole environment.
You have used labels three times now without being told what they
are. A Deployment finds its pods with them. A Service finds its
endpoints with them. kubectl get pods -l app=web filtered on them.
That is not a coincidence, and it is worth a lesson of its own, because labels are how Kubernetes relates objects to each other at all. There are no foreign keys and no parent pointers. By the end of this lesson you will know how to label things so the rest of the system can find them, and how namespaces divide a cluster into parts that do not collide.
Coming from almost any other system, the surprising part is what is missing. A Service does not contain a list of its pods. A Deployment does not hold references to the pods it created. Delete a Deployment and look at what it stored about its children — there is nothing.
Instead, objects carry labels, and other objects run queries over them:
pods carry: app=web, tier=frontend
Service asks for: app=web -> finds all three
Deployment asks for: app=web -> counts them, makes more
you ask for: tier=frontend -> gets a listingThis is loose coupling taken seriously, and it has real consequences. A Service can front pods from two different Deployments during a migration. A pod can be pulled out of a Service's rotation by changing one of its labels, without deleting anything. Nothing needs to know what created anything else.
Labels are free-form, which means an unlabelled scheme drifts into
app, application and service-name on three objects in one
namespace. Kubernetes documents a set of recommended names, and using
them costs nothing:
metadata:
labels:
app.kubernetes.io/name: notes-api
app.kubernetes.io/instance: notes-api-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: notes
app.kubernetes.io/version: 1.4.2
app.kubernetes.io/managed-by: helmPrefixed names like these are conventional and slightly verbose. For your own work a shorter scheme is fine, as long as it is a scheme:
metadata:
labels:
app: notes-api # which application
component: api # which part of it
environment: production # whereTwo rules make labels behave. Pick the keys once and use them everywhere, so a query works across an application rather than on whichever objects happen to agree. And keep the set that a Deployment or Service selects on stable, because a selector cannot be changed after creation.
Two forms of query. Equality-based is what you have used:
Set-based is more expressive:
Both forms exist in manifests too, and this is where set-based selection earns its keep:
matchLabels and matchExpressions are combined with AND. Operators
are In, NotIn, Exists and DoesNotExist.
Some everyday uses that are worth having to hand:
kubectl get pods --show-labels deserves special mention: when a
selector finds nothing, it is the command that tells you what the
labels really are, as opposed to what you believe you set.
Because membership of a Service is decided by a live query, you can change it by editing one label — with nothing restarting.
Take a pod that is misbehaving in production. You want to look at it, and you do not want it serving traffic:
Three things happen at once, from that one edit.
It leaves the Service
The pod no longer matches the Service's selector, so it is removed from the endpoints and receives no traffic.
The Deployment replaces it
It no longer matches the Deployment's selector either, so the Deployment counts two pods instead of three and creates a new one.
The pod itself is untouched
Still running, still holding whatever state you wanted to inspect. Nothing restarted. Delete it when you are done.
Labels group objects. Namespaces divide the cluster itself.
A namespace is a scope for names: two objects of the same kind can share a name if they are in different namespaces, and cannot if they are in the same one. It is a boundary for resource quotas, for role-based access control, and for network policy.
kube-system is where the control plane components run. default is
where everything you have created so far went, because you never said
otherwise.
Then every command needs to know which namespace it is talking about:
Forgetting -n is the most common namespace mistake, and it produces
No resources found in default namespace — which reads like the object
failed to create. Set a default for your context instead:
The word "isolation" oversells it, and the gap catches people.
Names, listings, quotas, permissions.
Object names, kubectl listings by default, ResourceQuota
and LimitRange, RBAC permissions, and the middle part of a
Service's DNS name.
Networking. Not at all.
By default every pod can reach every other pod in the cluster, whatever namespace either is in.
A pod in staging can connect to the database in
production if it knows the name — and the names are
trivially guessable.
The guessable name looks like this, from any pod anywhere:
Restricting that needs a NetworkPolicy, which is a separate object and requires a network plugin that implements it. Namespaces are an organisational boundary; a security boundary is something you build on top.
Some objects are also cluster-scoped and live in no namespace at all — Nodes, PersistentVolumes, StorageClasses, ClusterRoles, Namespaces themselves:
Deleting a namespace deletes everything inside it, and that is not reversible:
Which is genuinely useful for a per-branch environment, and worth being careful with everywhere else.
Two patterns are common and one is a trap.
By environment — staging, production — inside one cluster. It
works, and the shared control plane and shared node pool mean a
misbehaving staging workload can affect production. Fine for a small
team; separate clusters are the safer answer once the stakes rise.
By team or application — payments, search, notes. This is
what namespaces are best at: each group gets its own names, its own
quota, and its own permissions.
The trap is a namespace per microservice in one application. Every call becomes cross-namespace, every name has to be fully qualified, and you have paid the ceremony for nothing. Services in one application belong in one namespace, distinguished by labels.
You can now organise a cluster and relate objects to each other deliberately rather than by accident. Next comes configuration — ConfigMaps and Secrets — which is where your application finally gets its database URL and its credentials.
Before that, try the quarantine trick on a pod behind a Service, with
kubectl get pods --watch running. Seeing a replacement appear the
instant you change a label makes the loose coupling concrete in a way
no diagram does.
NAME STATUS AGE
default Active 2h
kube-node-lease Active 2h
kube-public Active 2h
kube-system Active 2hkubectl get pods -l app=web
kubectl get pods -l app=web,tier=frontend # AND, not OR
kubectl get pods -l app!=webkubectl get pods -l 'environment in (staging,production)'
kubectl get pods -l 'tier notin (cache)'
kubectl get pods -l 'app' # has the key at all
kubectl get pods -l '!app' # does not have itspec:
selector:
matchLabels:
app: web
matchExpressions:
- key: environment
operator: In
values: [staging, production]kubectl get all -l app=notes-api # every object in it
kubectl logs -l app=notes-api --tail=20 # every replica's logs
kubectl delete pods -l app=notes-api # restart them all
kubectl get pods --show-labels # what is actually set
kubectl label pod web tier=frontend # add one
kubectl label pod web tier- # remove onekubectl label pod web-6c9f7d4b58-mn4pq app=web-quarantine \
--overwritekubectl get namespacesapiVersion: v1
kind: Namespace
metadata:
name: stagingkubectl apply -f namespace.yaml
kubectl create namespace staging # or imperativelykubectl get pods -n staging
kubectl get pods --all-namespaces # or -A
kubectl apply -f k8s/ -n stagingkubectl config set-context --current --namespace=staging
kubectl config view --minify | grep namespace# from any pod, anywhere in the cluster
wget -qO- http://api.production.svc.cluster.localkubectl api-resources --namespaced=true
kubectl api-resources --namespaced=falsekubectl delete namespace staging # everything in it, gone# Labels: for selecting. Annotations: for everything else.
metadata:
name: notes-api
namespace: production # self-contained beats a forgotten -n
labels:
app: notes-api # keep the selected set STABLE
component: api
environment: production
annotations:
git-commit: 8f3c1a9 # never selected on# Selectors in a manifest
spec:
selector:
matchLabels:
app: notes-api
matchExpressions:
- key: environment
operator: In # In | NotIn | Exists | DoesNotExist
values: [staging, production]# Selecting on the command line
kubectl get pods -l app=web
kubectl get pods -l app=web,tier=frontend # AND
kubectl get pods -l 'environment in (staging,production)'
kubectl get pods -l '!app' # missing the key
kubectl get pods --show-labels # what is really set
# Acting on a whole application
kubectl get all -l app=notes-api
kubectl logs -l app=notes-api --tail=20
kubectl delete pods -l app=notes-api # rolling restart
# Editing labels
kubectl label pod web tier=frontend
kubectl label pod web tier=backend --overwrite
kubectl label pod web tier- # remove
# Quarantine a pod: out of traffic, replaced, still alive
kubectl label pod web-abc123 app=web-quarantine --overwrite
# Namespaces
kubectl get namespaces
kubectl create namespace staging
kubectl get pods -n staging
kubectl get pods -A # all namespaces
kubectl config set-context --current --namespace=staging
kubectl delete namespace staging # everything in it
# What a namespace scopes
# yes: names, default listings, quotas, RBAC, Service DNS
# no: pod-to-pod networking — that needs a NetworkPolicy
kubectl api-resources --namespaced=false # cluster-scoped
# Cross-namespace Service name
# api.production.svc.cluster.local