Services and Reaching Your Pods
Pod IPs change constantly, so nothing addresses them directly. ClusterIP, NodePort and LoadBalancer, cluster DNS, and how a Service finds its pods.
Pod IPs change constantly, so nothing addresses them directly. ClusterIP, NodePort and LoadBalancer, cluster DNS, and how a Service finds its pods.
Your Deployment keeps three pods running, which is the good news. The bad news is that nothing can reliably talk to them. Each pod has its own IP address, those addresses change every time a pod is replaced, and there are three of them anyway — so which one would you connect to?
The Service is the answer to both halves. By the end of this
lesson you will have one stable name in front of a changing set of
pods, you will know the three types and when each is right, and you
will know why nslookup inside a pod is the fastest way to settle any
argument about cluster networking.
Every pod gets a real, routable IP inside the cluster:
kubectl get pods -l app=web -o wideNAME READY STATUS IP NODE
web-6c9f7d4b58-mn4pq 1/1 Running 10.244.1.7 learning-worker
web-6c9f7d4b58-vz8lt 1/1 Running 10.244.2.4 learning-worker2
web-6c9f7d4b58-jd5rw 1/1 Running 10.244.1.8 learning-workerYou can connect to 10.244.1.7 from another pod, and it works. It
also stops working the moment that pod is replaced — which happens on
every deploy, every node drain, every crash. The replacement has a new
address, and nothing told you.
So the problem is not "how do I reach a pod". It is "how do I reach the application, given that its pods are a moving target". That is a job for something with a stable identity and a live view of which pods are currently healthy.
apiVersion: v1
kind: Service
metadata:
name: web
spec:
selector:
app: web
ports:
- port: 80
targetPort: 80kubectl apply -f service.yaml
kubectl get service webNAME TYPE CLUSTER-IP PORT(S) AGE
web ClusterIP 10.96.142.31 80/TCP 6sThat CLUSTER-IP is stable for the life of the Service. So is the
name web. Neither changes when pods do.
Two fields do the work.
selector is a label query — the same mechanism a Deployment uses
to find its pods. Any pod carrying app: web is a candidate. Note
that the Service does not reference the Deployment at all; it
references labels. A Service can front pods from two Deployments, or
from none, and it neither knows nor cares.
ports maps the Service's port to the pods'. port is what
clients connect to on the Service. targetPort is the port on the
pod. They are often the same and do not have to be:
ports:
- port: 80 # clients use this
targetPort: 8000 # the container listens on thisThe Service is the stable front. Behind it is a list that changes constantly, and it is a real object you can look at:
kubectl get endpointslices -l kubernetes.io/service-name=web
kubectl describe service webName: web
Selector: app=web
Type: ClusterIP
IP: 10.96.142.31
Port: 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.7:80,10.244.2.4:80,10.244.1.8:80Those endpoints are maintained by a controller watching pods that match the selector. Scale the Deployment and the list grows. Delete a pod and it is removed before the pod dies, then the replacement is added when it becomes ready.
This is the single most useful diagnostic in cluster networking:
kubectl describe service web | grep EndpointsAn empty Endpoints list means nothing will ever answer, and it
has only two causes.
Does any pod carry the selector's labels?
kubectl get pods -l app=web. Nothing back means the
selector and the pod labels disagree — and Kubernetes will
never tell you, because a Service may legitimately exist
before its pods.
Are the matching pods ready?
A pod joins the endpoint list when it passes its readiness
probe and leaves the moment it fails one. READY 0/1 means
it exists and receives nothing.
Both fine? Then the Service is not your problem
Endpoints listed means traffic is being delivered. Look at the application, the port numbers, or the client.
You have seen the cluster IP, and you will almost never type one.
Every Service gets a DNS name, served by CoreDNS — the component you
saw running in kube-system.
From any pod in the same namespace, the Service's name is enough:
kubectl run tmp --rm -it --image=busybox:1.36 --restart=Never -- sh
# inside:
wget -qO- http://web
nslookup webName: web.default.svc.cluster.local
Address 1: 10.96.142.31 web.default.svc.cluster.localThe full form shows the structure:
web.default.svc.cluster.local
^^^ ^^^^^^^ ^^^ ^^^^^^^^^^^^^
| | | └── cluster domain
| | └── it is a Service
| └── namespace
└── Service nameWhich gives you three ways to refer to the same thing, and a rule for picking:
web # same namespace — use this
web.default # a different namespace
web.default.svc.cluster.local # fully qualified, in config filesTwo details that matter in practice. The name resolves whether or
not any pod is behind it, so a successful nslookup proves the
Service exists and nothing more — check Endpoints for the rest. And
DNS gives you the Service's address, not a pod's; the load
balancing happens after that, in the kernel.
The cluster IP is a virtual address. No network interface has it, and nothing listens on it.
When a pod sends a packet to 10.96.142.31:80, rules programmed on
that node by kube-proxy rewrite the destination to one of the
current endpoints, chosen roughly at random, and the packet goes
straight to that pod. No proxy process sits in the path; it is
kernel-level rewriting, which is why it costs almost nothing.
Two consequences worth knowing:
Balancing is per connection, not per request. A client that opens one HTTP connection and keeps it alive — which is most modern clients — sends every request to the same pod. That surprises people who scale up and see load stay uneven. Fixing it means either short-lived connections or a real layer-7 proxy.
There is no health checking in the data path. Endpoints are managed by readiness, so a pod that fails between health checks receives traffic until the next one. Your clients should retry.
Same object, one extra field, three different scopes.
Internal only. The default.
Your database, your cache, one service calling another. This is the right answer for almost everything.
A high port on every node.
Useful on a local cluster or for a quick test. Awkward in production: the port range is 30000–32767 and clients would have to know node addresses.
A real external address.
One cloud load balancer per Service, each with its own address
and its own monthly bill. On a local cluster nothing provides
one, so EXTERNAL-IP sits at <pending> forever.
The default, and the right answer for anything internal:
spec:
type: ClusterIP # can be omittedReachable only from inside the cluster. Your database Service, your cache, one microservice calling another — all ClusterIP.
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30080 # optional; 30000-32767NAME TYPE CLUSTER-IP PORT(S) AGE
web NodePort 10.96.142.31 80:30080/TCP 4sKubernetes opens port 30080 on every node, and traffic to any node's address on that port reaches the Service. Useful for a local cluster or a quick test; awkward in production, because the port range is high-numbered and clients would need to know node addresses.
spec:
type: LoadBalancerAsks the infrastructure for a real external load balancer and puts its
address in front of the Service. On a cloud provider you get a public
IP within a minute. On a local cluster nothing provides one, so the
EXTERNAL-IP sits at <pending> forever, which is not a bug.
Bad — the Service selects a label no pod carries:
# deployment.yaml
template:
metadata:
labels:
app: web
tier: frontend
---
# service.yaml
spec:
selector:
app: web-frontendGood — the selector matches labels the pods actually have:
spec:
selector:
app: webBoth objects apply cleanly. Kubernetes does not validate that a Service's selector matches anything, because a Service may legitimately be created before its pods. So you get a healthy-looking Service, a resolving DNS name, and every connection refused — with no error anywhere.
A Service selector may also be narrower than the pods' labels, which
is fine and useful: selecting app: web catches pods labelled
app: web, tier: frontend. It must never mention a label the pods do
not have.
Your laptop is not in the cluster network, so a ClusterIP is not reachable from it. Two ways in during development:
kubectl port-forward service/web 8080:80A tunnel from your machine to one pod behind the Service, for as long as the command runs. Note it picks a pod — it is not load balanced.
kubectl run tmp --rm -it --image=busybox:1.36 --restart=Never -- shA throwaway pod inside the cluster, from which every Service name resolves. This is the honest way to test cluster networking, because it tests it from where your application actually sits.
One variant worth recognising. Setting clusterIP: None produces a
headless Service: no virtual IP, and DNS returns the pod addresses
directly, one record per pod.
spec:
clusterIP: None
selector:
app: postgresThat is what you want when a client needs to know about individual pods rather than being balanced across them — a database replica set, or anything where the caller does its own discovery. It is the addressing model StatefulSets use.
apiVersion: v1
kind: Service
metadata:
name: web
spec:
type: ClusterIP # ClusterIP | NodePort | LoadBalancer
selector:
app: web # must match labels the pods HAVE
ports:
- name: http
port: 80 # what clients connect to
targetPort: 8000 # the container's port (or its name)# Everyday
kubectl apply -f service.yaml
kubectl get services
kubectl describe service web # ...and read Endpoints
# The one diagnostic that matters
kubectl describe service web | grep Endpoints
# empty -> no pod matches the selector, or none is ready
# listed -> the Service is fine; look elsewhere
kubectl get pods -l app=web # do any pods match?
kubectl get endpointslices -l kubernetes.io/service-name=web
# Testing from inside the cluster (the honest way)
kubectl run tmp --rm -it --image=busybox:1.36 --restart=Never -- sh
# wget -qO- http://web
# nslookup web
# Reaching it from your own machine
kubectl port-forward service/web 8080:80 # one pod, not balanced
# DNS names
# web same namespace
# web.default another namespace
# web.default.svc.cluster.local fully qualified
# The three types
# ClusterIP internal only — the default, and usually right
# NodePort a high port on every node — local clusters, tests
# LoadBalancer a real external address — one per Service, costly
# Headless, for per-pod addressing
# clusterIP: None -> DNS returns pod IPs, no balancingYour application has a stable internal address. What it does not have
is a way for anyone outside the cluster to reach it on a real hostname
over HTTPS — NodePort is a high-numbered port and LoadBalancer is
one address per Service.
Ingress is the answer, and it is the next lesson: one entry point,
routing by hostname and path to as many Services as you like. Before
that, run the busybox pod and try nslookup on a Service, then
delete every pod behind it and try again. Watching the name resolve
while nothing answers is what makes the endpoints check stick.