Kubernetes · Cloud cost · 8 min read ·
Kubernetes rightsizing: where the money actually goes
Most Kubernetes clusters are paid for twice: once for the nodes and once for the requests that keep them half empty. A practical order for finding and fixing it.
Kubernetes bills are a function of nodes, and nodes are a function of requests. Not usage — requests. A pod that asks for 2 CPU and uses 0.2 still reserves 2 CPU on a node, and the cluster autoscaler adds nodes to satisfy reservations, not reality. This is why a cluster can show 30% CPU utilisation in Grafana while the autoscaler insists it is full.
The fix is unglamorous and works. This is the order we do it in.
1. Measure requests against usage, per workload
Before touching anything, get a table of every Deployment and StatefulSet with its CPU and memory requests next to its p95 usage over at least two weeks. Kubecost and OpenCost produce this; so does a Prometheus query over kube_pod_container_resource_requests and container_cpu_usage_seconds_total. Goldilocks (a VPA in recommendation mode) will do it for you and suggest values.
You are looking for two things:
- Reservation ratio: total requests divided by total usage. Above 2× on CPU is common and means the cluster is at least twice the size it needs to be.
- The long tail: the ten workloads that account for most of the excess. Rightsizing is Pareto-shaped; fix those first.
2. Fix requests, keep limits honest
Set CPU requests near p95 usage with headroom for growth, and memory requests at p99 plus a margin — memory is not compressible, and an OOM-killed pod costs more than the memory would have. Resist the reflex to set CPU limits everywhere: CPU throttling under limits produces latency that looks like an application bug. Memory limits are worth keeping.
Do this workload by workload with the owning team. It is their service; they know the seasonal peaks the graphs do not show.
3. Let the nodes follow the requests
Once requests reflect reality, the autoscaler can do its job. Two things make it do the job well:
- Node pool shape: a few well-chosen instance sizes bin-pack better than one large size. Karpenter (AWS) and the cluster autoscaler with multiple pools (AKS) both choose from what you give them.
- Scale-down settings: default scale-down is conservative. Tune utilisation thresholds and unneeded-time so nodes actually leave when the load does, and use pod disruption budgets so that scale-down is safe rather than disabled.
4. Scale the workloads, not just the nodes
Horizontal scaling on the right signal is cheaper than a fixed replica count sized for the peak. The HPA on CPU is the baseline; KEDA lets you scale on queue depth, request rate or a schedule, which is what most workloads actually follow. Non-production namespaces that scale to zero overnight are the single most reliable saving in most estates.
5. Stop paying for what is idle
Every estate has PersistentVolumes with no pod, load balancers pointing at nothing, and namespaces belonging to projects that ended. A weekly report of unattached volumes and unused services, with an owner and an expiry, removes this class of cost permanently.
6. Commit only once the shape is stable
Reservations and savings plans lock in a baseline. Buy them after the rightsizing, not before, or you commit to the waste. Cover the steady floor of the cluster, leave the variable part on demand, and revisit quarterly.
What to expect
Rightsized estates typically run at materially higher node utilisation without any change in application behaviour. The exact number depends on how oversized the requests were, which is why step one is measuring. Nothing here requires re-architecting an application; all of it requires the discipline to keep the numbers visible afterwards, which is what a dashboard of reservation ratio per namespace is for.
If you would like the measurement done for you, a cloud cost assessment produces exactly this table, ranked, in about two weeks.