This article was originally published at sivaro.in

Kubernetes Node Sizing for Cost Efficiency

Slug: kubernetes-node-sizing-for-cost-efficiency

Every quarter, a client opens a ticket that reads the same way: "Our EKS bill doubled. We didn't change anything." Nine times out of ten, they didn't. The workloads didn't grow. The pods didn't move. But five m5.xlarge nodes that should have been retired months ago were still carrying traffic at 12% CPU utilization. That's not a Kubernetes problem. It's a node sizing problem.

Kubernetes node sizing for cost efficiency is the practice of matching machine types, node pool boundaries, and allocation policies to what your pods actually request — instead of what you provisioned in a hurry back in 2023. It's the difference between a cluster that runs and a cluster that pays for itself. In this guide, I'll walk through the decisions I've made running data infrastructure at SIVARO since 2018: what worked, what failed, and where I wasted real money so you don't have to.

The 40% Tax Hidden in Your Node Pool

Most teams don't have a cost problem. They have a waste problem.

Let me give you a concrete example. In March 2025, a fintech client in Bangalore asked us to audit their EKS cluster before an IPO cost review. They had 23 nodes spread across three node groups. Total monthly spend: $31,000. Average node utilization across 30 days: 17% CPU, 41% memory.

The numbers were not an accident. Each node group had been sized in 2022 by someone who picked the "safe" instance from a blog post. They added headroom on top of headroom. No one looked again.

We ran a simple analysis. Their pods, if packed efficiently, fit on seven nodes. Seven. The other sixteen were running because of an old cluster-autoscaler config that scaled on sum(requests) — and their requests were written like a wish list, not a measurement.

This pattern is everywhere. The CloudBolt guide on Kubernetes cost optimization frames it correctly: most cost overruns trace back to overprovisioning, not to instance pricing. You can negotiate reserved instance discounts all day. If your fleet is 50% idle, the discount means you're paying half price for nothing.

Requests and Limits Are Your Cost Ledger

Here's the thing nobody tells you on day one: Kubernetes bills you on requests, not on usage.

The scheduler doesn't look at your actual CPU consumption. It looks at spec.containers[].resources.requests and decides where to place the pod. The node must have that much allocatable capacity. So when your team sets requests at 1000m CPU for a service that averages 150m, you're paying for four times the compute you need on every single pod.

Look at any busy cluster and you'll see the gap:

kubectl top nodes --sort-by=cpu

NAME                       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
ip-10-1-12-34.ec2.internal   850m        21%    6144Mi          39%
ip-10-1-56-78.ec2.internal   920m        23%    5888Mi          38%

Requests say those nodes need to handle 4.2 cores each. Reality says they're using less than one. I've stopped counting how many clients respond to this with "well, we need headroom for spikes." Do you? Or do you need a burst strategy?

A better posture: set requests to the steady-state p95 of your workload, and let the Horizontal Pod Autoscaler handle spikes by adding pods. The node doesn't need to pre-reserve your worst-case memory footprint. It needs to survive your realistic one.

Bin Packing: The Algorithm You Already Paid For

The scheduler uses a bin-packing algorithm called First Fit Decreasing. It sorts pending pods by request size and places them on the first node that fits. It's not trying to minimize nodes. It's trying to place pods, period.

That's why node sizing matters more than scheduling nuances. If your nodes are homogenous — say, all m5.xlarge — the scheduler has one bucket shape, and your pods will be scattered with gaps everywhere. Small pods don't stack into four-core nodes cleanly. Large pods don't fit on two-core nodes at all.

Karpenter approaches this differently. It computes an optimal packing per node before it creates it, instead of reusing a static node group. The Karpenter scheduling docs describe this well: Karpenter considers each pending pod's resource requirements, topology spread, and taints, then picks an instance type that fits the whole batch. That's a different game than Cluster Autoscaler, which just adds another copy of whatever your node group template defines.

I've seen teams switch from static node groups to Karpenter and drop node count by 35% without touching a single deployment. Not because Karpenter is magic. Because it sizes nodes to the pod batch instead of forcing pods to fit a pre-cut node.

One Fat Node vs. Ten Skinny Nodes

Most people think bigger nodes are cheaper. They're wrong, and it costs them.

Let's compare r5.large and r5.4xlarge in us-east-1. The 4xlarge is about 5.5x the price and has 4x the resources. On paper, you're paying a premium for larger machines. But the real cost is granularity — a single 4xlarge can only be filled in chunks of at least one pod. If your pods request 512m CPU each, the largest node that packs cleanly is the one where 512m divides evenly into allocatable CPU. Larger nodes create fractional waste at the edges.

We tested this at SIVARO in 2024. Two clusters, identical workloads, one using m5.2xlarge, the other using m5.large. Same total vCPU footprint per cluster. The m5.large cluster ran 22% cheaper because the scheduler could pack pods with less residual slack.

But the opposite is true for monolithic workloads. A batch job that needs 6 cores and 24 GiB of memory will never fit on an m5.large (2 cores, 8 GiB). You need the fat node. The right answer is not one node size — it's two or three sizes built around your actual pod shapes.

Karpenter Changes the Math

Karpenter's model inverts the problem. Instead of you predicting node demand, you give it a list of acceptable instance types and let it choose. The list matters more than you think.

I've seen NodePools with forty instance families in spec.provider because someone copied a sample config. Karpenter may then pick a spot p4d.24xlarge to run 12 small web pods. Technically valid. Practically absurd — and expensive at the margin.

Our NodePool templates look like this:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    metadata:
      labels:
        workload: general
    spec:
      requirements:
        - key: "kubernetes.io/arch"
          operator: In
          values: ["amd64"]
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot", "on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values:
            - "m5.large"
            - "m5.xlarge"
            - "m6i.large"
            - "m6i.xlarge"
            - "c6i.large"
            - "c6i.xlarge"
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy:
      when: "Underutilized"
      consolidateAfter: 60s

Notice what's missing: the 4xlarge machines, the compute-optimized monsters, everything with more than 4 vCPUs. We deliberately narrow the list to sizes that match our pod inventory. The AWS blog on optimizing compute costs with Karpenter consolidation makes the same point — the consolidation feature only works if your instance list is broad enough to consolidate onto, but constrained enough to stay relevant.

Consolidation: The Feature Everyone Skips Setting Up

Karpenter's consolidation feature is the closest thing to free money in Kubernetes. It watches running nodes and asks: "could these pods fit on fewer or cheaper nodes?" If yes, it drains and replaces them.

But it's off by default. And I've audited at least six clusters where someone deployed Karpenter, never enabled consolidation, and then complained about the bill. It's in the spec, in plain sight.

There are two consolidation policies: WhenUnderutilized and WhenEmpty. The first replaces nodes that are partially used with smaller ones that still fit the pods. The second only removes completely empty nodes. Use WhenUnderutilized. It's the one that actually saves money.

One warning: consolidation can churn your cluster if you set consolidateAfter too low. We ran it at 15 seconds once. Bad idea. Every prometheus scrape spike triggered disruption, and we burned more time on pod rescheduling than we saved. 60 to 120 seconds is a sane default. The AWS repost article on EKS cost optimization with Karpenter walks through the same tradeoff with real numbers.

Spot Instances: The 70% Discount That Flips Your Nodes

Spot instances are the biggest lever in kubernetes overprovisioning cost reduction with karpenter. You can cut your EC2 spend by 60-70% on the same hardware.

The problem is that spot pools get reclaimed. Amazon can terminate your spot instance with two minutes of notice. In a stateless web cluster, that's fine — pods re-schedule elsewhere. In a stateful workload running Kafka or a database, it's a disaster.

Our approach:

• Spot for stateless workloads. Web services, workers, cron jobs, CI runners.
• On-demand for stateful workloads. Databases, message brokers, control planes.
• Never mix both in the same NodePool unless you know exactly which pods tolerate spot interruption.

The CloudBolt Kubernetes cost optimization guide has a section on spot strategies that lines up with our experience: spot is not a universal answer. It's a tool for workloads designed to be rescheduled.

But here's a contrarian take: for our AI inference workloads in 2025 and 2026, we moved toward spot on GPU instances. The reasoning: inference pods are stateless, the API gateway around them already does retries, and the 70% discount let us buy three times the GPU capacity for the same budget. The p95 latency barely moved, because the few spot reclamations got absorbed by the retry layer.

The CPU-to-Memory Ratio Nobody Calculates

Most instance families come in a fixed CPU-to-memory ratio. m5.large is 2 vCPU / 8 GiB. r5.large is 2 vCPU / 16 GiB. c5.large is 2 vCPU / 4 GiB.

Your workload has its own ratio. If your pods average 1 vCPU per 2 GiB, you want m5-family ratios. If you're memory-heavy — caches, in-memory databases — you want r5. If you're CPU-heavy, you want c5 or m7g.

Yet I keep finding clusters where everything runs on m5 because it's the default in every Terraform example. Then they wonder why nodes are memory-constrained and CPU-idle.

The fix is a quick script against your monitoring data:

kubectl top pods --all-namespaces --containers | awk '{cpu += $3; mem += $4} END {printf "Total CPU: %.2f cores\nTotal Memory: %.2f GiB\nRatio: 1 CPU per %.2f GiB\n", cpu/1000, mem/1024, (mem/1024)/(cpu/1000)}'

Compare that ratio to your instance families. If your ratio is 1:3.5 and you're running c5 nodes, you're bleeding money. Switch to r5 or r6i and watch the same pods pack into half the nodes.

AWS Graviton instances complicate this further, in a good way. A m7g.large offers 2 vCPU / 8 GiB at a lower price than Intel equivalents. In 2025, we migrated a set of Java services from m5 to m7g and saw 18% lower cost with identical throughput. Graviton4 generation instances have gotten mature enough that our default recommendation is now arm64 for new NodePools.

What to Watch When You Can't Watch Everything

Karpenter's own cost visibility tools are limited out of the box. The controller tracks managed nodes, but it doesn't expose spend-per-workload. We've published our Grafana dashboards on the SIVARO article about Kubernetes cost monitoring with Karpenter dashboards so teams can skip the painful part of building them from scratch.

The other thing to watch: Karpenter itself. There's a known issue about high CPU usage in Karpenter when clusters grow past a few hundred nodes. In large fleets, the controller's own resource use becomes non-trivial. We ran it on a t3.small once and had to move it to t3.medium because its CPU throttling was delaying our scaling decisions. Put Karpenter's own deployment on a dedicated node with enough headroom. It's the kind of thing you only notice at 3 AM when a scaling event stalls.

The Real Workflow: Measure, Set Requests, Then Size

Before you touch node sizes, fix your resource requests. That's the unglamorous part, and it's why so many people give up. Kubernetes overprovisioning cost reduction with karpenter only works when requests reflect reality. If your requests are 4x your actual usage, Karpenter will happily provision 4x the nodes you need.

Our standard engagement looks like this:

1. Export pod usage data using a tool like Prometheus and kube-state-metrics.
2. For each deployment, calculate p95 CPU and p99 memory over 14 days.
3. Set requests to those percentiles. Set limits to a spike threshold (usually 2x request).
4. Restart workloads and watch for Kubernetes eviction events.
5. Then enable Karpenter consolidation and narrow your instance list.

That sequence takes a week of calm engineering. It pays for itself in the first month.

But don't burn time doing this manually for every deployment. Kubernetes node scaling cost optimization is a system-level habit, not a one-time cleanup. Automate it: use Vertical Pod Autoscaler in recommendation mode for straight-up hints, or write your own scraper to generate a report. We use a custom cron job that posts a Slack summary every Monday: "17 deployments have requests > 3x p95 usage. Here's a suggested YAML patch."

FAQ

How many nodes should a Kubernetes cluster have?
It depends entirely on pod sizes and scheduling spread requirements. For most workloads, 10-30 nodes of 2-4 vCPUs is a sweet spot. Going under 5 nodes means losing high-availability spread. Going over 200 nodes means you're managing complexity that consolidation could solve.

What's the difference between Karpenter and Cluster Autoscaler for cost?
Cluster Autoscaler adds nodes from a fixed node group template. Karpenter selects instance types per batch of pending pods. Karpenter can pack more tightly and use spot intelligently. It's strictly better for cost — if you set up the NodePool requirements right.

Should I run everything on spot instances?
Run stateless workloads on spot. Keep stateful workloads on on-demand or use storage rebalancing. The discount is real, but so is the interruption. Don't learn this the hard way with a production database.

Is the instance size the most important cost factor?
No. Resource requests are. A perfectly-sized instance running pods with inflated requests is still wasting 60% of its capacity. Kubernetes node sizing for cost efficiency starts at the pod spec, not the EC2 page.

How often should I re-evaluate node sizes?
Every quarter at minimum. Workloads change. Instance families change. Graviton prices change. A sizing decision that made sense in January is stale by May.

Does Karpenter consolidation actually save money in production?
Yes, in every production deployment we've run since 2024. The savings range from 15% to 45% depending on how overprovisioned the cluster was to start. But if your resource requests are inflated, consolidation alone won't fix the bill — it just packs inflated pods onto fewer nodes.

Conclusion

Kubernetes node sizing for cost efficiency isn't a one-time project. It's a rotation: measure usage, set honest requests, constrain your instance list, let consolidation do the rest. The market