This article was originally published at sivaro.in

Kubernetes Node Scaling Cost Optimization

You're paying for compute you don't need. I know this because we built a platform at SIVARO that processed 200K events per second, and our Kubernetes bill was a mess. The fix wasn't more nodes. It was smarter scaling.

Kubernetes node scaling cost optimization is the practice of right-sizing and dynamically scaling your worker nodes to match actual workload demand, ensuring you never pay for idle capacity. This guide covers the strategies, tools, and hard-won lessons from the field.

Let me be clear: most Kubernetes cost optimization articles are garbage. They tell you to "monitor your usage" and "set resource requests." No shit. The real work is understanding how your workloads interact with the scheduler, the autoscaler, and your cloud provider's billing model.

We tested these approaches in production. Some worked brilliantly. Some failed embarrassingly. I'll tell you which is which.

The Overprovisioning Trap

Most teams overprovision Kubernetes nodes by 40-60%. They don't mean to. It happens gradually.

A developer needs to run a new service. They set resource requests conservatively high because they're scared of OOM kills. The cluster adds a node. The service uses 5% of that node's capacity. The node stays forever.

This is how you end up with 200 nodes running at 12% utilization.

At SIVARO, we inherited a cluster that was costing €47,000 per month. Our actual workload needed maybe €18,000 of infrastructure. The rest was wasted on idle nodes, oversized requests, and a complete lack of bin-packing awareness.

The core problem: default Kubernetes behavior assumes you'll manage capacity manually. The Horizontal Pod Autoscaler handles pod counts, but node scaling requires something else entirely.

What most people get wrong: They think the Kubernetes Cluster Autoscaler is the solution. It's not. It's a band-aid.

Why Cluster Autoscaler Falls Short

The Kubernetes Cluster Autoscaler (CA) has been the default node scaling solution for years. It scales nodes up when pods are pending, and it scales down when nodes are underutilized.

The issues are structural.

First, consolidation is terrible. CA only removes nodes that are completely empty or have utilization below a hard threshold (typically 50%). It won't consolidate workloads across nodes to reduce the cluster size. You can have 6 nodes each running at 45% utilization, and CA will do nothing.

Second, scheduling latency. CA takes 60-90 seconds to provision new nodes. For batch workloads or spikes, that's unacceptable.

Third, it doesn't respect spot instances well (more on that later).

The right approach for cost optimization is Karpenter, an open-source node autoscaler that was originally built for AWS and is now a CNCF project. Karpenter fundamentally changes the node provisioning model.

Karpenter: The Game Changer

Karpenter isn't just a better Cluster Autoscaler. It's a different philosophy.

Instead of managing node groups, Karpenter manages individual nodes. It watches pods that are pending and provisions the cheapest node that can satisfy their requirements. No node groups. No manual instance type selection.

Here's the key difference: CA operates on node groups—you tell it the instance types, and it scales within that group. Karpenter can dynamically select instance types based on the pod's requirements. CPU-heavy pod? It'll pick a C-series instance. Memory-heavy? R-series. Small batch job? Maybe a t3.small.

This doesn't just reduce cost—it redefines how you think about capacity.

Our experience: After migrating to Karpenter, our cluster went from 47 nodes to 31 nodes while running the same workload. Monthly compute costs dropped by 43%. The provisioning time for new nodes went from 90 seconds to under 10 seconds.

The AWS blog on Karpenter consolidation outlines the core consolidation behavior: Karpenter continuously looks for ways to consolidate workloads onto fewer nodes, and it uses a bin-packing algorithm to do so. This is the single biggest cost lever you can pull.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h

This NodePool configuration allows Karpenter to use spot instances and automatically consolidate nodes when they're underutilized. The expireAfter: 720h setting forces node rotation every 30 days, which prevents configuration drift.

Node Sizing: Right-Sizing Your Instances

Kubernetes node sizing for cost efficiency isn't just about picking the right instance type. It's about understanding your workload's actual resource consumption patterns.

Let me give you a concrete example. In 2025, we had a customer with a real-time analytics workload that was memory-hungry but CPU-light. They were running 5 c5.4xlarge instances (16 vCPU, 32 GiB RAM each).

These nodes were running at 8% CPU utilization but 91% memory utilization. They were paying for compute they weren't using.

We switched them to r5.xlarge instances (4 vCPU, 32 GiB RAM). Same memory capacity, quarter of the CPU. The bill dropped by 55%. Workload performance was unaffected—it turned out the CPU headroom was never needed.

Here's the mental model I use:

1. Profile your workloads by resource ratio. Measure the CPU-to-memory ratio of actual usage, not requests
2. Match instance types to that ratio. Compute-heavy workloads go on C-series, memory-heavy on R-series, balanced on M-series
3. Use Karpenter's bin-packing to mix instance types based on your actual pod mix

The old model of homogenizing your node groups is dead. Every node pool should have a mix of instance types so Karpenter can choose the cheapest one for each pod.

The "Capacity is Free" Mindset

This is the contrarian take.

Most cost optimization advice focuses on reducing node count and instance size. But the real cost killer is scheduling efficiency.

Here's the thing: if you have 10 pods that each request 1 GiB of memory, and you have 11 GiB of total capacity, the scheduler puts all 10 pods on the node with the most availability. Sometimes it ends up distributing them across multiple nodes, are terrible at utilizing your node capacity.

To fix this, we built a bin-packing strategy. We run a smaller number of larger nodes and let the scheduler pack as many pods onto each node as possible.

Wait, that's not right either. If you have a node that fails, you lose more workloads. It's a trade-off between capacity utilization and failure blast radius.

What worked for us: Use on-demand instances for your non-negotiable, always-running workloads. Use spot instances for everything else. The spot price is typically 60-90% cheaper than on-demand, and for stateless workloads, the risk is manageable.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-pool
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
            - "m5.large"
            - "m5.xlarge"
            - "m5.2xlarge"
  disruption:
    consolidationPolicy: WhenUnderutilized

This NodePool limits Karpenter to spot instances only, with a small selection of instance types. The consolidated policy ensures that if a spot instance is reclaimed, Karpenter will provision a replacement immediately.

Don't Be Afraid of Spot Instances

This is the part where most people get nervous. "Spot instances will interrupt my workloads!" they say.

That was true in 2018. It's mostly solved in 2026.

AWS reclaims spot instances when the spot price exceeds your bid, or when capacity is needed. The interruption rate has dropped significantly over the years, and with the right architecture, interruptions are a blip.

The key insight: Most Kubernetes workloads are stateless just by nature. Even stateful ones can survive interruptions if you're using proper persistence (EBS volumes, EFS, etc.).

We run roughly 70% of our production workloads on spot instances. Our interruption rate is under 1% per month. And when interruptions do happen, Karpenter replaces the node within minutes. The blip is measured in seconds of pod rescheduling time.

The math is simple: spot instances cost about 70% less than on-demand. Even if 5% of your spot workloads get interrupted, you're saving 65% on those workloads. It's a dumb trade if you're paying attention.

But there's a trap: Karpenter's consolidation behavior can sometimes remove spot nodes that are in use. You need to set the right consolidation policies—for workloads that can't tolerate any interruption, use the disruption.consolidateAfter setting instead of WhenUnderutilized.

The Cost of Overprovisioning: Real Numbers

Let me show you why this matters. Here's a real example from a financial services company in 2025.

Setup: 3 worker nodes, each m5.2xlarge (8 vCPU, 32 GiB), running 40 pods across 6 services.

Observed usage: Each node was running at 22% CPU utilization, 34% memory utilization. That's an average of 1.76 vCPU and 10.88 GiB per node.

Monthly cost: 3 nodes × $416 = $1,248/month.

What they should have run: 1 node. Maybe 2 for HA. The workload needed 5.3 vCPU and 32.6 GiB total — that fits on a single m5.2xlarge with a small buffer.

The fix: We moved them to a Karpenter-backed cluster with a mix of m5.large and m5.xlarge spot instances. The cost dropped to $410/month. Same workload. Same performance. 67% reduction.

This is what kubernetes overprovisioning cost reduction with karpenter looks like in practice. It's not a magic trick. It's just removing the waste.

Horizontal Pod Autoscaler (HPA): How It Interacts with Node Scaling

HPA scales pod replicas based on CPU/memory or custom metrics. But it doesn't know anything about node capacity.

Here's the flow:

1. Workload demand increases
2. HPA adds pod replicas
3. Pods get scheduled to existing nodes
4. If nodes can't fit the pods, Karpenter provisions new nodes
5. Workload demand decreases
6. HPA removes replicas
7. Karpenter consolidates nodes

The problem is step 2 to step 3. HPA might scale pods aggressively, but if the cluster lacks capacity, you get a "pending pods" situation. Karpenter handles this by provisioning new nodes quickly, but HPA can outpace Karpenter if it's scaling too fast.

What we learned: Set your HPA to scale slowly. The default metrics behavior is purely reactive, and it can cause flapping. Use the stabilization window in the HPA spec to prevent rapid scaling.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-service
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-service
  minReplicas: 3
  maxReplicas: 20
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

This HPA configuration uses a 5-minute stabilization window for scale-down, which prevents the HPA from removing replicas that might be needed soon. The 10% max scale-down rate per minute prevents dramatic capacity drops.

The Cost-Game with Spot Instances

Let's get into the weeds a bit. AWS spot pricing is dynamic. It can fluctuate based on supply and demand. At this GitHub issue filed in 2023, users reported unexpected spot capacity issues, which is actually a common concern with Karpenter's consolidation.

But here's the thing: the spot market has matured. In 2026, AWS has much better capacity management than it did in 2022. The spot interruption rate for modern instance types is consistently below 2% per month in most regions.

What we do differently: We designate certain workloads as "spot-optional." These are workloads that can tolerate interruption (batch jobs, workers, etc.). When spot capacity is plentiful, they run on spot. When spot capacity is tight, they fail over to on-demand.

Karpenter handles this automatically if you configure two different NodePools: one for spot, one for on-demand. The scheduler will prefer the spot pool, but if capacity is unavailable, pods will land in the on-demand pool.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: on-demand-fallback
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
  disruption:
    consolidationPolicy: WhenUnderutilized

This fallback NodePool ensures that critical workloads always have capacity, even if the spot market tightens. The catch: it's more expensive. That's the trade-off you accept for resilience.

Understanding Your Cluster's Workload

The single biggest lever for cost optimization is knowing what's actually running in your cluster.

A lot of companies waste money on workloads that don't belong in Kubernetes. We once saw a customer running a MySQL database on Kubernetes with no persistence, which is a classic mistake. They were losing data daily. They thought they'd solved it by increasing node sizes.

The database needed 60 GiB of storage. A single m5.2xlarge with an EBS volume of 200 GiB would have cost $400/month. Instead, they were running 4 nodes, each with 20 GiB of EBS, plus a MySQL container that couldn't access persistent storage. The cluster was a mess.

We moved them to Amazon RDS for MySQL. Their Kubernetes cluster dropped from 4 nodes to 2. Their compute cost dropped by half, and RDS cost $150/month.

Lesson: Not everything belongs in Kubernetes. If your workload needs persistent storage, consider a managed database service.

Monitoring: SIVARO's Approach

You can't optimize what you can't see. This is where the Kubernetes Cost Monitoring Karpenter Dashboards - Sivaro article comes in. We built dashboards that show you exactly where your money is going.

The dashboards show:

• Cost per namespace — which teams are spending the most
• Cost per workload — which deployments are the most expensive
• Resource utilization — how much of each node is actually being used
• Spot vs on-demand cost — the savings generated from spot instances
• Consolidation efficiency — Karpenter's consolidation score, showing how many nodes could be eliminated

We use these dashboards for our customers. The typical discovery: 30% of workloads don't need Kubernetes. Another 30% are oversized request-wise. The remaining 40% is the actual workload.

The Real Price of Overprovisioning

Here's the thing people often miss: overprovisioning isn't just about wasted spend. It also affects:

• Team productivity — managing unnecessary nodes adds operational complexity
• Security — larger clusters have a bigger attack surface
• Failure domains — more nodes = more failure points

When we migrated that financial services customer above, their cluster became more stable, not less. Fewer nodes meant simpler networking, fewer problems with kernel upgrades, and easier security patching.

Here's our full guide on Kubernetes Cost Monitoring Karpenter Dashboards, which includes the exact Grafana dashboard JSON files we use. They're free to use.

Node Scaling Cost Optimization: The Playbook

After years of doing this, here's the playbook I'd give you.

Step 1: Audit Your Cluster

Run kubectl top nodes and kubectl top pods to see actual usage. Compare it to requests and limits. The gap is where your money goes.

Step 2: Set Resource Requests and Limits Correctly

This is the most impactful single change you can make. We've seen teams reduce their cluster size by 40% overnight by fixing their requests. The schema: requests should reflect your workload's normal usage. Limits should be the maximum you'd ever want that pod to use.

resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    cpu: 1000m
    memory: 1Gi

This says: "I expect to use 250m CPU and 512Mi memory, but I'm okay with bursts up to 1000m and 1Gi." Requests are what the scheduler uses for bin-packing.

Step 3: Install Karpenter

If you're on open-source, Karpenter is now the standard. It's a CNCF project, and it's production-ready. If you're on AWS EKS, it's the recommended node autoscaler in the EKS console.

We've found that Karpenter works best with:

• A mix of spot and on-demand in your NodePools
• Consolidation policy set to WhenUnderutilized
• Reasonable consolidateAfter values for workloads that can't be interrupted

Step 4: Set Up Cost Monitoring

Use the SIVARO dashboards or equivalent. Your goal: know the cost per workload in real-time.

Step 5: Review Weekly

Don't treat cost optimization as a one-time activity. We review our cluster costs every Monday. We look for:

• New workloads with huge resource requests
• Deployments with 0 replicas
• Idle nodes
• Spot niche opportunities

If your cluster is static, your cost optimization is dead.

The Takeaway

Kubernetes node scaling cost optimization is the difference between paying for compute you never use and paying for what you actually need. Most teams don't know they're overpaying. The tools exist to fix it.

But here's the closing thought: cost optimization is not the end goal. It's the result of doing the right things—right-sizing your workloads, using spot intelligently, consolidating nodes, and monitoring continuously. Do these, and your bill shrinks.

Do it wrong, and you end up with a cluster that's 30% utilized, a team that's burnt out managing it, and a CFO wondering why your cloud bill is draining the budget.

FAQ

What is Kubernetes node scaling cost optimization?

It's the practice of dynamically scaling your node pool to match actual workload demand, using tools like Karpenter to consolidate workloads onto fewer nodes, and strategically using spot instances to reduce compute costs.

How does Karpenter reduce costs?

Karpenter automatically selects the cheapest instance types that match pod requirements, consolidates workloads from underutilized nodes, and supports spot instances out of the box.

What is consolidation in Karpenter?

Karpenter's consolidation policy discovers when workloads could fit on fewer nodes, replaces the nodes with cheaper alternatives, and terminates the surplus nodes.

Is it safe to use spot instances in production?

In 2026, yes. With proper workload architecture (stateless, rescheduling-tolerant), spot interruptions are rare and manageable. Karpenter supports defining a "disruption budget" to limit disruption impact.

What does the "disruption budget" do?

Karpenter's disruption budget pauses consolidation on nodes within a time window. For example:

disruption:
  budgets:
    - nodes: 10%
      schedule: "@daily"

This prevents the cluster from being disrupted during critical business hours.

What's the difference between Karpenter and Cluster Autoscaler?

Karpenter uses in-place scheduling and dynamic provisioning at the pod level. Cluster Autoscaler operates at the node-group level and is limited by pre-defined node groups.

Do I still need Horizontal Pod Autoscaler?

Yes. HPA scales pod replicas; Karpenter scales nodes to fit those replicas. They are complementary.

What's the best way to set resource requests?

Set requests to your workload's normal usage. Add a 20-30% safety margin. Don't set them to your peak usage—that's what limits are for.

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.