Autoscaling is often presented as a way to save money, and it can be, but only when it is configured with a clear understanding of what the workload needs. Many Kubernetes clusters run with generous resource requests, conservative replica counts and idle nodes that nobody wants to remove. A cost-aware approach treats scaling as a continuous optimisation between reliability and spend, and uses data to find the right trade-off for each workload.
Begin with right-sizing, because scaling rules built on wrong requests amplify the waste. Compare each container’s requested CPU and memory with its observed usage across a representative period, including peak hours and batch windows. Requests set far above real use reserve capacity that other workloads cannot use, which forces the cluster to add nodes unnecessarily. Recommendations based on percentiles of usage, with a safety margin, usually reclaim a large share of reserved but unused resources.
Horizontal scaling should follow a signal that tracks demand, not just CPU. For web services, requests per second or concurrent connections is often a better trigger than processor utilisation. For queue consumers, queue depth or message age is the right indicator. Choosing the signal that reflects user-facing load prevents both late scaling that hurts latency and over-eager scaling that wastes money, and it keeps behaviour predictable when the workload’s efficiency changes after a release.
Respect the time it takes to scale. New pods need to be scheduled, images pulled, applications started and caches warmed, and new nodes can take minutes to join the cluster. If demand rises faster than capacity can arrive, users will see errors regardless of how clever the rules are. Keep a modest buffer of headroom for latency-sensitive services, use predictive or scheduled scaling for known peaks and tune scale-down stabilisation windows so that the system does not shrink just before the next surge.
Node-level decisions drive most of the bill. Mix instance types and purchasing options deliberately: a stable baseline on committed or reserved capacity, variable load on on-demand nodes, and interruption-tolerant work such as batch jobs and stateless workers on discounted spot capacity. Spread workloads across zones and instance families so that the loss of a spot pool does not take down a service. Use disruption budgets and graceful termination handling so that reclaiming a node is a routine event instead of an outage.
Make costs visible at the level where decisions are made. Allocate spend to namespaces, teams and services using labels, and show each team its cost per request or per transaction next to its latency and error dashboards. Unit costs reveal efficiency changes that total spend hides, such as a release that doubled memory use without any increase in traffic. Review the top few cost drivers regularly and assign owners.
Let forecasting and anomaly detection watch over the system. Forecasts of demand support scheduled scaling and commitment purchases, while anomaly detection catches runaway scaling caused by retry storms or misbehaving clients before they inflate the bill. Keep guardrails in place, such as maximum replica counts and budget alerts, so that a faulty metric cannot scale a service to hundreds of pods overnight. The best autoscaling setups are boring: they follow demand closely, cost what they should and rarely need anyone’s attention.
Great article on Kubernetes autoscaling considerations.