Black Friday: HPA goes from four to twenty API pods — green chart, calm team. Yet checkout times out: PostgreSQL hits max_connections, Redis single-thread saturated. New pods multiply connections without increasing transactional capacity. Kubernetes autoscaling "success," business failure.
Scaling pods fixes stateless bottlenecks. It often worsens stateful or shared bottlenecks if nobody mapped the full chain. Before automating anything, identify which resource breaks first under spike — database, broker, third-party API, egress bandwidth — and who is allowed to size it.
HPA: default metric and blind spots
Horizontal Pod Autoscaler compares a metric (CPU, memory, custom) to a target and changes replica count. On paper, that is simple. In production, CPU is only an imperfect proxy for real load.
| Metric | Good signal when | Bad signal when |
|---|---|---|
| CPU % | CPU-bound work (render, crypto) | Disk or database wait |
| Memory | In-pod cache | Leak masked by scale |
| RPS / latency (custom) | Stateless API | Upstream saturation |
| Queue depth (KEDA) | Async workers | Queue without healthy consumers |
Without metrics-server and a Prometheus adapter (or equivalent), HPA stays blind to business load. A pod can show twenty percent CPU while waiting eight seconds on a blocked PostgreSQL response. The metric rises too late — or not at all.
Adding pods to a database at one hundred percent connection usage only creates more waiting clients.
To escape default CPU, expose signals that reflect user experience: 95th-percentile latency on /api/checkout, Redis queue length, RabbitMQ message lag. HPA becomes useful only when the chosen metric correlates with timeouts under load.
VPA and rightsizing before multiplying
Vertical Pod Autoscaler recommends or applies CPU and memory requests and limits. Too-low requests cause CPU throttling and push HPA to scale pointlessly. Too-high requests fill nodes: new pods stay Pending, cluster autoscaler kicks in urgently, and the bill follows.
Pragmatic workflow: stabilize requests first (VPA in "Off" mode to collect recommendations), then enable HPA on replicas. On managed European clusters, watch cost per node: cluster autoscaler without budget alerts turns a traffic spike into a month-end surprise.
Cluster autoscaler and Pending pods
When HPA creates pods with no room on existing nodes, they stay Pending. Cluster autoscaler provisions a node — with minutes of delay (image pull, boot, registration). During that gap, traffic keeps hitting already saturated pods.
Plan ahead with a PodDisruptionBudget so node scale-down does not drain a critical deployment. Use multiple node pools (general vs compute-heavy) for heterogeneous workloads. Set a max node cap to avoid runaway billing from a misconfigured scale loop.
KEDA for workers and queues
Symfony Messenger workers, Laravel queues, Kafka consumers: the right signal is lag or queue depth, not fifteen percent CPU while thousands of messages wait. KEDA can scale to zero — useful for batch workloads, but watch cold start vs real-time SLA commitments.
Document exchange and queue topology before automating. A scaler wired to the wrong queue scales workers while the critical queue grows elsewhere.
Stateful services: do not autoscaler blindly
PostgreSQL, Elasticsearch, some Redis deployments: horizontal scale is non-trivial. Sometimes the answer is a bigger node (VM resize), a read replica, or upstream cache — not more API pods multiplying connections.
Map the chain: for +1000 RPS, which resource breaks first? Measure under load test before automating the wrong lever. Kubernetes autoscaling does not replace thoughtful data architecture.
The peak: the reassuring chart
The right question is not "how many pods?" but "which resource fails first under spike — and who scales it?"
Decide and move forward without blind spots
Start with a load test and per-layer metrics: application, database, broker, network. Pick the right scaler (HPA, KEDA, cluster autoscaler) on the signal that correlates with user timeouts, not the easiest metric to chart. Set max replicas, disruption budgets, and alerts on 95th-percentile latency.
Compare managed clusters and node pools via the directory and compare tool. Guides round out cloud architecture framing. Validate under load: if 95th-percentile latency stays stable as replicas rise, the bottleneck is elsewhere — and that is where to invest.
Frequently asked questions
Is HPA on CPU enough in production?
Rarely alone. Stable CPU can hide a growing Redis queue or a database with exhausted connections. Combine application metrics (queue lag, RPS, 95th-percentile latency) via Prometheus adapter or KEDA to scale on what correlates with user timeouts.
What is the difference between HPA, VPA, and cluster autoscaler?
HPA adjusts pod replica count against a target metric. VPA recommends or applies CPU and memory requests and limits. Cluster autoscaler adds nodes when pods stay Pending for lack of cluster capacity.
What does KEDA add over classic HPA?
KEDA provides scalers based on queue depth, cron triggers, and external metrics. For async workers, these signals are often more meaningful than low average CPU while thousands of messages wait.
How do I avoid flapping?
Configure stabilizationWindowSeconds, hysteresis thresholds, cooldown between scale events, and a realistic minimum replica count. Without guardrails, HPA oscillates and destabilizes persistent connections.
Mature autoscaling scales what makes users fail — not what is already in Grafana.