Configure Kubernetes autoscaling from queue depth instead of CPU

TL;DR — CPU-based autoscaling is the wrong signal for queue workers because idle-but-blocked consumers can have a huge backlog with low CPU. The fastest reliable setup is KEDA with your queue's native scaler or Prometheus-exposed queue-depth metric; set a backlog threshold per replica, apply a ScaledObject, and verify kubectl get hpa shows scaling from an external metric instead of CPU. Reading time: ~5 min

Goal

When you are done, your worker deployment scales out and in based on queue depth (for example, RabbitMQ messages ready, SQS visible messages, or a Prometheus backlog metric), and kubectl get hpa / kubectl describe scaledobject shows scaling decisions tied to backlog rather than CPU utilization.

Prerequisites

• Kubernetes cluster access with permission to create namespaces, deployments, HPAs, and CRDs: check with kubectl auth can-i create deployment -A
• kubectl installed and pointed at the right cluster: kubectl version --client
• Helm 3.x if you will install KEDA with Helm: helm version
• A running worker deployment you want to scale, with a stable label selector
• A queue system and credentials already created: RabbitMQ, AWS SQS, Azure Service Bus, Redis streams/list, Kafka lag via exporter, or any queue depth metric available in Prometheus
• If using Prometheus metrics: a working Prometheus endpoint reachable from the cluster, and the exact metric name/labels for queue depth
• If using cloud queues: the queue URL/name and the auth method your cluster uses (secret, IAM role, workload identity, etc.)

Steps

Step 1: Confirm your current worker deployment name and replica behavior

Run:

kubectl get deploy -A | grep worker
kubectl get deploy -n jobs worker -o wide
kubectl get pods -n jobs -l app=worker

You should see the deployment name, namespace, and current replica count you intend to autoscale.

Step 2: Remove CPU-based autoscaling if one already exists for this deployment

If an HPA already targets the worker deployment, list it and delete the CPU-based one before adding queue-depth scaling:

kubectl get hpa -A
kubectl delete hpa -n jobs worker

You should see horizontalpodautoscaler.autoscaling "worker" deleted or no matching HPA if none existed.

⚠️ Deleting an existing HPA temporarily stops autoscaling for that deployment until the new scaler is applied. Do this during a low-risk window if your backlog is sensitive.

Step 3: Install KEDA in the cluster

Install KEDA into its own namespace:

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm upgrade --install keda kedacore/keda --namespace keda --create-namespace
kubectl rollout status deploy/keda-operator -n keda --timeout=120s

You should see Helm report STATUS: deployed and rollout status end with successfully rolled out.

Step 4: Create queue credentials as a secret or use your cluster identity

For RabbitMQ, create a secret with the AMQP connection string:

kubectl create namespace jobs --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic rabbitmq-auth -n jobs \
  --from-literal=host='amqp://workeruser:workerpass@rabbitmq.default.svc.cluster.local:5672/'

For AWS SQS with static credentials, create:

kubectl create secret generic sqs-auth -n jobs \
  --from-literal=AWS_ACCESS_KEY_ID='AKIAEXAMPLE' \
  --from-literal=AWS_SECRET_ACCESS_KEY='secretExample'

You should see secret/<name> created or configured.

Step 5: Apply a KEDA TriggerAuthentication

For RabbitMQ:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: rabbitmq-auth
  namespace: jobs
spec:
  secretTargetRef:
    - parameter: host
      name: rabbitmq-auth
      key: host

Apply it:

kubectl apply -f triggerauth-rabbitmq.yaml

For SQS:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: sqs-auth
  namespace: jobs
spec:
  secretTargetRef:
    - parameter: awsAccessKeyID
      name: sqs-auth
      key: AWS_ACCESS_KEY_ID
    - parameter: awsSecretAccessKey
      name: sqs-auth
      key: AWS_SECRET_ACCESS_KEY

Apply it:

kubectl apply -f triggerauth-sqs.yaml

You should see triggerauthentication.keda.sh/<name> created.

Step 6: Apply a ScaledObject that targets queue depth

Use one of the following exact examples.

RabbitMQ queue depth example (QueueLength: 25 means scale roughly one replica per 25 queued messages):

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-queue-scaler
  namespace: jobs
spec:
  scaleTargetRef:
    name: worker
  pollingInterval: 15
  cooldownPeriod: 120
  minReplicaCount: 1
  maxReplicaCount: 20
  triggers:
    - type: rabbitmq
      metadata:
        protocol: amqp
        queueName: jobs
        mode: QueueLength
        value: "25"
      authenticationRef:
        name: rabbitmq-auth

SQS visible messages example (queueLength: 50):

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-queue-scaler
  namespace: jobs
spec:
  scaleTargetRef:
    name: worker
  pollingInterval: 30
  cooldownPeriod: 180
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/123456789012/jobs
        awsRegion: us-east-1
        queueLength: "50"
      authenticationRef:
        name: sqs-auth

Prometheus backlog metric example (sum(queue_depth{queue="jobs"})):

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-queue-scaler
  namespace: jobs
spec:
  scaleTargetRef:
    name: worker
  pollingInterval: 15
  cooldownPeriod: 120
  minReplicaCount: 1
  maxReplicaCount: 20
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus-server.monitoring.svc.cluster.local:9090
        metricName: jobs_queue_depth
        query: sum(queue_depth{queue="jobs"})
        threshold: "25"

Apply one file:

kubectl apply -f scaledobject-worker.yaml

You should see scaledobject.keda.sh/worker-queue-scaler created.

Step 7: Check that KEDA created the HPA and can read the metric

Run:

kubectl get scaledobject -n jobs
kubectl describe scaledobject worker-queue-scaler -n jobs
kubectl get hpa -n jobs

You should see the READY condition as True on the ScaledObject, and an HPA with a generated name similar to keda-hpa-worker-queue-scaler.

Step 8: Push backlog into the queue and watch replicas change

Create enough messages to exceed your threshold. Use the producer you already have, or if RabbitMQ management API is enabled:

for i in $(seq 1 200); do curl -su workeruser:workerpass -H 'content-type: application/json' -X POST http://rabbitmq.default.svc.cluster.local:15672/api/exchanges/%2F/amq.default/publish -d '{"properties":{},"routing_key":"jobs","payload":"test-'$i'","payload_encoding":"string"}'; echo; done

Then watch scaling:

kubectl get deploy worker -n jobs -w

You should see DESIRED and AVAILABLE replicas increase within one or two polling intervals.

Verify it works

Check the scaler, HPA, and deployment together:

kubectl describe scaledobject worker-queue-scaler -n jobs
kubectl get hpa -n jobs
kubectl get deploy worker -n jobs

Expected shape:

Name:         worker-queue-scaler
Namespace:    jobs
...
Status:
  Conditions:
    Type    Status  Reason    Message
    Ready   True    ScaledObjectReady   ScaledObject is defined correctly and is ready for scaling
    Active  True    ScalerActive        Scaling is performed because triggers are active

NAME                         REFERENCE           TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-worker-queue-scaler Deployment/worker  160/25      1         20        7          3m

NAME     READY   UP-TO-DATE   AVAILABLE   AGE
worker   7/7     7            7           12d

Then drain the backlog and confirm scale-in after cooldownPeriod:

kubectl get deploy worker -n jobs -w

Expected result: replicas decrease back toward minReplicaCount or 0 if you set scale-to-zero.

Common pitfalls

Wrong queue name or URL

Mistake: queueName, queueURL, or Prometheus label value does not match the real queue. Symptom: kubectl describe scaledobject shows inactive scaling or auth succeeds but metric stays 0; backlog exists in the broker UI. Fix: replace the exact queue identifier in the ScaledObject, then kubectl apply -f scaledobject-worker.yaml.

Auth secret key names do not match TriggerAuthentication parameters

Mistake: secret contains username/password, but TriggerAuthentication expects host, or SQS keys are misnamed. Symptom: kubectl describe scaledobject shows errors like error parsing rabbitmq metadata or missing awsAccessKeyID. Fix: recreate the secret with the exact keys referenced in secretTargetRef.

Threshold set too low, causing replica thrash

Mistake: value: "1" or queueLength: "1" on a bursty workload. Symptom: replicas jump rapidly up and down; worker startup cost exceeds useful work. Fix: set threshold to backlog per pod your worker can clear during one polling window, and increase cooldownPeriod to 120-300 seconds.

Scaling on queued messages but ignoring in-flight work

Mistake: using only visible/ready message count when each job runs for a long time. Symptom: HPA scales down while many workers are still busy, or scales too slowly for long-running jobs. Fix: use a threshold that accounts for average processing time, or expose a custom Prometheus metric that includes backlog plus in-flight reservations if your queue supports it.

Existing HPA or GitOps controller keeps overwriting the generated HPA

Mistake: a manually managed HPA or reconciliation tool reapplies CPU-based autoscaling. Symptom: kubectl get hpa alternates between definitions, or KEDA logs reconciliation conflicts. Fix: remove the old HPA manifest from Git and keep only the ScaledObject as the source of truth.

Scale-to-zero breaks consumers that require warm connections

Mistake: minReplicaCount: 0 for workers with heavy startup, JVM warmup, or expensive broker handshakes. Symptom: backlog spikes, then clears slowly because first pod startup takes too long. Fix: set minReplicaCount: 1 and keep one warm worker running.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.