Horizontal Scaling

Run multiple replicas of your service to handle higher traffic and improve availability. New replicas become ready before traffic is redistributed.

Setting replica count

Open the service → Scaling and set Replicas. New instances come up, pass health checks, then take traffic.

Replica limits by plan

Plan	Min replicas	Max replicas
Free	1	1 (no scaling)
Starter	1	3
Pro	1	10
Enterprise	1	Unlimited

Traffic distribution

Requests are spread across healthy replicas. There is no sticky session by default — any replica may handle the next request. Keep services stateless (sessions in Redis, files in object storage).

Designing for horizontal scaling

For horizontal scaling to work reliably, your service should be stateless. Here’s what “stateless” means in practice:

• Don’t store user sessions in memory. Use Redis or a database-backed session store so any replica can serve any request.
• Don’t store uploaded files on disk. Use object storage (S3-compatible) so files are accessible from all replicas.
• Don’t use in-memory caches for shared state. Each replica has its own memory. Use Redis for shared cache.
• Use database-level locking for coordination. If multiple replicas might process the same job, use advisory locks or atomic updates to prevent double-processing.

Stateless session example

server.ts

import session from 'express-session'
import { createClient } from 'redis'
import { RedisStore } from 'connect-redis'

// Sessions stored in Redis, accessible from all replicas
const client = createClient({ url: process.env.REDIS_URL })
await client.connect()

app.use(session({
  store: new RedisStore({ client }),
  secret: process.env.SESSION_SECRET!,
  resave: false,
  saveUninitialized: false,
}))

Zero-downtime scaling

When you increase replica count, Kubernetes starts new pods and waits for them to pass their readiness probe before sending traffic to them. When you decrease replica count, pods are terminated gracefully, they finish processing in-flight requests before shutting down.

This means scaling up or down never causes a request error for your users, as long as your service implements graceful shutdown correctly.

Graceful shutdown (Node.js)

server.ts

const server = app.listen(process.env.PORT || 8080)

process.on('SIGTERM', () => {
  console.log('SIGTERM received, draining connections')
  server.close(() => {
    console.log('Server closed')
    process.exit(0)
  })

  // Force exit after 30s if connections don't drain
  setTimeout(() => process.exit(0), 30_000)
})

Tip

Kubernetes sends SIGTERM when terminating a pod. Your application has 30 seconds (configurable via terminationGracePeriodSeconds) to finish in-flight requests before being force-killed. Always handle SIGTERM in your application.

Under the hood

Horizontal scaling updates the .spec.replicas field on the Kubernetes Deployment object. The ReplicaSet controller reconciles the actual pod count to match. Traffic is load-balanced by the ClusterIP Service across all Ready pods. Pods are only added to the Service’s endpoints list after passing their readiness probe.