Table of Contents
Load balancing distributes traffic across multiple servers to improve reliability and performance. Choosing the right algorithm depends on your workload characteristics.
Load Balancing Basics
A load balancer sits between clients and servers, distributing requests according to a specific algorithm. Key goals:
- Maximize throughput
- Minimize response time
- Avoid overloading any single server
- Maintain session affinity when needed
Round Robin
Simplest algorithm: cycle through servers sequentially.
class RoundRobin {
constructor(servers) {
this.servers = servers;
this.current = 0;
}
next() {
const server = this.servers[this.current];
this.current = (this.current + 1) % this.servers.length;
return server;
}
}When to Use
- Servers have similar capacity
- Requests have similar processing time
- No session state to maintain
Nginx Configuration
upstream backend {
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
}Least Connections
Route to the server with fewest active connections.
class LeastConnections {
constructor(servers) {
this.connections = new Map();
servers.forEach(s => this.connections.set(s, 0));
}
next() {
let minServer = null;
let minCount = Infinity;
for (const [server, count] of this.connections) {
if (count < minCount) {
minCount = count;
minServer = server;
}
}
return minServer;
}
recordConnection(server) {
this.connections.set(server, this.connections.get(server) + 1);
}
recordCompletion(server) {
this.connections.set(server, this.connections.get(server) - 1);
}
}When to Use
- Long-lived connections (WebSockets, streaming)
- Variable request processing time
- Persistent connections to databases
Weighted Algorithms
Assign different capacities to different servers.
upstream backend {
server backend1.example.com weight=3; # 3x capacity
server backend2.example.com weight=2;
server backend3.example.com weight=1;
}Use Cases
- Servers with different hardware specs
- Gradual traffic migration to new infrastructure
- A/B testing with controlled traffic splits
Consistent Hashing
Route requests based on a key (e.g., user ID) to maintain cache locality.
class ConsistentHash {
constructor(servers, replicas = 150) {
this.ring = new Map();
servers.forEach(server => {
for (let i = 0; i < replicas; i++) {
const hash = this.hash(`${server}:${i}`);
this.ring.set(hash, server);
}
});
this.sortedHashes = Array.from(this.ring.keys()).sort((a, b) => a - b);
}
getServer(key) {
const hash = this.hash(key);
for (const ringHash of this.sortedHashes) {
if (ringHash >= hash) {
return this.ring.get(ringHash);
}
}
return this.ring.get(this.sortedHashes[0]);
}
hash(str) {
// Simple hash function (use better one in production)
let hash = 0;
for (let i = 0; i < str.length; i++) {
hash = ((hash << 5) - hash) + str.charCodeAt(i);
hash |= 0;
}
return Math.abs(hash);
}
}When to Use
- Caching systems (CDNs, Redis clusters)
- Session affinity requirements
- Minimizing cache invalidation when scaling
Health Checks
Remove unhealthy servers from the pool automatically.
upstream backend {
server backend1.example.com max_fails=3 fail_timeout=30s;
server backend2.example.com max_fails=3 fail_timeout=30s;
# Active health check (nginx plus)
health_check interval=5s fails=3 passes=2 uri=/health;
}Health Check Endpoint
app.get('/health', async (req, res) => {
const checks = await Promise.all([
checkDatabase(),
checkRedis(),
checkDiskSpace()
]);
const healthy = checks.every(c => c.healthy);
res.status(healthy ? 200 : 503).json({ healthy, checks });
});Summary
Choose round-robin for simple, uniform workloads. Use least connections for variable request durations. Apply weighted algorithms when servers have different capacities. Implement consistent hashing for caching scenarios. Always enable health checks to automatically remove failed servers. Monitor distribution patterns and adjust based on actual traffic.