How Load Balancers Distribute Traffic

The Problem: One Server Isn't Enough

A popular website receiving far more traffic than a single server can handle needs a way to spread that traffic across multiple servers. A load balancer sits in front of a group of servers and decides which one should handle each incoming request.

                     [ Load Balancer ]
                    /       |        \
            Server A    Server B    Server C
           (each handles a share of the total traffic)

Health Checks: Avoiding a Failed Server

A load balancer continuously checks whether each backend server is actually healthy and responding correctly. If a server fails a health check, the load balancer automatically stops sending it new traffic, redirecting requests to the remaining healthy servers instead - without any human needing to intervene.

Why This Directly Improves Reliability

Without a load balancer, a single failed server means the entire service goes down for every user. With one in place, a single server failure just means slightly reduced total capacity - the service as a whole stays up, since traffic simply routes around the failed machine.

Why This Also Enables Easy Scaling

Adding more capacity becomes as simple as adding another server behind the load balancer - it automatically starts receiving a share of new traffic once it passes its health checks, without requiring any change to how users actually access the service.

Where Load Balancers Typically Operate

Load balancers can operate at different layers - some make decisions purely based on IP address and port (Layer 4), while others can inspect actual application-level content like URLs or headers to make smarter routing decisions (Layer 7), similar in spirit to the layered awareness difference covered for firewalls earlier in this section.