Load balancers, explained without the diagram

The name suggests the job is dividing traffic between servers. That is the least interesting thing a load balancer does. Its more valuable functions are noticing when a server is unhealthy, allowing you to remove one without interrupting anyone, and giving you a single stable address in front of infrastructure that changes underneath.

Layer 4 and layer 7

The main distinction, and it determines what is possible.

Layer 4 balancers work at the transport layer. They forward connections based on IP address and port without inspecting the contents. Very fast, protocol-agnostic, and able to handle anything built on TCP or UDP. They cannot make decisions based on the request, because they do not read it.

Layer 7 balancers understand the application protocol, normally HTTP. They can route by path or hostname, modify headers, terminate TLS, and make decisions based on anything in the request. Slightly more overhead, considerably more capability.

For web applications, layer 7 is almost always what you want. For databases, message brokers or custom protocols, layer 4 is the option that works.

Health checks are the important part

A load balancer that spreads traffic evenly across servers including the broken one has made things worse rather than better. Health checks are what make it useful.

Three properties matter and are commonly misconfigured.

The check should exercise the dependencies that matter. A check returning a static string proves the process is running. A check that verifies the database connection proves the instance can actually serve requests. But a check that fails whenever a non-essential dependency is slow will remove healthy instances during a partial outage, turning a degradation into an outage.

The usual resolution is two endpoints: a liveness check that only proves the process is alive, and a readiness check that verifies the dependencies required to serve traffic.

The thresholds should not be twitchy. Removing an instance after a single failed check causes flapping during a transient blip. Two or three consecutive failures before removal, and two or three successes before returning, is a reasonable default.

The check must be cheap. It runs constantly from every balancer node. A health check performing a heavy query is a self-inflicted load problem.

Connection draining is what makes deployments invisible

When you remove an instance, connections already in progress need somewhere to go. Without draining, they are cut mid-request and users see errors during every deployment.

Draining stops sending new connections to an instance while allowing existing ones to complete, up to a timeout. Set the timeout slightly longer than your longest normal request.

This is the mechanism that makes rolling deployments genuinely seamless, and it is frequently left at a default that is shorter than the application’s real request duration.

Session handling, and why to avoid needing it

Sticky sessions route a given client to the same instance every time. It solves the problem of session state held in an instance’s memory, and it creates several others: uneven distribution, sessions lost when an instance is replaced, and difficulty draining.

The better answer is nearly always to make the application stateless by keeping session state in a shared store or in a signed token. Then any instance can serve any request, and the load balancer’s job becomes simple.

Reach for stickiness when you cannot change the application, not as a design choice.

TLS termination

Terminating TLS at the load balancer centralises certificate management, removes the cryptographic work from your application servers, and gives you one place to configure protocol versions and cipher suites.

The traffic behind it then travels unencrypted unless you re-encrypt. Whether that matters depends on your threat model and your compliance obligations; within a private network it is often acceptable, and in regulated environments it frequently is not. Re-encrypting to the backend is supported by most layer 7 balancers if you need it.

What it does not solve

A single balancer is a single point of failure, unless the managed service is itself redundant, which most cloud load balancers are. Check rather than assume.

It does not fix a slow application. Distributing slow requests across more servers increases capacity, not speed.

It does not fix a shared bottleneck. If every instance queries the same database and the database is saturated, adding instances makes the database worse.

It does not remove the need for capacity planning. Autoscaling behind a balancer still needs sensible bounds, and scaling has a delay that a sudden spike can outrun.

A short configuration checklist

• Health check path exercises the dependencies needed to serve, and is cheap.
• Separate liveness and readiness checks where the platform supports them.
• Two or three consecutive failures before removal, not one.
• Connection draining enabled, with a timeout longer than your longest normal request.
• TLS terminated with a current protocol configuration and automated certificate renewal.
• Access logging enabled and going somewhere you can query.
• Application is stateless, so stickiness is unnecessary.

For a view of how the providers differ on the points above, the cloud account catalogue lays out their respective strengths and configurations.

For the vendor’s own reference on the services involved here, see the DigitalOcean documentation.

None of this takes long to try once you have somewhere to run it. Our DigitalOcean accounts come in six configurations from $20, and the cloud account catalogue shows the nine other providers we carry.

Questions people ask