Private networking between cloud servers

It is entirely possible to build a multi-server application where every component reaches every other over public IP addresses. It works, and it is worse in three separate ways: the traffic is exposed, it is often billed, and it takes a longer path.

Private networking fixes all three, and the effort is modest. The catch is that retrofitting it onto a running system is considerably more annoying than doing it at the start.

What a private network gives you

Traffic that never leaves the provider’s network. Not routed over the public internet, not visible to anything in between.

Addresses that are not reachable from outside. A database bound only to a private address cannot be connected to from the internet at all, regardless of firewall rules. That is a structural protection rather than a configured one.

Usually free or cheaper transfer. Traffic within a private network in the same location is typically not billed, where public traffic between the same two machines might be.

Lower latency. A shorter path with fewer hops.

The security property is the most valuable. Firewall rules can be misconfigured; a service with no public address has nothing to misconfigure.

Plan the address space before you create anything

Choose a private address range and divide it into subnets with a scheme you can live with. This is the decision that is genuinely painful to change later, because changing it means re-addressing running services.

Two pieces of advice repay themselves.

Leave far more room than you need. A range with space for a handful of machines becomes a constraint surprisingly quickly, and expanding it later is disruptive.

Do not overlap with anything you might connect to. If you may one day link this network to an office, another provider, or a partner, overlapping ranges make that either impossible or dependent on address translation. Pick something specific rather than the most obvious default range, which is exactly the one everything else also chose.

The shape that works

A conventional and sensible layout separates by exposure.

A public-facing tier containing only the things that must accept internet traffic: the load balancer, perhaps a reverse proxy. These have both public and private addresses.

An application tier with private addresses only, reachable from the public tier and able to reach the data tier.

A data tier with private addresses only, reachable only from the application tier.

Outbound internet access for the private tiers, needed for package updates and third-party APIs, goes through a gateway rather than by giving each machine a public address.

This is a familiar pattern because it works: each tier is reachable only from the tier that needs it, and the blast radius of a compromise is bounded.

Reaching private machines to administer them

Once your servers have no public addresses, you need a way in. Three options, in increasing order of preference.

A bastion host: one hardened machine with a public address, from which you reach everything else. Traditional, well understood, and it is a machine you must maintain and secure.

A provider session service: many clouds offer browser or CLI access to instances without any public address or open SSH port, authenticated through the platform’s identity system and logged centrally. Where available this is better than a bastion in almost every respect.

A VPN or mesh network: your laptop joins the private network. Convenient for regular work across several environments, and it is another piece of infrastructure to run.

Whichever you choose, avoid the pattern of giving a machine a temporary public address to fix something. Temporary rarely is.

Cross-location and cross-provider connections

Connecting private networks in different regions, or across providers, is possible and each option has a distinct trade-off.

Provider peering between two networks in the same cloud is usually simple, fast, and billed for traffic that crosses regions.

Site-to-site VPN over the internet is inexpensive and its performance depends on internet conditions.

Dedicated interconnect offers consistent performance at a substantially higher cost and with a lead time to provision.

A caution on peering: it is commonly non-transitive. If A peers with B and B peers with C, A cannot generally reach C. Designs that assume otherwise fail confusingly.

Getting DNS right

Machines on a private network should reach each other by name, not by address. Hardcoded IP addresses in configuration files are how a routine instance replacement becomes an outage.

Most providers supply internal DNS automatically. Use it. Where you need your own naming, run a private zone rather than distributing hosts files, which drift immediately.

Setting it up afterwards

If you already have a running estate on public addresses, the migration is doable but deliberate: create the network, attach existing instances, switch internal traffic to private addresses one service at a time, verify each, then remove public addresses from anything that does not need one.

Do it service by service rather than all at once, and expect to find at least one hardcoded address that nobody remembered. There always is one.

Where you run this matters less than how, but if the provider is still open, the cloud account catalogue compares the options side by side.

For the vendor’s own reference on the services involved here, see the Hetzner documentation.

This is cheap to practise on, which is rather the point of Hetzner. Our Hetzner accounts come in five configurations, and the cloud account catalogue sets out the alternatives.

Questions people ask