Sizing a server for a small production workload

Server sizing is usually done once, in a hurry, before the application has ever seen real traffic, and then never revisited. The result is predictable: a fleet of machines running at fifteen percent utilisation, and a monthly bill two or three times what the workload requires.

The fix is not to guess better. It is to guess small, measure, and adjust, which works because vertical resizing on any modern VPS provider takes minutes.

Start from the constraint, not the traffic

“How much traffic will we get” is the wrong opening question, because it does not translate into a machine size without a great deal of information you do not have yet.

The useful question is: what will run out first? For most small applications the answer is memory, not CPU. A web application with a runtime, a handful of worker processes and a database on the same box will exhaust memory long before it saturates a modern core.

That gives a practical starting rule: size for memory, then check whether CPU is adequate, rather than the reverse.

A rough memory budget

Work it out additively rather than picking a round number.

• The operating system wants a few hundred megabytes for itself once it settles.
• Each application worker process wants whatever your runtime uses, multiplied by the number of workers. Measure one worker under load rather than guessing; the difference between runtimes is large.
• The database, if co-located, wants both its own working memory and enough page cache to keep the hot part of the dataset in memory. This is usually the largest single item and the one most often underestimated.
• A reverse proxy wants very little.
• Headroom of at least twenty five percent, because a machine that swaps under load performs far worse than one that is merely busy.

Add those up and choose the plan above the total. If the sum is uncomfortably close to a plan boundary, take the larger plan; the price difference is small relative to the cost of an outage.

Then check CPU honestly

Once memory is provisioned, CPU is usually adequate. The exceptions are real but specific: image or video processing, cryptographic work, compilation, anything doing heavy serialisation, and applications that do computation the database should be doing.

The signal to watch is not CPU utilisation alone. It is utilisation together with request latency. A server at seventy percent CPU with flat latency is fine. A server at forty percent CPU with latency climbing has a different problem, usually blocking I/O or lock contention, and a bigger machine will not fix it.

On shared-CPU plans, also watch steal time. Sustained steal time means the hypervisor is giving your cores to someone else, and that is the moment to consider a dedicated-CPU plan rather than a larger shared one.

Measure under something like real load

Sizing from an idle machine tells you nothing. You need load, and it does not need to be sophisticated.

Run a load generator against a realistic endpoint mix, ramping until latency degrades. Note the request rate at which the ninety-fifth percentile latency crosses whatever you consider acceptable. That number, not the maximum the server can technically absorb, is your capacity.

Watch memory, CPU, steal time and disk I/O throughout. Whichever saturates first is your constraint, and it is the only dimension worth spending money on.

Leave room, but not that much

The temptation after a load test is to provision for the peak you measured plus a large multiple. Resist it.

A reasonable target is to run at somewhere between forty and sixty percent of capacity at normal peak. That leaves genuine headroom for a traffic spike or a slow dependency, without paying for a machine that is idle most of the time.

If your traffic pattern has a large daily swing, the answer is usually not a permanently larger machine but either an autoscaling arrangement or simply accepting slightly higher latency at peak.

Revisit it, briefly, on a schedule

Put a recurring twenty-minute task in the calendar every quarter: look at the utilisation graphs for the previous three months, and ask whether the machine is the right size.

Applications get more efficient as well as less. Caching gets added, queries get fixed, dependencies get upgraded. It is entirely normal for a machine sized correctly a year ago to be twice as large as it needs to be now, and nobody notices because nothing is broken.

When to stop scaling up

Vertical scaling is the right first answer far more often than architecture discussions suggest, but it has two natural ends.

The first is redundancy. A single machine, however large, is a single point of failure. At the point where downtime becomes genuinely costly, two smaller machines behind a load balancer are worth more than one large one, at roughly the same price.

The second is when the constraint stops being the machine. If the database is the bottleneck, a larger application server changes nothing. Identify what actually saturates before spending, every time.

Where you run this matters less than how, but if the provider is still open, the cloud account catalogue compares the options side by side.

For the vendor’s own reference on the services involved here, see the Linode documentation on Akamai TechDocs.

Working through this on a real server is worth the hour it takes. We stock Linode accounts in two configurations, and the cloud account catalogue compares all ten providers side by side.

Questions people ask