Self-hosted TTS or an API: the crossover

The self-hosting question usually gets answered with an ideology instead of a number. Here is the number, along with the parts of it that do not appear on an invoice.

Where do the cost curves actually cross?#

Around 290 million characters a month, against a $5 per million API and a GPU costing about $2 an hour. That is roughly 4,800 hours of synthesised audio, or 160 hours a day.

The arithmetic, so you can substitute your own inputs:

Below that line the API is cheaper in raw spend, before you count any operational cost. Above it, the GPU wins and keeps winning, because its cost is flat while the API bill grows linearly.

The crossover moves sharply with the API rate you are comparing against. That is the part worth internalising:

Against a $50 API, self-hosting pays off at 29 million characters a month, which a modestly successful voice product hits quickly. Against a $5 API you need ten times the volume before the GPU is worth the trouble. Cheap inference does not just save money, it moves the point where you have to become an infrastructure team.

Can one GPU even handle that volume?#

Comfortably, and that is why the flat line is flat.

Gepard's aggregate throughput reaches 204 times real time at 256 concurrent streams on a single server-class GPU, a figure that comes from vLLM's continuous batching and PagedAttention doing the heavy lifting, with 64 to 128 streams the recommended range for live traffic. At 102x real time, a GPU running flat out produces about 102 hours of audio per wall-clock hour, which is roughly 6.1 million characters an hour.

At 100% utilisation that is 4.5 billion characters a month, so capacity is not the constraint. Utilisation is.

What does utilisation do to the number?#

It is the variable most estimates quietly assume away, and it is usually the one that decides the answer.

A GPU bills for every hour it is powered on, not for the audio it produced. Voice traffic is rarely flat: it peaks in business hours, collapses overnight, and differs by weekday. A workload that looks like 8 busy hours on weekdays averages under 20% utilisation across the month.

At 5% average utilisation, a self-hosted GPU costs more per character than our API does. You have taken on a GPU, a deployment and an on-call rotation to spend more money.

Autoscaling helps and does not eliminate the problem, because model load time means you cannot scale to zero between calls without paying a cold start on the next one.

What does self-hosting cost that is not the GPU?#

The line items that never make it into a comparison table, and routinely exceed it for a small team.

Someone on call. A voice agent that stops speaking is an outage. That is a rota, not a task.

Upgrades. Model versions, vLLM releases, CUDA drivers, the host OS. Each is a scheduled risk against a live service.

Autoscaling and cold starts. Loading weights takes time. Handling a traffic spike without dropping calls means either paying for idle headroom or accepting a cold start on the edge of a peak.

Monitoring. Not just uptime. Time to first audio drifting from 50 ms to 200 ms is an outage your users feel and your dashboard may not show.

Capacity headroom. Sizing for the median means failing at the peak, so you buy for the peak and pay for it constantly.

Multi-region. Latency is geography. One GPU in one region gives users on another continent a worse experience than an API with edge presence.

So which should you pick?#

Use the API when you are below the crossover, your traffic is spiky, you have nobody to own a GPU, you need multi-region latency, or you are still changing the product weekly. This covers most teams for longer than they expect.

Self-host when you are meaningfully past the crossover with steady traffic, your audio cannot leave your infrastructure, you need the network round trip gone, or you need a version pinned for longer than any vendor will support it.

Run both when you have a steady base load and spiky peaks. Self-host the base, burst to the API. This only works if both sides run the same model, which brings us to the part that is not about money.

The thing that decides this more than cost#

Whether you can change your mind later.

With a closed vendor, "self-host" is not a slower option, it is not an option. Crossing the volume threshold, or winning a customer with an air-gap requirement, means a migration to a different model with different voices and different output. Every prompt retuned, every voice re-chosen, every regression re-found.

With Apache 2.0 weights, the same model runs in both places. Start on the API because it ships fastest. Move to your own GPUs when the arithmetic says so. Deploy on-premise when a customer requires it. Same weights, same voices, same wire protocol.

That is the argument for open weights that survives contact with a spreadsheet, and it is worth understanding before you commit either way. If you are still comparing rates, ten vendors on one axis is the place to start.

Every figure here is arithmetic you can redo with your own GPU price and your own utilisation. We would rather you ran it than took our word for which side of the line you are on.