When not to self-host an LLM.
Self-hosting an LLM is not worth it when you have no compliance pressure, light usage, and no sensitive data: a cloud API is cheaper, simpler, and better at frontier tasks. It is worth it when data is regulated or confidential, usage is heavy and steady, or auditors keep asking where your data goes.
We build on-premise AI for a living, and nearly every page ranking for this question is written by someone who does too. So instead of a pitch, here is the disqualification list we actually use in assessments, including the cases where we tell people to stay on the cloud.
Stay on the cloud if this is you.
- No compliance driver. If no regulator, auditor, or client contract cares where your prompts go, the strongest reason to self-host doesn't apply to you.
- A small API bill. At low volume, per-token pricing is a genuinely good deal. Hardware and upkeep won't pay back against a modest monthly spend.
- You need frontier reasoning. For open-ended research and novel hard problems, the big cloud models still win. Fine-tuned open models are competitive on scoped tasks, not open-ended ones.
- Nobody can own a server. A supported deployment still needs someone on your side who knows it exists. If there is genuinely no IT person or partner, the cloud's convenience is worth its trade-offs.
- Spiky, unpredictable usage. Hardware sized for your peaks sits idle the rest of the time. Clouds absorb spikes; a box in your rack doesn't.
Self-hosting earns its keep when.
- Sensitive or regulated data. Patient records, privileged documents, financial data. Here the question isn't cost; it's whether you may use a third-party AI service at all.
- Heavy, steady usage. API bills scale with every prompt, forever. Hardware you own amortises, and steady load is exactly what it amortises against.
- Security questionnaires keep flagging AI. SOC 2 and ISO 27001 audits ask about AI subprocessors. That list problem disappears when the model is yours.
- A ChatGPT ban your staff ignore. A ban without an alternative is a data-leak policy. A sanctioned internal model gives people the tool without the leak.
- Offline or air-gapped requirements. Some environments simply cannot have data leave. Open models run fully offline once installed; cloud APIs can't.
- IP that must not leave the building. CAD files, process know-how, client work product. It leaves the building the moment it hits a public API.
Is self-hosting an LLM worth it? A quick test.
| Ask yourself | If yes |
|---|---|
| Would a regulator, auditor, or client contract object to your data sitting in a third-party AI service? | Self-host |
| Is your monthly AI spend small, and likely to stay small? | Stay on the cloud |
| Do you use AI heavily every working day, on the same kinds of tasks? | Self-host |
| Does your best use of AI involve open-ended, frontier-level reasoning? | Stay on the cloud |
| Has an auditor or customer already asked where your AI data goes? | Self-host |
Mixed answers are normal; most companies have some workloads that belong on their own hardware and some that don’t. That split is what an assessment maps. If you want the fuller picture of what running your own model involves, start with what a local LLM for business actually takes.