The Short Answer
The best providers for self-hosted LLMs are inference runtimes, not platforms: Ollama and llama.cpp for local and edge, vLLM and Hugging Face TGI for production throughput, LocalAI and Open WebUI for the interface. But a runtime only serves tokens. On ibl.ai you own all the code and the data around it — the agents, memory, and governance — model-agnostic across any of those engines.
Every shortlist of "self-hosted LLM providers" returns the same six open-source projects. They are all genuinely good, and choosing between them is mostly a throughput-and-hardware question that this guide answers directly.
The part those shortlists leave out is what happens after the runtime is running. An inference engine hands you a token stream.
It does not hand you agents, retrieval over your own systems, identity, audit logs, or governance — and those are what a security review actually asks about.
What are the best providers for self-hosted LLMs?
There is no single provider, because "self-hosted LLM" spans three separate layers that different projects occupy.
The inference engine loads model weights onto your hardware and serves completions. vLLM is the production default; llama.cpp is the CPU-and-consumer-GPU specialist; Hugging Face TGI is the Hugging Face-native production server.
The local runtime wraps an engine in a one-command developer experience. Ollama is the dominant option here, and LocalAI exposes an OpenAI-compatible API so existing client code works unchanged.
The interface gives humans somewhere to type. Open WebUI is the standard self-hosted chat front end, and it speaks to Ollama or any OpenAI-compatible endpoint.
A complete self-hosted deployment usually means one from each layer — commonly Ollama or vLLM underneath, Open WebUI on top.
What is the difference between an inference engine and an AI platform?
An inference engine and an AI platform solve different problems, and conflating them is the most common self-hosting mistake.
An inference engine — vLLM, TGI, llama.cpp — takes a prompt and returns tokens. It is stateless, single-purpose, and deliberately so. That focus is why vLLM is fast and why llama.cpp runs on a laptop.
An AI platform is everything that turns token generation into a system somebody will approve: agents that take actions, memory that persists across sessions, retrieval over your SIS, LMS, CRM, or document stores, identity and role-based access, audit logging, and policy guardrails.
Self-hosting the engine gives you model sovereignty. It does not give you the platform, and teams routinely discover this after the proof of concept works and the security review begins.
Which self-hosted LLM runtime should you choose?
Choose a self-hosted LLM runtime by where it runs and how many concurrent users it serves, not by popularity.
| Project | Layer | Best for | Runs on |
|---|---|---|---|
| vLLM | Inference engine | High-throughput production serving, many concurrent users | Datacenter GPU |
| Hugging Face TGI | Inference engine | Teams already standardized on the Hugging Face ecosystem | Datacenter GPU |
| llama.cpp | Inference engine | Quantized models on CPU, consumer GPU, or edge hardware | CPU / consumer GPU |
| Ollama | Local runtime | Fastest path from zero to a running local model | Laptop / workstation |
| LocalAI | Local runtime | Drop-in OpenAI-compatible API for existing client code | CPU / GPU |
| Open WebUI | Interface | Self-hosted chat UI over any OpenAI-compatible endpoint | Any |
| ibl.ai | Platform | Agents, memory, governance, and integrations over any of the above | Your cloud, on-premise, or air-gapped |
For production serving, vLLM is the default recommendation for a specific technical reason.
Its PagedAttention memory manager reports 2–4× higher throughput than prior serving systems at the same latency, by eliminating the KV-cache fragmentation that wastes GPU memory under concurrency.
What hardware do you need to self-host an LLM?
Hardware for self-hosting an LLM is set by model size and quantization, not by the runtime you pick.
A quantized 7B–8B model in GGUF format runs on a modern laptop CPU or a consumer GPU through llama.cpp or Ollama. This is enough for development, single-user work, and edge deployments with no network.
A mid-size open-weight model serving a real user population needs datacenter GPUs. A single reserved H100 at roughly $1.50–3/hour handles tens of thousands of requests per day, which covers most organizations running one department's workload.
Frontier-scale open-weight models need multiple GPUs and tensor parallelism, which vLLM and TGI both support and llama.cpp does not target.
The practical rule: start quantized on hardware you already own, measure real concurrency, and only then size the GPU purchase. Most teams over-provision because they size for peak marketing numbers rather than measured load.
How much does self-hosting an LLM cost versus per-seat AI licences?
Self-hosting an LLM and licensing a per-seat assistant scale on completely different variables, which is why the gap widens with headcount rather than staying constant.
Per-seat SaaS bills every employee every month whether or not they use it. Self-hosted inference bills GPU hours, which track actual load. At 5,000 users the difference stops being a discount and becomes a different category of expense.
| Option | Basis | Monthly @ 5,000 users |
|---|---|---|
| ChatGPT Enterprise | ~$60 / user / month | $300,000 |
| Glean | ~$40 / user / month | $200,000 |
| Microsoft Copilot | ~$30 / user / month | $150,000 |
| Self-hosted inference (2× reserved H100) | GPU hours | ~$2,920 |
One honest caveat, because it is the whole point of this page: the GPU line is inference only.
The per-seat products include an application layer — search, agents, connectors, administration. The difference between those two numbers is the platform layer, which you either build yourself or own outright.
That is the real decision. Not "is self-hosting cheaper" — it is — but "who builds and owns everything the inference engine does not do." Our full breakdown of what AI actually costs in 2026 works the token math for both paths.
Who owns the stack once you self-host an LLM?
Self-hosting an LLM gives you ownership of the model layer and nothing above it, which is the gap most deployments discover late.
You own the weights, the GPUs, and the inference process. Your prompts and documents never leave your perimeter. That is real sovereignty, and it is why regulated buyers self-host in the first place.
But the agents, memory, retrieval connectors, identity integration, audit trail, and guardrails are still unbuilt. Assembling them in-house is a multi-quarter engineering program. Buying them from a managed vendor hands back the control you self-hosted to obtain.
On ibl.ai you own all the code and the data — the platform layer under a perpetual licence, running on your infrastructure, model-agnostic across vLLM, Ollama, TGI, or a hosted API, with no per-seat pricing. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.
ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.
Frequently asked questions
Is Ollama production-ready for many concurrent users?
Ollama is optimized for single-user and small-team local use. For many concurrent users, vLLM or TGI are the appropriate choices because they implement continuous batching and paged KV-cache management that Ollama does not target.
Can you run a self-hosted LLM with no internet connection at all?
Yes. llama.cpp, Ollama, vLLM, and TGI all run fully offline once weights are on disk, which is what makes air-gapped deployment possible. Our guide to running LLMs with zero external calls covers the network posture in detail.
Does choosing a runtime lock you into a model?
No, and this is the main advantage over per-seat assistants. All six projects load open-weight models from disk, so switching from one model family to another is a configuration change rather than a vendor migration.
What is the difference between this and a self-hosted AI platform?
A runtime serves tokens; a platform provides agents, memory, governance, and integrations on top. Our comparison of self-hosted enterprise AI platforms covers the platform layer, including Onyx, Cohere, Glean, and ibl.ai.
The bottom line
Pick the runtime on throughput and hardware: vLLM or TGI for production, llama.cpp or Ollama for local and edge, Open WebUI for the interface. That decision is well-solved and takes an afternoon.
Pick the platform on ownership, because that decision is not reversible cheaply. The runtime is the easy half of self-hosting; everything above it is where deployments stall, and where the question of who owns the code and the data actually gets settled.