Ollama, LM Studio, llama.cpp or vLLM: which engine to use

What you'll see in this lesson

1. What a local engine does

   The engine loads the model into memory, generates the text and serves it to your programs; the model is the same, how it runs changes.

2. Ollama: one command and you're running

   It's the fastest way to start on Windows, Mac or Linux: pick a model from its catalog and run it.

3. LM Studio: all in windows, and MLX on Mac

   An app to search, download and test Hugging Face models with your mouse, free for business use too.

4. llama.cpp: the engine for those who want control

   Many tools are built on its code; using it directly you decide every setting, and you usually get new models first.

5. Same code, different server: the OpenAI API

   All these engines mimic the OpenAI API: change the base URL and your scripts, n8n or apps work with the local model.

6. vLLM and SGLang: when the model is for the whole team

   They're built to handle many requests at once on NVIDIA or AMD cards, with much higher total throughput than Ollama.

7. Giant MoEs on one GPU: experts to RAM

   Keep attention and the cache on the GPU, since they're used for every token, and send the experts, the heaviest part, to RAM.

8. Which one to choose for your case

   Start simple and switch engines only when you hit a specific limit.

9. Summary

   The model is the same; the engine decides how easy, how fast and how many people it can serve.