Local LLMs with Ollama
Ollama is the standard tool to run LLMs locally. With 5-minute install and 10-minute first model run, you get "AI that runs on your own PC."
Why Run Locally
- Privacy: data doesn't leave (important for medical/legal)
- Cost: no API fees. Just electricity
- Offline: no internet needed
- Customization: freely switch/fine-tune models
- No rate limit: call as much as you want
Required Specs
| Model size | RAM | GPU (recommended) | Speed feel |
|---|---|---|---|
| 3B | 8 GB | Not needed | Quite fast |
| 7B | 16 GB | M1 / RTX 3060 | Normal |
| 13B | 24 GB | M2 Pro / RTX 4070 | Slowish |
| 70B | 64+ GB | M3 Max / RTX 4090+ | Quite slow |
Install
Mac / Linux
brew install ollama # or curl -fsSL https://ollama.com/install.sh | sh
Windows
Download the installer from ollama.com
