Run an AI model locally with Ollama — install to first answer
Install Ollama, pull a model and get your first local answer in about ten minutes — no account, no API key, no per-token cost, and nothing leaving your machine.
Cloud AI is convenient until one of three things bites: you are working with data you would rather not upload, you are watching a per-token bill, or you have no connection. A local model solves all three — and the setup is far less work than it used to be.
Ollama is the shortest path. It handles downloading, quantisation and serving, so getting a model running is two commands.
1. Install Ollama
Download the installer for your platform from ollama.com — macOS, Windows and Linux are all supported. On Linux you can install from the shell instead:
curl -fsSL https://ollama.com/install.sh | sh
Check it is on your PATH:
ollama --version
2. Pull and run a model
One command does both — it downloads the model if you don’t already have it, then drops you into a chat prompt:
ollama run llama3.2
The first run downloads several gigabytes, so it takes a few minutes. Every run after that starts in seconds.
To download without starting a chat, use pull instead:
ollama pull llama3.2
ollama list
ollama list shows what you have locally, with sizes — useful when you start
collecting models and your disk starts complaining.
3. Pick a size that fits your RAM
This is where most first attempts go wrong. A model has to fit in memory, and picking one that doesn’t gives you something that technically runs but is too slow to use.
The rough guide, for 4-bit quantised models:
| Model size | Approximate RAM | Realistic use |
|---|---|---|
| 1–3B | 8 GB | Summarising, simple rewriting, classification |
| 7–8B | 16 GB | General chat, drafting, light code help |
| 13–14B | 32 GB | Better reasoning, longer context |
| 70B | 64 GB+ | Comfortably needs a dedicated GPU |
Start with a 7–8B model on a 16 GB machine. If output is too slow, drop to a 3B. A model that fits and answers in two seconds beats a bigger one that takes a minute.
4. Use it from your own code
Once Ollama is running, it serves a local HTTP API on port 11434. This is
the part that makes it genuinely useful — anything that can make an HTTP
request can use it.
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2",
"prompt": "Summarise this in one sentence: ",
"stream": false
}'
For multi-turn conversations use /api/chat, which takes a messages array
rather than a single prompt:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "Explain webhooks in two sentences." }
],
"stream": false
}'
Set "stream": false to get one complete JSON response. Leave it at the
default and you get a stream of partial JSON objects instead, which is what
you want for a UI that types out the answer but a nuisance for a script.
Expected result
ollama run llama3.2 gives you an interactive prompt that answers without a
network connection, and curl against localhost:11434 returns JSON you can
pipe into anything. No API key exists, because there is no service to
authenticate against — the model is running on your own hardware.
Type /bye to leave the chat prompt.
Where this is worth it
Local models are a clear win for summarising private documents, classifying or tagging large batches of text where per-token cost would add up, and offline drafting. They are still behind the best hosted models on hard reasoning and long-context work — so the practical setup is both: local for the bulk and the sensitive material, hosted for the genuinely difficult asks.
If you want to automate something on top of this, the same localhost:11434
endpoint works as an HTTP step in a workflow tool — see
the n8n beginner guide for
the wiring.
Sources
Stay Ahead. Work Smarter.
Get practical AI tips, automation ideas and useful technology tricks — one useful thing at a time. Follow the feed in any reader.