Ollama: run open-source LLMs right on your laptop

Sometimes you don't want to, or aren't allowed to, send data to a cloud API: sensitive internal documents, environments without internet access, or you simply want to try open models for free. Ollama is the fastest way to run open-source LLMs on a laptop: one command to install, one to run, with a built-in REST API for integrating into apps.

Up and running in 2 minutes

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh

ollama run <model-name>    # download (first time) and open a chat
ollama pull <model-name>   # download only
ollama list                # installed models
ollama ps                  # models currently loaded in memory

The model library on ollama.com has many open model families in different sizes and quantisation levels. Pick a size that fits your machine's RAM/VRAM.

Calling it via the API

curl http://localhost:11434/api/chat -d '{
  "model": "<model-name>",
  "messages": [{"role": "user", "content": "Summarise the following passage…"}],
  "stream": false
}'

Ollama serves on port 11434, has official Python and JavaScript libraries, and offers an OpenAI-compatible endpoint, so many existing tools only need a base URL change.

Customising with a Modelfile

FROM <model-name>
PARAMETER temperature 0.2
SYSTEM """You are a ticket classifier. Return exactly one label:
billing, bug, feature, other."""

ollama create ticket-classifier -f Modelfile
ollama run ticket-classifier

Local or API?

Risks and caveats

• Hardware decides the experience: a model that exceeds memory runs very slowly or not at all. Start with small, quantised models.
• Small models are clearly weaker: don't expect a 7–8B model on a laptop to run complex agents like large cloud models.
• Exposing the port: the server listens on localhost by default; if you open it to the network, add authentication or a firewall, or anyone can call it.
• Model licences: each open model has its own licence; check before commercial use.
• "Local" isn't automatically private: third-party integrations or plugins can still send data out; check the whole toolchain.

How to apply it

• Dev: Use Ollama as a stand-in backend for offline development; code against the OpenAI-compatible format so switching to a real API is easy.
• Tech Lead: Try local models for simple, high-volume tasks (labelling, classification) to cut API cost; measure quality with the same eval set.
• PM: For customers who require data to stay on-premises, Ollama is a quick way to build an on-site proof of concept.