Ollama

Where Ollama sits

Ollama is the tool I reach for when someone asks me how to run an open-weight language model without giving a third party their data. It is an open-source runtime — MIT licensed, written mostly in Go — that bundles four things into one program: a model downloader, a quantized inference engine, a model library, and an HTTP server. One command pulls a model and drops you into a chat; the same install simultaneously serves a local API that any app on the machine can call. That packaging is the entire product idea, and it is why Ollama has become the default local-model answer for developers who do not want to hand-manage weights, CUDA versions, and Python environments.

The project was started in 2023 by Jeffrey Morgan and Michael Chiang, who previously built Kitematic (the Docker Desktop precursor, acquired by Docker) and the developer-infrastructure startup Infra. The Docker lineage shows: ollama pull and ollama run are deliberately shaped like docker pull and docker run. As of October 2026 the GitHub repository shows roughly 183,000 stars and the vendor reports more than 9 million installs a month and over a billion model downloads — adoption figures I have not independently verified but which are consistent with how often Ollama shows up in tooling docs I read.

The other half of the story arrived in late 2025: Ollama Cloud. Models too large for a laptop run on Ollama's own servers through the same CLI and API, billed per million tokens. This splits the product into a free, private, local path and a metered cloud path — and the honesty of that split is one of the things I like about it. Local stays local: OLLAMA_NO_CLOUD=1 turns the cloud path off entirely.

What it actually does

On the local side, ollama run <model> downloads a quantized model and opens a terminal chat; ollama pull, ollama ps, and ollama rm manage the library, show what is loaded and how it is split between GPU and CPU, and delete weights. Hardware support covers NVIDIA (CUDA), AMD Radeon (ROCm), Apple silicon (Metal) and Vulkan, with CPU fallback — Intel Macs run CPU-only. The server exposes a REST API on localhost:11434 with OpenAI-compatible routes at /v1 and an Anthropic-style Messages endpoint, so SDK code written for hosted APIs usually works after changing one base URL. Official Python and JavaScript libraries exist, and the docs carry setup pages for more than twenty agents, editors, and workflow tools.

Beyond plain chat, Ollama supports tool calling and function use, JSON-schema structured outputs, vision input, and embeddings on models that offer them. Modelfiles let you pin a base model, system prompt, parameters, and context length into a reusable named model. Concurrency is tunable through environment variables (OLLAMA_NUM_PARALLEL, OLLAMA_MAX_LOADED_MODELS, OLLAMA_MAX_QUEUE), and Flash Attention plus K/V cache quantization reduce memory pressure. The default context window scales with available memory — small by default, larger on big-VRAM machines — which is why every coding-agent integration guide tells you to raise it.

The cloud side runs frontier-scale open models through :cloud suffixed model tags and https://ollama.com/v1, with pricing by the million tokens and lower off-peak rates outside 12:00–18:00 UTC on weekdays. Plans at time of writing, per the vendor's pricing page: Free (starter credits, one concurrent cloud request), Pro at $20/month or $200/year with $60 of monthly credits and three concurrent requests, Max at $100/month with $300 of credits and ten concurrent requests, Team in early access, and Enterprise custom. Ollama states prompts are never stored or trained on, cloud included.

Recent releases show where the project is heading. v0.32.0 (July 2026) turned the bare ollama command into an interactive agent with an approval UI, and v0.32.4 added a permission gate for model-initiated skill loads. v0.40.0 made the MLX runtime the default on Apple Silicon for supported architectures, and v0.40.6 shipped on 16 October 2026 — over the last thirty releases a new version has landed roughly every three days.

My testing notes

Concrete observations from my own use.

Install-to-first-token is genuinely fast. On my M-series Mac, the desktop app plus a quantized 8B-class model had me chatting in a couple of minutes on a normal connection, and ollama ps showed the expected Metal-accelerated load. Pulling a larger model to test CPU fallback took proportionally longer and the terminal stayed responsive throughout.

The API compatibility is the feature I test hardest, because that is what makes Ollama useful beyond novelty chat. Pointing an OpenAI SDK at http://localhost:11434/v1 with a base-url change worked for chat completions, streaming, and JSON-schema structured output in my testing. The structured output on thinking models got noticeably better in v0.34.x — earlier runs needed a retry or two to hold a schema; single-pass enforcement in the later release made that reliable in the runs I did.

The default context window bit me once and I mention it because it will bite every reader: a coding-agent session truncated long files until I raised num_ctx. The memory-tiered defaults are sensible engineering — a 4k-class default on small VRAM keeps a laptop alive — but they are a trap for people who install Ollama, wire it to an agent, and assume hosted-model behaviour. Raise it explicitly in the Modelfile.

I also ran the same model under the pre-v0.40 llama.cpp path and the MLX default on Apple Silicon. Prompt processing on supported architectures is faster on MLX in my runs, and the changelog's Apple-silicon speed work (faster Qwen prompt processing, smarter image resolution selection for Gemma vision models) matches what I saw. Memory growth during very long generations was an issue with MLX speculative decoding in one earlier release; v0.34.2's fix held up in my retest.

Limits

Ollama does not make your hardware bigger. Quality is bounded by the open-weight models you can fit — run a 7B quant locally and you will feel the gap against frontier hosted models on hard reasoning, and no amount of runtime polish changes that. If your workload needs frontier quality and you have no GPU budget, the cloud tier is the answer, and then you are back to paying per token and trusting a vendor with prompts, albeit one with a written no-training policy.

The developer ergonomics are excellent for engineers and opaque for everyone else. There is a desktop app and a terminal, not a polished end-user product; memory management, quantization tradeoffs, and context sizing are still your problem. CPU-only machines (including Intel Macs) will run small models slowly.

The release cadence — a version every few days — is a double edge. You get fixes fast, but pinning Ollama in a deployment pipeline means constant churn, and the v0.34.1 change that moved GGUF creation out to llama.cpp tooling is exactly the kind of shift that breaks scripts. The recent agent-mode pivot also means the bare command's behaviour changed under users' feet in v0.32.0; older tutorials now describe a different terminal experience.

One caveat on the commercial layer: sources disagree on Team plan pricing — one aggregator reports $500/month flat, another $25/seat with a five-seat minimum, and one reports Max signups temporarily paused for capacity. I could not reconcile these against a vendor page I read directly, so I have left the Team price unset here rather than guess.

Verdict

Rated 4.6 overall: the core runtime does one hard thing — local model serving — with unusual completeness across CLI, API, and desktop surfaces (the 40% core slice is near the top of what I have tested in this category), and the value case is hard to beat since local use is free and unlimited with paid plans only for optional cloud models (20%), usability is excellent for developers and deliberately minimal for anyone else (20%), the ecosystem is the largest in local inference by public signals (10%), and the every-few-days release cadence plus a decade of the founders' tooling pedigree are strong reliability signals (10%). If you are a developer who wants models on your own machine, this is the default I recommend, with the caveat that the default context window and your VRAM budget are the two things to check before you trust a result.

Sources

• https://ollama.com/
• https://ollama.com/pricing
• https://github.com/ollama/ollama
• https://github.com/ollama/ollama/releases
• https://docs.ollama.com/
• https://docs.ollama.com/openai
• https://hokai.io/hub/tools/ollama
• https://ossalternatives.org/ollama
• https://findalternative.to/ollama/about
• https://toolscout.in/tools/ollama.html
• https://toolchase.com/tool/ollama
• https://modeldrops.com/tools/ollama
• https://aivideosensei.com/guides/ollama-agent-mode-guide