Blog / Build with AI /
Ollama explained for beginners
Most AI chat tools work like a restaurant: you send an order to someone else’s kitchen and wait. Ollama is closer to cooking at home. It is a free tool that lets you download an open language model and run it on your own computer, with no account and no message leaving your machine.
If that sounds appealing and slightly intimidating, good. This post explains what Ollama is in plain words, what you can realistically do with it, and when you should not bother.
What Ollama actually is
A language model is a very large file of numbers that has learned patterns from text. To use one, you need software that loads the file and turns your question into an answer. Setting that up used to mean installing a stack of tools, matching versions and fighting with drivers.
Ollama packages all of that. You install one app, type a command naming a model, and it downloads the model and gives you a chat. It started on Mac and also runs on Linux, and it works from a terminal. Check its site for the current platforms before you plan around it, because this is a young project that is moving fast.
It works with open models, such as Meta’s Llama 2, released in July 2023, and Mistral 7B, released only days ago. Those are models whose weights are published, so anyone can run them within their licence.
How it works, with an analogy
Think of a model as a huge cookbook, and your computer as a kitchen. A big cloud service is a restaurant with a huge kitchen and a team. Your laptop is a home kitchen: slower, smaller, but yours.
A model is described by its size. You will see names like “7B”, meaning roughly seven billion parameters, the numbers the model learned. Bigger models usually give better answers and need more memory. To make them fit on ordinary machines, they are often quantised, which means the numbers are stored with less precision. It is like saving a photo as a smaller file: you lose a little detail and gain a lot of room.
Ollama handles that choice for you. You ask for a model by name, it picks a sensible version, downloads it once, and keeps it.
The basic flow looks like this:
- Install Ollama.
- In a terminal, ask it to run a model by name.
- Wait for the first download, which can be several gigabytes.
- Type questions at the prompt and read the answers.
It also runs a small local service in the background, which other programs can talk to. That is the part builders get excited about.
Why it matters to a normal person
Privacy. Your text stays on your computer. For notes, drafts or anything you would hesitate to paste into a website, that is a real benefit.
It works offline. After the download, no connection is needed. If your data bundle is expensive or your connection drops, that is a practical advantage, not a nerdy one.
No per-message cost. Running a model costs electricity and patience, not tokens. For experiments, that is freeing.
Why it matters if you run websites
If you build small tools, a local model is a safe place to try ideas before paying for an API. You can test prompts, draft product descriptions or summarise notes without sending client content anywhere. We looked at the privacy side of this in why AI forgets your project and how to work around it, and the hardware side in laptops for running local AI models.
What it will not do
- Match the best cloud models. Small open models are impressive for their size, but for hard reasoning, long documents or tricky code, the large hosted models are still ahead.
- Run well on any machine. Memory is the limit. An older laptop with little RAM will be slow, or will not manage larger models at all.
- Think for itself. It makes mistakes, including confident ones. Everything in AI hallucinations in code: spotting invented functions applies here.
- Be a one-click chat app. At this stage it lives in a terminal. Friendlier interfaces are likely to appear around it, but do not assume that today.
Common myths
- “Local means private and safe.” Private, yes, because your text does not leave your computer. Safe, no: the model can still be wrong, and a downloaded file is software you chose to trust.
- “Open means free for any use.” Each model has its own licence. Read it before building something commercial.
- “I need a graphics card.” Helpful for speed, not always required. Smaller models can run on a normal processor, only more slowly.
Who should try it
You will enjoy it if you like tinkering, care about privacy, or want to understand how these models actually run. If you only want good answers fast, a hosted chat tool will serve you better today.
A sensible first afternoon
If you want to try it, keep the first session small. Pick one small model, ask it to summarise a paragraph you wrote, then ask it something you already know the answer to, so you can judge its accuracy. Notice how long it takes and how warm your laptop gets. Then try the same question on a hosted chat tool and compare. That comparison teaches you more about the trade-offs than any article can, including this one.
Key takeaways
- Ollama downloads and runs open language models on your own computer.
- Its value is privacy, offline use and zero per-message cost.
- Model size and memory decide how well it runs.
- It does not beat the biggest hosted models, so expect trade-offs.
Related reading
- Prompt injection explained for builders
- Software licences explained: free, open-source, subscription
If you tried a model on your own machine, what would you ask it first?