What you will build
By the end of this tutorial, you will have a downloaded language model that accepts prompts in your terminal and through an HTTP endpoint. Ollama provides the runtime and server; the model supplies the language capabilities. Local models require no Ollama API key. Ollama quickstart
We will use llama3.2:1b, a small, one-billion-parameter text model from the Llama 3.2 family. The explicit tag makes the examples consistent. It is a practical starting point for learning the workflow, rather than a promise of high-quality answers. Check the model page for its license and download information. Model listing
You need a terminal, internet access for installation and the initial download, and enough disk space and memory for the model. Download size and runtime memory are different: serving also needs working memory. Hardware and prompt length affect performance.
1. Install Ollama
Windows and macOS: Download the installer from Ollama, install it, and open the application. Open a fresh PowerShell window on Windows or Terminal on macOS.
Linux: The official installation command is:
curl -fsSL https://ollama.com/install.sh | sh
This downloads and executes the installer. You can inspect the script at the linked URL before running it. On a Linux machine with systemd, check the installed service:
sudo systemctl start ollama
sudo systemctl status ollama
The official Linux guide also covers manual installation and service configuration. Linux installation
Check that your terminal can find Ollama:
ollama --version
2. Make sure the server is running
The desktop application or Linux service may already run the server. If you are running Ollama manually, start it with:
ollama serve
Leave that terminal open and use a second terminal for subsequent commands. If you get an “address already in use” error, check the existing server before starting another instance. CLI reference
Ollama binds to 127.0.0.1:11434 by default. This tutorial keeps the server local to your computer. Server configuration
Verify the API on macOS or Linux:
curl http://localhost:11434/api/tags
On Windows PowerShell:
Invoke-RestMethod -Uri 'http://localhost:11434/api/tags'
This endpoint returns downloaded models. An empty list is normal before your first download. List models API
3. Download and chat with the model
Run these commands on any supported platform:
ollama pull llama3.2:1b
ollama list
ollama run llama3.2:1b
pull downloads the model, list shows installed models, and run opens an interactive chat. Try this prompt:
Explain what an HTTP API is to a beginner, in three sentences.
Type /bye to leave the chat. Leaving the chat does not stop the separately running server. You can inspect running models with ollama ps. CLI reference, Quickstart
4. Serve a prompt through HTTP
The running Ollama server already serves your downloaded model. Your application selects it by name in the request; you do not need a separate server process for each model.
On macOS or Linux:
curl http://localhost:11434/api/generate
-H 'Content-Type: application/json'
-d '{"model":"llama3.2:1b","prompt":"Explain local AI in three sentences.","stream":false}'
On Windows PowerShell, build the JSON explicitly:
$payload = @{
model = 'llama3.2:1b'
prompt = 'Explain local AI in three sentences.'
stream = $false
} | ConvertTo-Json
$result = Invoke-RestMethod `
-Uri 'http://localhost:11434/api/generate' `
-Method Post `
-ContentType 'application/json' `
-Body $payload
$result.response
stream: false requests a single JSON response instead of the default streamed output. The generated text is in response; done indicates completion. The wording varies between runs. Generate API
5. Call the model from Python
Save this as local_llm.py. It uses Python's standard library, so no extra package is needed:
import json
from urllib.request import Request, urlopen
payload = {
"model": "llama3.2:1b",
"prompt": "Give me three ideas for a beginner Python project.",
"stream": False,
}
request = Request(
"http://localhost:11434/api/generate",
data=json.dumps(payload).encode("utf-8"),
headers={"Content-Type": "application/json"},
method="POST",
)
with urlopen(request, timeout=300) as response:
result = json.load(response)
print(result["response"])
Run it with python local_llm.py, or python3 local_llm.py if that is your system's Python command. Keep Ollama running. The generous timeout accommodates slow first requests; adjust it for your application. This example is a minimal client, so connection errors will appear as Python exceptions.
6. Manage the model and troubleshoot
Useful commands:
ollama ps
ollama stop llama3.2:1b
ollama rm llama3.2:1b
ps shows loaded models. stop unloads the specified model from memory; rm deletes its local download. CLI reference
If something goes wrong:
- Command not found: Reopen the terminal after installation and confirm Ollama is installed.
- Connection refused: Start the application, Linux service, or manual server, then repeat the
/api/tagsrequest. - Model not found: Run
ollama list, then pull the exact tag used in your request. - Slow responses or memory errors: Close memory-heavy applications and inspect
ollama ps. Its processor column indicates CPU/GPU use. A larger model or longer context can require more memory. Ollama FAQ
Before sharing the endpoint
The local API requires no authentication. Keep its default loopback binding for this exercise. If you later make it reachable by other machines, add access controls, authenticated HTTPS, and network restrictions before sharing it. API introduction, Network configuration
You now have the full local workflow: install Ollama, pull a model, test it interactively, and send prompts from code. Review generated answers before using them as facts or incorporating them into an application.
Leave a Reply