Installing Ollama on Raspberry Pi: complete tutorial and practical guide

Tutorials

Introduction

Artificial intelligence is now available to everyone, and not just on powerful servers. With Ollamayou can run language models (LLMs) directly in local.
This tutorial shows you how to install Ollama on a Raspberry Pi 5 (or Pi 4)and how to start using your first AI models locally.

1. Prerequisites

Recommended equipment

  • Raspberry Pi 5 (or Pi 4, but less powerful).
  • 8 GB RAM recommended (4 GB possible with smaller models).
  • Card microSD 32 GB minimum or better : SSD NVMe / USB.
  • Internet connection to download Ollama and the models.

Software

  • Raspberry Pi OS (64-bit) based on Debian Bookworm or Bullseye.
  • An accessible terminal (via keyboard/screen or SSH).

2. System update

Before installing, update your Pi :

sudo apt update && sudo apt upgrade -y

Then restart :

sudo reboot

3. Installing Ollama on Raspberry Pi

Ollama offers an official installation script. Download it and run it:

curl -fsSL https://ollama.com/install.sh | sh

⚡ This installs Ollama and automatically configures the system service.

4. Check installation

Once installed, check that Ollama is active:

ollama --version

You should see a version number displayed.

You can also check that the service is running:

systemctl status ollama

5. Download and run an IA model

Example with LLaMA 2

To execute LLaMA 2type :

ollama run llama2

The template will be automatically downloaded (several hundred MB in size).

Example with Mistral

ollama run mistral

Example with a lighter model (TinyLlama)

ollama run tinyllama

👉 On the Raspberry Pi, prefer the streamlined and quantified models (Q4, Q5), otherwise performance will be very slow.

6. Interact with Ollama

Once the model has been launched, you can chat directly in the terminal.

Example:

> Hello, who are you?
I'm a language model powered by Ollama!

To exit :

/bye

7. Using Ollama via API REST

Ollama integrates a Local API. You can access it on the port 11434.

Example with curl :

curl http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt": "Explain the Raspberry Pi in 3 sentences."
}'

👉 Result: a JSON response containing the generated text.

This allows you to integrate Ollama into your applications, websites or home automation projects.

8. Optimizations for Raspberry Pi

  • Using an SSD rather than a microSD card → faster and more reliable.
  • Choose lightweight models TinyLlama, GPT4All-J, Vicuna quantisé.
  • Limit long prompts The Pi has little memory, so avoid overly large entries.
  • Ventilation If you use the Pi 5 intensively, you will need to install a fan or a ventilated housing.

9. Routine troubleshooting

Ollama won't start

Check the service:

systemctl restart ollama

Insufficient memory error

Try a smaller model or reduce the context size. Example:

OLLAMA_CONTEXT=512 ollama run tinyllama

Slow download

Use an Ethernet connection for the first download of models.

10. Concrete applications with Ollama on the Pi

  • Local personal assistant → command-line chatbot.
  • Home AI server → connected to Home Assistant to control your home in natural language.
  • Learning → understand how LLMs work and their limitations.
  • Development → create local apps (intelligent note-taking, summaries, text generators).

Conclusion

Install Ollama on Raspberry Pi is an accessible and exciting way to explore artificial intelligence locally. Of course, the Pi can't compete with GPU-equipped servers, but it does make it possible to test, learn and create concrete projects with minimal cost and energy consumption.

👉 With Ollama, your Raspberry Pi becomes a personal AI mini-serverentirely under your control.

Author