How to Run a Local LLM on a Mini PC — 2025 Setup Guide

Running a large language model locally used to require a £3,000 GPU workstation. That’s no longer true. A £400 mini PC with 32GB RAM and the right AMD chip can run a 7B or 13B parameter model at speeds that make it genuinely useful as a coding assistant, private document reader, or offline AI tool — no API key, no monthly subscription, no data leaving your machine.

This guide walks through the full setup: choosing the right hardware, installing Ollama, picking models that fit your RAM, and adding a browser interface so it looks and feels like ChatGPT. The whole process takes under 30 minutes.

What You Need Before You Start

Hardware Requirements

The single most important spec is RAM. LLMs are memory-bandwidth-bound — the model’s weights shuttle between RAM and CPU with every token generated. A Ryzen 9 8945HS with LPDDR5x at 6400MHz moves data faster than the same chip paired with slower memory, and that directly translates to higher tokens per second.

Not sure which mini PC to buy? See our full guide to the best mini PCs for local AI — we’ve tested six models specifically for LLM inference.

Step 1 — Install Ollama

Ollama is the easiest way to run LLMs locally. It handles model downloading, quantisation selection, and provides a CLI and REST API. Installation is a single command on Linux or a one-click installer on Windows.

On Linux (Ubuntu / Debian)

curl -fsSL https://ollama.com/install.sh | sh

That’s it. Ollama installs as a system service and starts automatically. Verify it’s running:

ollama --version

On Windows 11

Download the installer from ollama.com/download. Run it, follow the prompts. Ollama will run as a background service accessible from PowerShell or Windows Terminal. No WSL required — though WSL2 is an option if you prefer a Linux environment.

Step 2 — Download Your First Model

Choose a model based on your RAM. The rule of thumb: a Q4-quantised model needs roughly 1GB of RAM per billion parameters, plus about 2GB overhead.

Pull Llama 3.1 8B as your starting point — it’s the best general-purpose model at this size and the most actively maintained:

ollama pull llama3.1:8b

The download is around 4.7GB. Once it’s complete, start a chat session:

ollama run llama3.1:8b

You’ll see a >>> prompt. Type a message and press Enter. On a Minisforum UM890 Pro, first-token latency is about 2–3 seconds; after that you’ll see around 14 tokens per second for 8B models.

Step 3 — Performance Benchmarks by Mini PC

These are tokens per second on Llama 3.1 8B Q4, measured after a 5-minute warmup to reach thermal equilibrium:

14 tokens per second is the point where a conversation feels responsive — you’re reading the output roughly as fast as it’s generated. 6–8 t/s on a 13B model is workable for longer prompts, but you’ll notice the lag on multi-paragraph responses. The Intel N100 machines are not practical for LLM use — stick to AMD Ryzen 7000 or 8000 chips.

Step 4 — Add Open WebUI (ChatGPT-Style Interface)

The CLI is functional but limited. Open WebUI gives you a proper browser-based interface with conversation history, model switching, and document upload. It runs as a Docker container and connects to your local Ollama instance.

If you don’t have Docker, install it first:

# Ubuntu
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER

Then run Open WebUI:

docker run -d \
  -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000 in your browser. On first launch, create an admin account. Your Ollama models will appear in the model dropdown automatically — no extra configuration needed.

Step 5 — Use LM Studio for a Desktop App

If you prefer a native desktop application over a browser interface, LM Studio is the alternative. It offers a cleaner GUI, a built-in model browser for downloading from HuggingFace, and a local server mode that exposes an OpenAI-compatible API endpoint. This lets you connect tools like Continue (VS Code AI assistant) to your local model.

Download from lmstudio.ai. On Windows, it uses DirectML for GPU acceleration on AMD integrated graphics — which can improve throughput on Ryzen 8000 chips by 20–30% compared to CPU-only inference.

Troubleshooting Common Issues

Model is very slow (under 4 tokens/second)

This usually means the model is too large for your RAM and is swapping to disk. Check RAM usage with htop (Linux) or Task Manager (Windows) during inference. If RAM is maxed out, use a smaller model or a more aggressively quantised version (Q4 instead of Q8, or Q3 if available).

Ollama won’t start after reboot

On Linux, check the service status: systemctl status ollama. If it failed, run sudo systemctl enable ollama && sudo systemctl start ollama. On Windows, check that the Ollama tray icon is running — it should start automatically with Windows.

Open WebUI can’t connect to Ollama

The --add-host=host.docker.internal:host-gateway flag in the Docker command is what connects the container to your host’s Ollama instance. If you missed it, stop and remove the container, then re-run the command with the flag included.

Which Mini PC Should You Buy for Local LLMs?

For the best token throughput, choose a mini PC with AMD Ryzen 8000 series and LPDDR5x memory. The Minisforum UM890 Pro leads the field at 14+ t/s on 7B models. For developer use that combines LLM inference with a full dev environment, it pairs well with Docker and VS Code — see our guide to the best mini PCs for developers.

For a full comparison of AI mini PCs across models and use cases, see the best mini PC for local AI roundup.

FAQ — Running Local LLMs on a Mini PC

Can a mini PC run Llama 3?

Yes. Llama 3.1 8B runs well on any mini PC with 16GB+ RAM and an AMD Ryzen 7000 or 8000 CPU. On a Minisforum UM890 Pro with 32GB, it runs at 14 tokens per second — fast enough for a responsive coding assistant.

Is 16GB RAM enough to run local LLMs?

16GB is enough for 7B models (4–8GB model size) with room for your OS and other apps. 32GB is recommended if you want to run 13B models or keep a browser and dev environment running simultaneously without memory pressure.

What’s the fastest mini PC for Ollama?

The Minisforum UM890 Pro (Ryzen 9 8945HS, 32GB LPDDR5x) recorded the highest tokens per second in our testing at 14.2 t/s on Llama 3.1 8B Q4. The Beelink SER8 is a close second at 12.8 t/s for less money.

Can I use a mini PC instead of a cloud GPU?

For personal use and experimentation, yes. A local mini PC has no ongoing costs, no data privacy concerns, and no rate limits. For high-volume production workloads, a cloud GPU or a machine with a dedicated Nvidia GPU is significantly faster. The break-even on hardware cost vs. cloud API costs depends on your usage, but for daily personal use, local hardware pays for itself quickly.

• Best Mini PCs for Local AI — Full Rankings
• Best Mini PCs for Developers
• All Minisforum Mini PCs Reviewed
• Best Mini PCs Overall 2025