I wanted a chat model I could talk to on hardware I own, not a token meter on someone else’s GPU. I already had a Mac Mini in the rack running Stable Diffusion through ComfyUI, so the plan was to put a local LLM on the same box and share a models folder between them. Ollama runs models locally, exposes an OpenAI-compatible API, and uses Apple’s GPU. Open WebUI gives it a chat interface. Both run on the Mini.

The machine is an M1 Mac Mini with 8GB of unified memory. As with the image-generation setup, that number decides everything that follows.

Native, because Docker can’t see the GPU

The first instinct is a container. On macOS that’s the wrong move for anything GPU-bound. Docker Desktop runs your containers inside a Linux VM, and that VM has no path to the Mac’s Metal GPU. A containerized Ollama on macOS runs on the CPU, which on an 8GB Mini means painfully slow inference. To get the GPU you run Ollama natively.

Homebrew installs it:

brew install ollama

I run it as a background service through launchd so it survives reboots, the same pattern I use for the image-generation backend. Once it’s up, the API answers on localhost:11434 and ollama pull fetches models.

Open WebUI for the chat interface

Open WebUI is the front end. It also runs natively, installed with uv:

curl -LsSf https://astral.sh/uv/install.sh | sh

It serves a chat UI on port 3000 and talks to Ollama’s API on 11434. Running it on the same host keeps the setup simple; it’s a single-user lab service, so there’s nothing to gain from splitting the UI onto another box.

Picking a model that fits 8GB

This is where the memory ceiling bites. A model has to load into the same 8GB the OS and everything else share. The lever is quantization: a 7B model in Q4_0 is about 3.8GB on disk and loads small enough to run. I started on Llama 2 7B at Q4_0. It fits, but it doesn’t leave much headroom, which is the whole story of this machine.

Keeping memory in check

There’s no magic flag that caps Ollama’s GPU memory, and on macOS you don’t pick the device either, Metal is automatic. What you actually control is how much work Ollama tries to hold at once. Two environment variables did the real work:

OLLAMA_NUM_PARALLEL=2
OLLAMA_MAX_LOADED_MODELS=1

OLLAMA_MAX_LOADED_MODELS=1 stops Ollama from keeping a second model resident, which on 8GB is the difference between responsive and swapping. OLLAMA_NUM_PARALLEL=2 caps concurrent requests so one chat doesn’t get starved by another. The biggest lever, though, isn’t a setting. It’s the model size you choose.

On the Open WebUI side I raised the client timeout so long generations don’t get cut off, and turned off its automatic title generation and autocomplete. Both of those fire extra model calls in the background, and on this hardware every spare call costs you.

Reaching it from the network

A DNS record points ollama.internal at the Mini, so the UI is at http://ollama.internal:3000 on the LAN. For HTTPS it sits behind Traefik like the rest of the lab, reachable at https://ollama.example.net with a real certificate. It also went onto my Homepage dashboard next to the image-generation service, and into Uptime Kuma so I’d know if it fell over.

How fast, really

Honest numbers from this Mini: the 7B model ran around 20 to 30 tokens per second. Usable for a single user, slow if you’re impatient. The fix that costs nothing is a smaller model. Llama 3.2 at 1B did roughly 82 tokens per second and the 3B around 56, both a clear 2 to 3 times faster, at the cost of some answer quality.

I looked at going faster with hardware and the answers were all walls. eGPUs aren’t supported on Apple Silicon at all. There’s no TPU path; Ollama accelerates on NVIDIA, AMD, or Apple Metal, nothing else. And the 8GB is soldered, so there’s no upgrade. The real options were a 16GB Mini for 30 to 50 percent more headroom, or a Linux box with an NVIDIA GPU for a few times the speed and a lot more complexity.

What I’d tell myself

It worked, and running a model entirely on my own hardware still feels like the right default for anything private. But 8GB of unified memory is a hard ceiling, not a starting point, and I eventually retired this setup: the services sat idle more than I used them while holding memory I wanted for other things. Pulling them back off the Mini is its own post.

• On macOS, GPU work runs native or not at all. Docker can’t reach Metal, so a containerized LLM is CPU-bound.
• Memory is the budget. On 8GB, model size plus OLLAMA_MAX_LOADED_MODELS=1 matters more than any other tuning.
• Smaller models are the cheapest speedup. Dropping from 7B to 3B or 1B was 2 to 3 times faster for free.
• Apple Silicon has no escape hatch for more GPU. No eGPU, no TPU, no RAM upgrade. If 8GB isn’t enough, the answer is different hardware.
• Match the service to how often you’ll use it. An idle model still holds resources; I retired mine once honest usage didn’t justify the footprint.