Linux homelab guide

Self-Host Ollama and Open WebUI with Docker on a Linux Homelab

A local language model is much easier to live with when you can open a browser instead of juggling shell commands and API endpoints. Ollama provides the model runtime; Open WebUI provides the browser-based chat interface.

This guide builds a two-container Docker Compose stack with CPU-first inference, persistent volumes, a private Ollama network path, and an optional NVIDIA GPU phase. Image tags, model names, and configuration behavior can change, so check the linked documentation before making a long-lived deployment. [1]

Table of contents

• What you are building
• Prerequisites
• CPU-first Compose stack
• Start and pull a model
• Connect Open WebUI
• NVIDIA GPU support
• Secure the deployment
• Updates and backups
• Troubleshooting
• FAQ
• At a glance

What you are building

Ollama downloads and serves models. Open WebUI provides the browser UI, accounts, settings, and chat data. They share Compose's default private network, where Docker resolves the service name ollama.

Stack summary

Service	Role	Published address	Persistent data
Ollama	Model runtime and API	None in this example; internal port 11434	ollama volume
Open WebUI	Browser interface and application data	Host 3000 to container 8080	open-webui volume

Original architecture diagram; this is not a screenshot.

Prerequisites

Use a Linux machine that stays on when you need the service and allow enough disk space for your models. CPU inference needs no discrete GPU, but speed depends on processor, memory, model size, quantization, and concurrent use.

Install Docker Engine and the Compose plugin using your distribution's current instructions. Ubuntu users should follow Docker's official guide, especially when replacing an older package. [6]

docker --version
docker compose version

If Docker works only with sudo, use that method consistently or complete the documented post-install configuration. Docker access can provide broad control over a host, so do not casually grant it on a shared machine.

Create the CPU-first Compose stack

mkdir -p ~/homelab/ollama-openwebui
cd ~/homelab/ollama-openwebui

Save the following as compose.yaml:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open-webui:/app/backend/data

volumes:
  ollama:
  open-webui:

Port 11434 is intentionally not published. The named volumes preserve models and Open WebUI data when containers are recreated. Moving tags such as latest and main are convenient for experiments; pin reviewed release tags for more controlled updates. Check Open WebUI's environment reference for supported variables. [3]

Start containers and download a model

docker compose config
docker compose up -d
docker compose ps
docker compose logs --tail=50 ollama open-webui

After Ollama starts, pull an example model into its persistent volume:

docker exec -it ollama ollama pull llama3.2:3b
docker exec ollama ollama list

The model name is an example. Pick one that fits available memory and your task. Do not remove the ollama volume casually: explicit volume removal deletes downloaded model data.

Connect Open WebUI to Ollama

Open http://localhost:3000 on the Docker host. From another LAN machine, use http://HOST_IP:3000 with the server's private address. Complete first-run account setup and select the pulled model.

If Open WebUI asks for an Ollama URL, use http://ollama:11434. Do not use http://localhost:11434 from inside Open WebUI: there, localhost means the Open WebUI container. If a reverse proxy is used, configure WebSocket forwarding and suitable idle timeouts for real-time features. [2]

Optional NVIDIA GPU support

GPU support is a second phase. First verify host hardware and a compatible driver, then install and configure NVIDIA Container Toolkit according to current vendor guidance. A host GPU is not automatically available inside a container.

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

docker compose up -d --force-recreate ollama

Driver, toolkit, Compose, architecture, and model compatibility matter. Follow the current Ollama GPU documentation; AMD needs a different device and ROCm setup. [5] See the Docker container guidance too. [4]

Secure the homelab deployment

Do not expose Open WebUI to the public internet just because port forwarding is easy. Keep it on a trusted LAN, or use a VPN or carefully configured HTTPS reverse proxy. Use a strong, unique administrator password and separate accounts where appropriate.

Restrict firewall access to port 3000. Be aware that Docker-published ports can interact differently with host firewall tools because Docker installs its own forwarding and NAT rules; verify reachability from an untrusted LAN device instead of assuming an uncomplicated UFW rule is sufficient. Since 11434 is not published here, Ollama is not directly reachable through the host LAN. If another trusted application needs it, publish it intentionally and restrict access.

Default Docker installations commonly involve a privileged daemon. Rootless Docker runs the daemon and containers without root privileges and may fit some setups; read its requirements and limitations before switching. [7] Do not share secrets in logs or public Compose files.

A practical backup and sizing routine

Before changing images or trying a new model, record the resolved stack and volume names. Compose can add a project prefix to volume names, so check the result rather than assuming the short names are literal.

docker compose config
docker volume ls
docker system df -v

For a simple archive of the Ollama volume, stop the writer first, create a compressed archive somewhere with enough free space, and start the services again:

docker compose stop
mkdir -p ./backups
docker run --rm -v ollama:/source:ro -v "$PWD/backups":/backup alpine \
  tar czf /backup/ollama-$(date +%F).tgz -C /source .
docker compose start

That command assumes the volume is literally named ollama; use the name shown by docker volume ls if Compose prefixed it. Repeat the pattern for open-webui, changing the volume and archive name. Test an archive by restoring it to a disposable volume or host, not by overwriting live data. A backup that has never been restored is only an assumption.

Model files can consume far more disk than the YAML suggests. Keep several gigabytes free before pulling, and remember that download size is not a complete performance specification: runtime memory, context length, and concurrent requests matter too. Start with a smaller quantized model, watch docker system df and host memory, then move up only when the workload is comfortable. A model may be re-downloadable, while accounts, settings, and conversations in open-webui may be harder to reproduce.

Updates, backups, and maintenance

Back up both named volumes: ollama holds models and open-webui holds application data. A temporary-container archive is one possible method; confirm you understand the volume location and can restore it.

docker compose pull
docker compose up -d
docker system df
docker compose ps
docker exec ollama ollama list

For pinned tags, edit deliberately and keep the previous file for rollback. Avoid blind docker system prune; its options can remove unused resources you expected to keep.

Troubleshooting

The page does not load

docker compose ps
docker compose logs --tail=100 open-webui

Confirm port 3000 is available, use the host's private IP, and check the firewall.

Open WebUI cannot find Ollama

docker compose logs --tail=100 ollama open-webui
docker exec open-webui getent hosts ollama

Confirm the URL is exactly http://ollama:11434 and recreate the affected service after configuration changes.

Pulls fail or responses are slow

Check outbound access, disk space, model spelling, memory, and logs. Try a smaller model or reduce simultaneous use. GPU acceleration is not automatic; verify its driver and toolkit path first.

Streaming fails behind a proxy

Check WebSocket forwarding and idle timeouts, then compare direct LAN access with proxy access. [2]

Frequently asked questions

Can I run it without a GPU?

Yes. The example requests no GPU. Speed and usable model sizes still depend on CPU and memory.

Should I expose Ollama's port?

Usually not. Open WebUI uses Docker's internal network. Publish 11434 only for a specific trusted integration and restrict access.

Where are models and chats stored?

Models are in the ollama volume at /root/.ollama. Open WebUI data is in open-webui at /app/backend/data. Removing volumes removes stored data.

Can I change models later?

Yes. Pull another model, run ollama list, and select it in Open WebUI. Requirements vary by model and quantization.

Do I need a reverse proxy?

No for a trusted LAN. A proxy is useful for HTTPS, a hostname, and centralized controls; configure WebSockets for streaming.

At a glance

Essential commands and settings

Task	Command or setting
Create directory	mkdir -p ~/homelab/ollama-openwebui && cd ~/homelab/ollama-openwebui
Validate	docker compose config
Start	docker compose up -d
Pull model	docker exec -it ollama ollama pull llama3.2:3b
Open WebUI	http://localhost:3000
Internal Ollama URL	http://ollama:11434
Stop containers	docker compose down

Conclusion

Ollama and Open WebUI stay manageable when the first deployment is small: two services, CPU-first inference, persistent volumes, and no unnecessary public exposure. Verify the service name, start with a model your host can handle, and add GPU acceleration only after the basic stack is stable. Back up both volumes and update images deliberately.

Sources

1. Open WebUI Quick Start — [1]
2. Open WebUI Connection Error Troubleshooting — [2]
3. Open WebUI Environment Configuration — [3]
4. Ollama Docker — [4]
5. Ollama GPU — [5]
6. Docker Engine installation on Ubuntu — [6]
7. Docker Rootless mode — [7]

Before deploying: Confirm current documentation, image tags, drivers, and model requirements for your host.