## Prerequisites - Windows 11 or Ubuntu 22.04 PC with at least 8GB of RAM (16GB recommended) - Nvidia GPU with Official Driver Installed (Optional, but recommended) On Windows, you don't need to install any GPU driver on WSL, the official Nvidia Driver on Windows will take care of everything. ## Docker Installation (Linux/WSL) Based on the docker documentation [[1](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script), [2](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)], you can install docker with the following commands: ```js curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh sudo usermod -aG docker $USER newgrp docker ##Test docker run hello-world ``` ## Activate GPU compatibility Based on Nvidia documentation [[1](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)], you can install the Nvidia Container Toolkit with the following commands: ```js curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list sudo apt-get update sudo apt-get install -y nvidia-container-toolkit ``` ```js sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker ##Test docker run --gpus all nvidia/cuda:11.5.2-base-ubuntu20.04 nvidia-smi ``` ## Installation Try any service without fear, you can always go back You can select any of the following docker-compose files to install the services you need.
You can even select a different one in the future if you need more a less services. It will replace the current docker-compose.yaml file with the new one.
Then just use the corresponding command from the How to run it section, and docker will handle the rest. ### Ollama Only ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-ollama-cpu-only.yaml ``` ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-ollama-gpu.yaml ``` ### OpenWebUI Only ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-open-webui-cpu-only.yaml ``` ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-open-webui-gpu.yaml ``` Open WebUI now includes some GPU accelerated features, like the document embembing, whisper TTS, and more. ### OpenWebUI + Ollama ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-ollama.yaml ``` ```js wget -O docker-compose.yaml https://raw.githubusercontent.com/fixtse/blueprints/main/docker/docker-compose-ollama-webui-gpu.yaml ``` ### How to run it ```js docker compose up -d ``` ```js docker compose up -d --remove-orphans ``` **Optional** [Activate WSL Server Mode](/blog/ollama-home-assistant#activate-wsl-server-mode).
To allow external connections to Ollama and OpenWebUI.

Easiest way to get your own Local AI: Ollama Tutorial