How to install AnythingLLM with Docker, HTTPS and Ollama

AnythingLLM is an all-in-one AI application: you collect documents in workspaces, chat with them through the language model of your choice, and run agents that can browse the web or call tools. The Docker edition is the multi-user version meant for servers, with roles, password protection and an API.

This guide runs the official mintplexlabs/anythingllm image with Docker Compose on Ubuntu or Debian. You keep all data in a host folder, set your own signing secrets, switch on multi-user mode through an SSH tunnel before the app is reachable from the internet, publish it over HTTPS with Caddy and connect it to an Ollama server on the same machine. The guide ends with backups, updates and fixes for the errors people hit most often.

lscpu | grep -o avx2

Step 1 — Create the project folder and secrets

Keep everything for this app under /opt/anythingllm. The storage folder becomes the container's data directory. AnythingLLM also writes its own settings to a .env file inside that folder, which must exist as a file before the first start; otherwise Docker would create a directory in its place.

sudo mkdir -p /opt/anythingllm/storage
sudo chown -R $USER:$USER /opt/anythingllm
cd /opt/anythingllm
touch storage/.env

Next, create a second .env file next to the Compose file. Compose reads it and passes three secrets to the container: JWT_SECRET signs login sessions, while SIG_KEY and SIG_SALT are the signing values the example configuration asks you to set to random strings of at least 32 characters.

cat > .env <<EOF
JWT_SECRET=$(openssl rand -hex 32)
SIG_KEY=$(openssl rand -hex 32)
SIG_SALT=$(openssl rand -hex 32)
EOF
chmod 600 .env
sudo chown -R 1000:1000 storage

The container runs as the user with UID and GID 1000, so it must own storage. Do not use chmod -R 777 as a shortcut: the folder holds your database, documents and any API keys you enter in the app.

Step 2 — Write the Compose file

Create /opt/anythingllm/compose.yaml:

services:
  anythingllm:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    restart: unless-stopped
    ports:
      - "127.0.0.1:3001:3001"
    cap_add:
      - SYS_ADMIN
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - STORAGE_DIR=/app/server/storage
      - JWT_SECRET=${JWT_SECRET}
      - SIG_KEY=${SIG_KEY}
      - SIG_SALT=${SIG_SALT}
      - DISABLE_SWAGGER_DOCS=true
    volumes:
      - ./storage:/app/server/storage
      - ./storage/.env:/app/server/.env

What each part does:

• 127.0.0.1:3001:3001 publishes the web interface on the loopback address only. Docker bypasses ufw for published ports, so binding to localhost is what keeps port 3001 off the internet; Caddy will be the only public entry point.
• cap_add: SYS_ADMIN follows the official run command. The documentation calls it required for scraping web pages, because the scraper runs Chromium in a sandbox. If you will never add websites as documents, you can delete these two lines.
• extra_hosts maps host.docker.internal to the host, so the container can reach Ollama on the same server in Step 6.
• DISABLE_SWAGGER_DOCS=true turns off the /api/docs page, which the example configuration recommends for production.

Step 3 — Start AnythingLLM and check it

docker compose up -d
docker compose ps
docker compose logs --tail 50 anythingllm

docker compose ps should show the anythingllm container as running with 127.0.0.1:3001->3001/tcp. Then ask the app for its start page from the server itself:

curl -I http://127.0.0.1:3001

You should see HTTP/1.1 200 OK. If the container restarts in a loop, read the logs and check the troubleshooting section below.

Step 4 — Finish setup through an SSH tunnel

Right after the first start, anyone who can open AnythingLLM can configure it. Do the first-run setup over an SSH tunnel, so the app is never exposed without a password. On your own computer, run:

ssh -L 3001:127.0.0.1:3001 youruser@203.0.113.10

Keep that session open and browse to http://localhost:3001. The onboarding screens ask for your LLM provider, the embedding model (the built-in embedder is fine to start) and the vector database (keep LanceDB). You can change all of these later.

Then open the settings and turn on Enable multi-user mode. Enter a username and a strong password for the first admin account; AnythingLLM logs you out, and you sign in again with the new account. Three roles are available:

• Admin has full access to the whole system.
• Manager sees all workspaces and most settings, but cannot change the LLM, embedder or vector database.
• Default can only chat in the workspaces an admin or manager adds them to.

Step 5 — Publish AnythingLLM over HTTPS with Caddy

Add a site block for your domain to /etc/caddy/Caddyfile:

anythingllm.example.com {
    reverse_proxy 127.0.0.1:3001
}

Reload Caddy and test the public address:

sudo systemctl reload caddy
curl -I https://anythingllm.example.com

Caddy obtains the certificate on the first request and returns HTTP/2 200. It proxies WebSocket connections automatically and applies no read timeout by default, which suits AnythingLLM's agent sessions and long answers. If you prefer Nginx, follow Nginx with Certbot and forward the Upgrade and Connection headers, as the AnythingLLM cloud guide shows; Traefik works too.

Make sure the firewall only allows SSH and web traffic:

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

Step 6 — Connect Ollama on the same server

If you followed Install Ollama, Ollama listens on 127.0.0.1:11434, which a container cannot reach. AnythingLLM's documentation explains that localhost inside the container means the container itself. Make Ollama listen on all addresses with a systemd drop-in, as the Ollama FAQ describes:

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf <<'EOF'
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
EOF
sudo systemctl daemon-reload
sudo systemctl restart ollama
ss -tln | grep 11434

The last command should show Ollama on 0.0.0.0:11434 or *:11434. If override.conf already exists, add the Environment line to it instead of replacing the file.

ufw also blocks traffic from Docker networks to the host by default. Find the subnet of the Compose network and allow only that subnet to reach Ollama:

docker network inspect anythingllm_default | grep Subnet
sudo ufw allow from 172.18.0.0/16 to any port 11434 proto tcp
sudo ufw status

Replace 172.18.0.0/16 with the subnet the first command printed. This rule only protects Ollama while ufw is active with its default deny policy, so confirm that sudo ufw status verbose shows Status: active and deny (incoming), and that nc -vz your-server-ip 11434 from another machine fails. Now open AnythingLLM's LLM settings, choose Ollama and set the base URL to http://host.docker.internal:11434. The model list fills with the models you pulled with ollama pull; pick one and save. You can also switch the embedder to Ollama (for example with an embedding model such as nomic-embed-text), but the built-in embedder works without extra setup. For larger models, consider a GPU server.

Back up and restore

All state lives in /opt/anythingllm: the storage folder (SQLite database, documents, vector data, app settings) plus compose.yaml and the secrets in .env. Stop the container briefly so the database and vector files are consistent, then archive the folder:

sudo mkdir -p /opt/backups
cd /opt/anythingllm
docker compose stop
sudo tar czf /opt/backups/anythingllm-$(date +%F).tar.gz -C /opt anythingllm
docker compose start

To restore on the same or a new server with Docker installed, unpack the archive and start the stack:

sudo tar xzf /opt/backups/anythingllm-2026-10-09.tar.gz -C /opt
cd /opt/anythingllm
docker compose up -d

Restore with the same image version or a newer one, never an older one. The archive contains your secrets and every document, so encrypt it if it leaves the server, and copy backups off the server, for example to object storage or another machine.

Update AnythingLLM

Read the release notes first and take a backup. Then pull the new image and recreate the container; the data in storage is kept:

cd /opt/anythingllm
docker compose pull
docker compose up -d
docker image prune -f

If you pinned a version tag, change it in compose.yaml before docker compose pull. Check docker compose logs --tail 50 anythingllm afterwards; database migrations run during startup.

Troubleshooting

The container exits with Illegal instruction

The CPU does not expose AVX2, which the default LanceDB vector database needs on x86. Confirm with lscpu | grep -o avx2. Move to a server whose CPU exposes AVX2, or set a different vector database in AnythingLLM's configuration; those run as separate services you would have to host.

unable to open database file

The container cannot write to the storage folder, usually because its owner is not UID 1000. Fix the ownership and restart:

sudo chown -R 1000:1000 /opt/anythingllm/storage
docker compose restart

ECONNREFUSED to port 11434 or an empty Ollama model list

Ollama still listens on 127.0.0.1, or ufw blocks the Docker subnet. Check ss -tln | grep 11434 on the host, compare the subnet from docker network inspect anythingllm_default | grep Subnet with sudo ufw status, and make sure the base URL is http://host.docker.internal:11434, not localhost.

Website scraping fails with No usable sandbox

The scraper's Chromium needs the SYS_ADMIN capability. Add the cap_add lines from Step 2 back to compose.yaml and run docker compose up -d.

Caddy returns 502 Bad Gateway

The container is stopped or listens on another port. Run docker compose ps and curl -I http://127.0.0.1:3001; the site block must point at the same port that compose.yaml publishes.

Next steps

• Run your own models with Install Ollama, or add a chat front end with Open WebUI and Ollama.
• Learn more about the Compose file format in Docker Compose basics.
• Read the official AnythingLLM documentation for agents, the developer API and embeddable chat widgets.
• See server options for this app on the AnythingLLM hosting page.