config(ollama): reduce VRAM usage for GPU sharing with other services

Change keep-alive from infinite to 5m and max loaded models from 2 to 1,
freeing VRAM for Stable Audio, TRELLIS, and other GPU services.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-26 10:34:09 +01:00
co-authored by Claude Opus 4.6
parent ec8c3f5925
commit dd141099e1
+4 -4
View File
@@ -24,10 +24,10 @@ services:
- NVIDIA_DRIVER_CAPABILITIES=all
# Process requests sequentially to avoid batch overflow panics
- OLLAMA_NUM_PARALLEL=1
# Keep models loaded in VRAM (don't unload after idle)
- OLLAMA_KEEP_ALIVE=-1
# Load multiple models concurrently (mistral-nemo + nomic-embed-text)
- OLLAMA_MAX_LOADED_MODELS=2
# Unload LLMs after 5 minutes idle (keeps VRAM free for other services)
- OLLAMA_KEEP_ALIVE=5m
# Only keep one model loaded at a time (embedding model stays, LLMs swap)
- OLLAMA_MAX_LOADED_MODELS=1
healthcheck:
test: ["CMD-SHELL", "curl -fSs http://localhost:11434/api/tags || exit 1"]
interval: 30s