config(ollama): reduce VRAM usage for GPU sharing with other services
Change keep-alive from infinite to 5m and max loaded models from 2 to 1, freeing VRAM for Stable Audio, TRELLIS, and other GPU services. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+4
-4
@@ -24,10 +24,10 @@ services:
|
||||
- NVIDIA_DRIVER_CAPABILITIES=all
|
||||
# Process requests sequentially to avoid batch overflow panics
|
||||
- OLLAMA_NUM_PARALLEL=1
|
||||
# Keep models loaded in VRAM (don't unload after idle)
|
||||
- OLLAMA_KEEP_ALIVE=-1
|
||||
# Load multiple models concurrently (mistral-nemo + nomic-embed-text)
|
||||
- OLLAMA_MAX_LOADED_MODELS=2
|
||||
# Unload LLMs after 5 minutes idle (keeps VRAM free for other services)
|
||||
- OLLAMA_KEEP_ALIVE=5m
|
||||
# Only keep one model loaded at a time (embedding model stays, LLMs swap)
|
||||
- OLLAMA_MAX_LOADED_MODELS=1
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -fSs http://localhost:11434/api/tags || exit 1"]
|
||||
interval: 30s
|
||||
|
||||
Reference in New Issue
Block a user