fix(stacks): configure ollama for stability and persistent model loading
- OLLAMA_NUM_PARALLEL=1: Process requests sequentially to avoid batch overflow panics when multiple services call embeddings concurrently - OLLAMA_KEEP_ALIVE=-1: Keep models loaded in VRAM indefinitely - OLLAMA_MAX_LOADED_MODELS=2: Allow both mistral-nemo and nomic-embed-text to stay loaded simultaneously 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,12 @@ services:
|
|||||||
- TZ=Europe/Amsterdam
|
- TZ=Europe/Amsterdam
|
||||||
- NVIDIA_VISIBLE_DEVICES=all
|
- NVIDIA_VISIBLE_DEVICES=all
|
||||||
- NVIDIA_DRIVER_CAPABILITIES=all
|
- NVIDIA_DRIVER_CAPABILITIES=all
|
||||||
|
# Process requests sequentially to avoid batch overflow panics
|
||||||
|
- OLLAMA_NUM_PARALLEL=1
|
||||||
|
# Keep models loaded in VRAM (don't unload after idle)
|
||||||
|
- OLLAMA_KEEP_ALIVE=-1
|
||||||
|
# Load multiple models concurrently (mistral-nemo + nomic-embed-text)
|
||||||
|
- OLLAMA_MAX_LOADED_MODELS=2
|
||||||
deploy:
|
deploy:
|
||||||
resources:
|
resources:
|
||||||
limits:
|
limits:
|
||||||
|
|||||||
Reference in New Issue
Block a user