fix(stacks): configure ollama for stability and persistent model loading
- OLLAMA_NUM_PARALLEL=1: Process requests sequentially to avoid batch overflow panics when multiple services call embeddings concurrently - OLLAMA_KEEP_ALIVE=-1: Keep models loaded in VRAM indefinitely - OLLAMA_MAX_LOADED_MODELS=2: Allow both mistral-nemo and nomic-embed-text to stay loaded simultaneously 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,12 @@ services:
|
||||
- TZ=Europe/Amsterdam
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
- NVIDIA_DRIVER_CAPABILITIES=all
|
||||
# Process requests sequentially to avoid batch overflow panics
|
||||
- OLLAMA_NUM_PARALLEL=1
|
||||
# Keep models loaded in VRAM (don't unload after idle)
|
||||
- OLLAMA_KEEP_ALIVE=-1
|
||||
# Load multiple models concurrently (mistral-nemo + nomic-embed-text)
|
||||
- OLLAMA_MAX_LOADED_MODELS=2
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
|
||||
Reference in New Issue
Block a user