portainer-core/docs/gpu-docker-config.md

GPU Docker Configuration - Working Setup

Successfully configured: 2025-11-11 System: tower-of-joy GPU: NVIDIA GeForce RTX 2080 Ti Driver: 470.256.02

Problem Encountered

Issue: nvidia-container-toolkit 1.18.0 has a compatibility bug with NVIDIA driver 470.x

• Version 1.18 changed default mode from "legacy" to "CDI"
• Legacy mode detection fails with older drivers
• Error: libnvidia-ml.so.1: cannot open shared object file

Working Solution

Downgrade to nvidia-container-toolkit 1.17.9-1

All nvidia-container packages must be downgraded together:

• nvidia-container-toolkit
• nvidia-container-toolkit-base
• libnvidia-container-tools
• libnvidia-container1

Installation Commands

# Remove all nvidia-container packages
sudo apt-get remove -y nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1

# Clean up
sudo apt-get autoremove -y

# Install all packages at version 1.17.9-1
sudo apt-get install -y \
  nvidia-container-toolkit=1.17.9-1 \
  nvidia-container-toolkit-base=1.17.9-1 \
  libnvidia-container-tools=1.17.9-1 \
  libnvidia-container1=1.17.9-1

# Hold packages to prevent auto-upgrade
sudo apt-mark hold nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1

# Configure Docker runtime
sudo nvidia-ctk runtime configure --runtime=docker --config=/etc/docker/daemon.json

# Rebuild library cache
sudo ldconfig

# Restart Docker
sudo systemctl restart docker

Verification

# Test GPU access
sudo docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi

Expected output: nvidia-smi showing RTX 2080 Ti

Current Configuration

/etc/docker/daemon.json

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

Installed Versions

libnvidia-container-tools     1.17.9-1
libnvidia-container1          1.17.9-1
nvidia-container-toolkit      1.17.9-1
nvidia-container-toolkit-base 1.17.9-1

All packages are held to prevent automatic upgrade to 1.18.x

Important Notes

1. Do NOT upgrade nvidia-container-toolkit to 1.18.x - it breaks compatibility with driver 470
2. If you run apt upgrade, the packages are held and won't upgrade
3. To check held packages: apt-mark showhold
4. To unhold (not recommended): sudo apt-mark unhold nvidia-container-toolkit

For Future Reference

If you need to update the NVIDIA driver:

1. Driver 470.x is compatible with CUDA 11.4
2. Driver 525+ is compatible with CUDA 12.x
3. After driver update, may be able to use newer nvidia-container-toolkit

Testing GPU in Containers

Quick Test

docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi

Test with Ollama

docker run --rm --gpus all ollama/ollama nvidia-smi

Test with Jellyfin (after deployment)

Check Jellyfin Dashboard → Playback → Transcoding for NVIDIA NVENC option

Troubleshooting

If GPU stops working after system update:

# Check if packages were upgraded
dpkg -l | grep nvidia-container

# If upgraded to 1.18.x, re-run downgrade script:
sudo bash /home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh

References

• NVIDIA Container Toolkit: https://github.com/NVIDIA/nvidia-container-toolkit
• Driver 470 Release Notes: https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-470-256-02/
• Issue with 1.18.0: https://github.com/NVIDIA/nvidia-container-toolkit/issues
• Our fix script: /home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh