Common Docker GPU issues

Error: "Failed to initialize NVML: No such file or directory"

NVIDIA Container Toolkit not installed or not configured. Fix:

# Install Container Toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Error: "Failed to initialize NVML: Unknown Error" (container loses GPU access)

Happens after systemctl daemon-reload when using cgroup v2 with systemd driver. Fixes:

Option 1 — Use cgroupfs driver:

# /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=cgroupfs"]
}
sudo systemctl restart docker

Option 2 — Use CDI (Container Device Interface) to inject devices — most robust solution.

Error: "NVML: Insufficient Permissions" on RHEL/SELinux

docker run --gpus all --security-opt=label=disable <image>

Error: Permission denied with nvidia-docker wrapper on SELinux

Use the runtime flag directly instead of the wrapper:

sudo docker run --gpus=all --runtime=nvidia --rm <image> nvidia-smi

Or create a local SELinux policy:

ausearch -c 'nvidia-docker' --raw | audit2allow -M my-nvidiadocker
semodule -X 300 -i my-nvidiadocker.pp

Conflicting apt repository entries (Ubuntu/Debian):

grep "nvidia.github.io" /etc/apt/sources.list.d/*

# Find conflicting files
grep -l "nvidia.github.io" /etc/apt/sources.list.d/* | \
  grep -vE "/nvidia-container-toolkit.list$"

# Delete conflicting files, then retry apt-get update