Kubernetes core concepts

Kubernetes (K8s) is the primary container orchestration system for AI workloads. NVIDIA exposes GPUs as extended resources via the Device Plugin.

Key objects:

Object	Description
Pod	Smallest deployable unit; one or more containers
Deployment	Manages a ReplicaSet; desired state for pods
Service	Stable network endpoint for pods
Namespace	Virtual cluster for resource isolation
Node	Physical or virtual machine in the cluster
DaemonSet	Runs one pod per node (used for GPU drivers)
Job	Run-to-completion workload
CronJob	Scheduled Job
PersistentVolumeClaim (PVC)	Pod's request for storage

GPU pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
spec:
  containers:
  - name: cuda-container
    image: nvcr.io/nvidia/cuda:12.0-base
    resources:
      limits:
        nvidia.com/gpu: 1