Storage architecture
| Tier | Technology | Use Case |
|---|---|---|
| High-speed scratch | Local NVMe SSDs | Fast checkpointing, caching |
| Parallel file system | Lustre, GPFS/Spectrum Scale | Large datasets, shared training data |
| Object storage | S3-compatible | Model registry, archiving |
| Network-attached | NFS, CIFS | Home directories, tools |
Key protocols:
- GPUDirect Storage (GDS) — Data moves directly from storage to GPU memory, bypassing the CPU
- NVMe-oF — Extends NVMe across a network fabric (RDMA or TCP)