Training-data pipelines: profile fetch, preprocessing and transfer separately

Tarun Sunkaraneni traces a multimodal training pipeline from serial image fetches through concurrent preprocessing, prefetch queues and object references. Each change exposes another bottleneck: waiting for data, copying large arrays, then network pressure as workers scale. Prefetching complicates checkpoint recovery, and spreading workers can trade locality for network capacity. Ray’s documentation limits zero-copy NumPy reads to workers on the same node; references do not eliminate cross-node transfer. The talk’s utilization figures describe its experiment, not an expected gain for other workloads.