Tigris has introduced a bundle API that lets you fetch thousands of objects from object storage in a single HTTP request, returning them as a streaming tar archive. This directly addresses the ML training bottleneck where dataloaders need thousands of small files per batch but object storage forces one GET per object. The post explains the latency math (4,000 sequential GETs at 30ms each = 120 seconds vs. one round trip), how the streaming tar format works server-side without buffering, error handling via skip/fail modes, limits (5,000 keys, 50 GB, 15-min timeout), and includes SDK examples for Python, Go, JavaScript, and cURL, plus a PyTorch IterableDataset integration example.
Table of contents
The problem: lots of little files How it works Why you'd actually want this Using it from the SDKs What happens when a key is missing The web console does it too Go forth and bundle 236 Impressions