OpenAI handles petabytes of logs daily from LLM training and inference workloads. Logs are routed through a load balancer to 90 shards, with recent data kept on disk and all data in blob storage. Each shard has two replicas for redundancy. During a viral ChatGPT image-generation event in March, log volume spiked 50%, overwhelming systems. After aggressive sampling and adding a third replica failed to fully resolve the issue, engineers traced the bottleneck to a bloom filter using a multiplication operation. Replacing it with a bit-shift operation cut CPU usage by 40%, stabilizing the system.
•1m watch time
2 Impressions