The Architecture of Grab's Data Lake

Grab, a leading tech company in Southeast Asia, uses a data lake to manage its vast data, generated from services like ride-sharing and food delivery. The company uses Apache Avro with a Merge on Read strategy for high-throughput data, allowing efficient writes and periodic compaction to manage read costs. For low-throughput data, Grab uses Parquet with Copy on Write to ensure fast reads and data consistency. The post also discusses various data storage formats and their trade-offs in terms of readability, compression, and schema evolution.