Hugging Face
Read post

The OlmoEarth Platform: Geospatial inference at planetary scale

Ai2 describes the engineering behind the OlmoEarth Platform, infrastructure for running Earth observation foundation models at planetary scale. The platform processes dozens of terabytes of satellite imagery per day at fractions of a penny per square kilometer by splitting jobs into three hardware-matched stages: CPU-based data acquisition, GPU inference, and CPU postprocessing. A recent North America wildfire-risk map used ~19,600 CPUs and ~994 GPUs in parallel, achieving a 155× speedup over serial compute. Key engineering challenges covered include managing satellite metadata across providers via a custom STAC index, windowed reads from cloud-optimized formats (COG, Zarr), idempotent task execution for fault tolerance, and overlap reconciliation for seamless output rasters. The roadmap includes automated scheduled runs, change-detection alerts, agentic interfaces, precomputed global embeddings, and multi-cloud deployment.

    #distributed-systems
Jul 28•9m read time•From huggingface.co
Post cover image
Table of contents
Why satellite inference is challengingThe right hardware for the right taskOne request, hundreds of workers, and thousands of processesFinding and fetching the right pixelsHandling failure at scaleWhere we're headed
96 Impressions
Hugging Face's image
Hugging Face

HuggingFace's platform is a resource for developers and researchers working in natural language proc...

639 Followers

•

2.2K Upvotes

Would you recommend this post?

Copy link
WhatsApp
Facebook
X
New Squad
  • © 2026 Daily Dev Ltd.
  • Guidelines
  • Explore
  • Tags
  • Sources
  • Squads
  • Leaderboard