Dagster Cloud: 5X Faster Deployments

This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).

Dagster Cloud's Serverless offering reduced deployment times from 3+ minutes to ~40 seconds (4-5x faster) by replacing Docker image builds with PEX-based builds. The approach splits code into a deps.pex (dependencies) and source.pex (user code), storing them in S3. On code changes without dependency updates, only the source.pex is rebuilt and uploaded, and existing warm containers are reused — skipping the costly Docker build and AWS Fargate cold-start steps. PEX provides isolation, determinism, and composability. Trade-offs include shared RAM/CPU across environments on the same container and limited support for packages without Linux wheels.

10m read timeFrom dagster.io
Post cover image
Table of contents
Building Docker imagesLaunching Docker containersOther constraintsRoundup of alternativesWhat is PEX?Fast DeploysTrade-offs and issuesGitHub workflows and pex