An ML training job was exhausting worker memory during Snowflake data fetches, peaking at 88.8 GiB RSS before training even started. The root cause was the shared Snowflake-to-pandas path materializing results as Python objects and SQLAlchemy rows before building the final DataFrame — consuming roughly 3x the final DataFrame size at peak. By adding an Apache Arrow path for supported reads, median peak RSS growth dropped 60–79% across three query shapes, and peak Python allocations fell 69–87%. In one large workflow, the lower fetch peak doubled the training-data window on the same cluster size. The implementation required normalizing dtype differences between Arrow and SQLAlchemy (integer widths, aggregate representations, identifier casing) and maintaining a SQLAlchemy fallback for unsupported cases. Because all ML jobs shared the same data-access layer, no query rewrites were needed.
