---
title: "How did I convert the 33 GB Dataset into a 3 GB file Using Pandas?"
url: https://daily.dev/posts/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas--jsmdqvwqx
source_url: https://medium.com/aatomz-research/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas-b21d8da205c0?source=rss------python-5
type: article
source: "Medium"
published: 2022-09-19T21:21:37.812Z
updated: 2022-09-19T21:21:37.811Z
tags: ["data-science", "pandas", "data-analysis", "linkedin"]
reading_time: 3
upvotes: 18
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# How did I convert the 33 GB Dataset into a 3 GB file Using Pandas?

**[Medium](https://daily.dev/sources/medium_js)** · 3 min read · 18 upvotes · 0 comments

## Summary

A dataset may contain thousands to millions of rows. We can load the smaller dataset quickly, but when the dataset gets bigger, our system may not be able to run the large memory files. So I thought to share how I converted the 33GB data file into a 3 GB file using Pandas.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://medium.com/aatomz-research/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas-b21d8da205c0?source=rss------python-5>

## Similar posts on daily.dev

- [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments
- [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments
- [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments
- [CNCF Unveils Schedule for KubeCon \+ CloudNativeCon Europe 2026](https://daily.dev/posts/cncf-unveils-schedule-for-kubecon-cloudnativecon-europe-2026-ikhcoa5cb) · CNCF · 2 upvotes · 0 comments
- [CNCF Debuts KubeCon \+ CloudNativeCon Japan 2026 Schedule](https://daily.dev/posts/cncf-debuts-kubecon-cloudnativecon-japan-2026-schedule-xp5pyudub) · CNCF · 1 upvotes · 0 comments

---

Tags: [#data-science](https://daily.dev/tags/data-science), [#pandas](https://daily.dev/tags/pandas), [#data-analysis](https://daily.dev/tags/data-analysis), [#linkedin](https://daily.dev/tags/linkedin)

[View this post on daily.dev](https://daily.dev/posts/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas--jsmdqvwqx)