A walkthrough of the data acquisition and preparation phase for building a product classification API using Amazon product metadata. Covers parsing 9.4 million products from gzipped JSON, loading into pandas DataFrames, extracting and formatting category paths, and applying multiple filtering steps (removing missing data, excluding non-title-classifiable categories like Books and Movies, keeping only deepest category paths, and dropping categories with fewer than 10 products) to arrive at a clean 4.59 million product dataset ready for model training.

8m read timeFrom eugeneyan.com
Post cover image
Table of contents
Where did I get the product data from?How does the product metadata look like?Converting the JSON to CSV formatFormatting and cleaning the category dataConverting category data into category path stringsExclude data where title or category is missingExclude certain categoriesExclude non-deepest/non-narrowest categoriesExclude categories that have too few productsAnd we’re done!