The post explores the use of text embeddings to map and analyze Hacker News posts and comments. The author discusses the power and applicability of embeddings for search, recommendations, and analysis. They also provide a demo of the application and invite others to explore and build upon the data and tools provided.
Table of contents
Fetching items from HNGenerating embeddingsAdding additional context by crawing the webEmbeddings, attempt twoUMAPCosine similiarityBuilding a mapPreparing the map dataBuilding the web appAdding some visual appeal and guidanceCitiesPushing things to the edgeTesting out the searchAutomatic virtual subcommunitiesAnalyzing the entire datasetBig data number crunching with a GPUDemoWhat's next23 Impressions