The article discusses the development of hybrid search in Meilisearch, combining keyword and semantic search algorithms. It explores the optimization of storage and search for high-dimensional embeddings by storing them on disk. The use of Spotify's Hyperplane Trees algorithm and the challenges of porting it to Rust are also highlighted. The article further mentions the use of LMDB for key-value storage and upcoming challenges with multithreading and incremental indexing.
Table of contents
Uniting Keyword and Semantic SearchOptimizing Storage and Search for High-Dimension EmbeddingsSpotify's Hyperplane Trees for Efficient ANNsAdapting Annoy to LMDBOptimizing Vector Handling with SIMDUpcoming Challenges: Multithreading, Microsoft's Filtered-DiskANN, and Incremental Indexing1 Impression