Three performance optimizations to Manticore Search's HNSW-based KNN engine deliver up to 29% throughput improvement at high k values, with 20%+ gains under concurrent load. The changes include: compile-time distance function specialization via C++ templates (eliminating indirect call overhead), a 2-pass neighbor processing loop that issues CPU prefetch hints before distance computation, and batched distance scoring that reuses the query vector across two candidates per SIMD iteration. Additionally, AVX-512 support was added, processing 16 floats per iteration vs. 8 with AVX2, with automatic CPU detection at startup. No API changes, index rebuilds, or configuration are required — improvements ship in Manticore Search 27.1.5.
Table of contents
Compile-time distance function specialization2-pass neighbor processingBatched distance computationAVX-512 supportBenchmark resultsWhat this means for youFurther reading882 Impressions