Binary quantization with reranking (HNSW+BQ) lets pgvector scale to hundreds of millions or billions of vectors on Amazon Aurora PostgreSQL by compressing float32 vectors to single bits, shrinking a 100M-vector index from 367 GB to ~38 GB so it fits in buffer cache. Benchmarks across LAION 100M, OpenAI 5M, and Cohere 10M datasets show HNSW+BQ often matches or beats disk-based ANN with much faster index builds (1.1 hrs vs 31.5 hrs at 100M scale), though effectiveness is distribution-dependent and fails badly on clustered embeddings like Cohere's. Sizing tables, recall validation steps, and NVMe-tiered scaling guidance (up to ~20B vectors theoretically) are provided, along with guidance on when to prefer full-precision HNSW or halfvec instead.
Table of contents
PrerequisitesHow binary quantization with reranking worksBenchmark setupPerformance characteristics across datasetsSizing guidance: how many vectors can you store?When to use each approachDistance metric guidanceFiltered queriesGetting started: implementing HNSW+BQ on Amazon Aurora PostgreSQLValidation before production deploymentOperational considerationsClean up resourcesConclusionAbout the authorsQuestions this post answers
How much can binary quantization shrink a pgvector HNSW index compared to full precision?
Binary quantization compresses each float32 dimension to a single bit, reducing a 768-dimension vector from 3,072 bytes to 96 bytes, a 32x compression ratio. For a 100-million-vector dataset at 768 dimensions, this shrinks the HNSW index from 367 GB down to approximately 38 GB, letting it fit in the buffer cache of standard instances instead of requiring larger memory-heavy instance types. Track practical scaling techniques like this on daily.dev when sizing vector search infrastructure.
Why does binary quantization fail on Cohere embed-english-v3.0 embeddings?
Binary quantization thresholds each vector dimension at zero, but Cohere embed-english-v3.0 embeddings have a clustered distribution where many dimensions concentrate near zero without balanced positive/negative spread. This means the sign bit carries little discriminative information, causing distinct vectors to map to similar bit patterns, capping recall at around 0.93 even with heavy reranking (3,000 candidates) and collapsing throughput to 16 QPS. daily.dev helps engineers compare embedding model quirks before committing to a quantization strategy.
What pgvector version is required for iterative HNSW scans with reranking depths beyond 1000 candidates?
pgvector 0.8.0 introduces iterative index scans via the hnsw.iterative_scan setting, which are required when reranking depths (quantized_fetch_limit) exceed 1,000 candidates. This version ships with Aurora PostgreSQL 16.8 and later, as well as Aurora PostgreSQL 17.x, and also enables relaxed_order scans that help post-filtered multi-tenant queries return enough results after filtering. Developers tuning Postgres vector search track version-specific requirements like this on daily.dev.