Sentence Transformers v6.0 adds a fourth model type, MultiVectorEncoder, for ColBERT-style late interaction retrieval. It loads PyLate, Stanford-NLP ColBERT, and colpali-engine checkpoints through the same API used for dense, sparse, and reranker models. The post covers multi-vector theory (MaxSim scoring, token-level embeddings), installation, loading various checkpoint formats, encoding, scoring, semantic search, retrieve-and-rerank patterns, indexing with Qdrant/Weaviate/Vespa/fast-plaid, visual/audio/video document retrieval, interpretability, token pooling for index compression, and inference speedups via Flash Attention and quantization.
Table of contents
Table of ContentsWhat are Multi-Vector Models?InstallationLoading a ModelEncoding Queries and DocumentsScoring with MaxSimSemantic SearchRetrieve and RerankIndexingVisual Document RetrievalAudio RetrievalVideo RetrievalInterpretabilityToken PoolingSpeeding Up InferenceEvaluating a ModelComing from PyLate or colpali-engineSupported ModelsAcknowledgementsAdditional ResourcesQuestions this post answers
How do I load a ColBERT-style late interaction model in Sentence Transformers?
Sentence Transformers v6.0 introduces a MultiVectorEncoder class that loads any PyLate checkpoint, any Stanford-NLP ColBERT checkpoint, or a ColPali-family visual retrieval model with the same API used for dense, sparse, and reranker models. Load it with MultiVectorEncoder('lightonai/LateOn') or similar, then use encode_query and encode_document to get per-token embeddings. daily.dev surfaces practical guides like this for developers wiring up retrieval stacks.
What are the minimum dependency versions required to use Sentence Transformers v6.0?
Sentence Transformers v6.0 requires transformers v5.x, torch 2.2 or newer, and huggingface-hub v1.x. Anyone pinned to lower versions of those libraries needs to plan an upgrade first, and a migration guide documents the full list of breaking changes introduced alongside the new MultiVectorEncoder model type. Developers planning library upgrades can track breaking changes like this one on daily.dev.
How much extra storage does a ColBERT-style multi-vector index need compared to a dense embedding index?
Encoding 4,874 Natural Questions passages with lightonai/LateOn produced 608,414 token vectors (about 124.8 per passage), roughly 42x the storage of a dense all-MiniLM-L6-v2 index, or about 62 KiB per passage in float32. Compressed with a PLAID-based index like fast-plaid, the same vectors take just 92 MB, comparable to a dense index from a large model like Qwen3-Embedding-8B. Weighing retrieval quality against index cost is easier when daily.dev keeps you current on tools like this.