What changed
Sentence Transformers v6 introduces `MultiVectorEncoder`, a fourth first-class model type for multi-vector, ColBERT-style late-interaction retrieval. Existing PyLate checkpoints and Stanford ColBERT checkpoints can load directly, while colpali-engine visual-document models can use the same API. The library now covers encoding, MaxSim scoring, training losses, evaluators, token pooling and multimodal retrieval patterns that previously required separate specialized libraries. The release also raises major dependency floors to Transformers 5.x, PyTorch 2.2+ and huggingface-hub 1.x and makes some migration behavior intentionally stricter.
Why it matters
Late-interaction retrieval sits between cheap dense embeddings and expensive cross-encoders: it preserves token-level matching information while still allowing documents to be encoded offline. That can materially improve retrieval for exact identifiers, rare terms, multi-constraint queries and visual documents, but historically the tooling was fragmented. Putting the model type inside Sentence Transformers reduces the integration and training barrier for RAG and search builders who already use that ecosystem. It does not remove the operational trade-off: multi-vector indexes can be much larger than dense indexes, and Sentence Transformers still does not replace PyLate's PLAID indexing/retrieval layer, so production architecture remains more complex than swapping one encoder class.
MultiVectorEncoder unifies formerly separate retrieval stacks
The new model class can load PyLate and Stanford ColBERT checkpoints and use colpali-engine models for visual document retrieval. It exposes query/document encoding, MaxSim similarity, training losses and evaluators through the same Sentence Transformers patterns used by dense, sparse and reranker models. That turns late interaction from a specialist side ecosystem into a supported path in a widely used embedding library.
Late interaction preserves token-level evidence at an index cost
Instead of compressing a document into one vector, a multi-vector encoder keeps a vector per token and scores each query token against its best document-token match. The approach can preserve rare identifiers and multiple independent constraints that a single dense vector may blur. The storage cost is much higher before compression: the project’s Natural Questions example produces about 42× the float32 storage of a MiniLM dense index, although PLAID-style compression cuts that example substantially.
The migration absorbs modeling, not every production component
Existing PyLate checkpoints load into `MultiVectorEncoder`, but models saved from the new API are not loadable back into PyLate. More importantly, Sentence Transformers v6 has no equivalent for PyLate's PLAID indexing and retrieval layer; the migration guide says builders should keep using PyLate for indexing, currently in a separate environment because PyLate pins an older Sentence Transformers version. That boundary matters for teams expecting a one-package production stack.
Version 6 is also a dependency migration
The release requires Transformers 5.x, PyTorch 2.2+ and huggingface-hub 1.x, with additional minimum-version increases. Custom module classes now require `trust_remote_code=True` even for local model directories, and several smaller output and evaluator behaviors changed. Teams adopting `MultiVectorEncoder` should therefore separate retrieval experiments from a fleet-wide v6 upgrade until their pinned model and training environments pass compatibility tests.