Pooling-Aware Training for Multi-Vector Embeddings

Hugging Face researchers have demonstrated a method to reduce storage costs for multi-vector embedding models by up to 75% without significant loss in retrieval performance. The approach, called pooling-aware training, integrates hierarchical token pooling directly into the model's optimization loop, teaching the encoder that its representations must remain effective even after compression. In tests using the NanoBEIR benchmark, this method retained 97.8% of the original model's nDCG@10 score while reducing document vectors by a factor of four. This allows practitioners to deploy more expressive search models within tighter memory and infrastructure budgets.