POQD: Performance-Oriented Query Decomposer for Multi-vector Retrieval

POQD is a framework that enhances retrieval accuracy by decomposing complex queries and large documents (images or text) into smaller, semantically meaningful concepts. This allows for a more granular and accurate matching process, moving beyond simple vector similarity to a "bag-of-concepts" retrieval model.

POQD benefits:

• 🚀 Enhanced Accuracy: Improves retrieval performance by matching on a more granular, conceptual level.
• ⚡ Flexible Retrieval: Supports standard, decomposed, and accelerated retrieval modes for both images and text.
• ⚙️ Optimized for Speed: Integrates clustering-based indexes for significant search speed-ups on large datasets.

---

What is POQD?

Traditional retrieval systems match a single query vector against a single document vector. This can be limiting for complex queries or multifaceted documents. POQD addresses this by breaking down both queries and documents into a set of conceptual sub-vectors.

This "decomposed" representation enables a more nuanced similarity search. Instead of a single monolithic comparison, POQD can identify multiple points of relevance between a query and a document, leading to more accurate results, especially when fine-grained details matter.

Getting Started

Get started by setting up the environment and preparing the datasets.

Prerequisites

• Python 3.10 or higher
• (Recommended) Access to a machine with a GPU for faster processing

Installation Steps

1. Clone the repository

git clone https://github.com/PKU-SDS-lab/POQD-ICML25.git
cd POQD-ICML25

2. Install Python dependencies.

pip install -r Decompose_retrieval/requirements.txt

pip install -r Decompose_retrieval/vllm_requirements.txt

[!NOTE] The environment of the main project conflicts with that of vllm. It is recommended to create two separate environments.

3. Prepare the data. To run the experiments, you need to download and structure the datasets correctly.

Running Experiments

The floder Decompose_retrieval contains the retrieval utils that input the decomposed sub-queries and return the Top-k candidates. The core file is your-dataset-name_utils.py, such as Decompose_retrieval/manyqa_utils.py.

Retrieval_utils

args = parser.parse_args(args=['--dataset_name', 'manyqa',
                                   '--data_path', '/data1/liuyaoyang/Papers/icml2025/images/',
                                   '--query_concept', 
                                   '--img_concept', 
                                   '--algebra_method', 'two'])

--dataset_name : The dataset name aligned with path name.
--data_path : The path of your dataset images. The program will get image file list under path: data_path/dataset_name/xxxxxx.jpg.
--query_concept : Control whether to decompose the query.
--img_concept : Control whether to decompose the corpus (image or text).
--algebra_method : Two for image retrieval and one for text retrieval. \

[!NOTE] The function get_eval_answer_llava is used in the query optimization stage.

Query Optimization

Before running the query optimization, please make sure you have run the vllm serve. And choose the appropriate utils in opro-para/opro/evaluation/eval_utils.py.
from opro.evaluation.text_rag_utils import * for text retrieval
from opro.evaluation.rag_utils import * for image retrieval \

The initial prompt is set in initial_instructions of opro-para/opro/optimization/optimize_instructions.py.

bash opro-para/opro/optimization/run.sh

[!IMPORTANT]
POQD is an ongoing research project. In its current state, it is not production-ready code.

Citation

If you use POQD in your research, please consider citing our paper:

@inproceedings{
  liu2025poqd,
  title={{POQD}: Performance-Oriented Query Decomposer for Multi-vector retrieval},
  author={Yaoyang Liu and Junlin Li and Yinjun Wu and zhen chen},
  booktitle={Forty-second International Conference on Machine Learning},
  year={2025},
  url={https://openreview.net/forum?id=oktw116wt2}
}