↖️ Vector-ICL ↗️

Vector-ICL: In-context Learning with Continuous Vector Representations (Zhuang et al., ICLR 2025).

Vector-ICL is a way of conducting in-context learning with data of any forms.
1. Encode data into embedding vector
2. Project the vector into LLMs' representation space
3. Build the context with the projected embeddings

And we only need to train a light-weight projector (a linear matrix works most of the time!).
(a) Pretraining with next token prediction (such as language modelling objective) enables Vector-ICL,
(b) task finetunes further improves LLMs' ability to conduct Vector-ICL.

We show that Vector-ICL works for a wide range of modalities and tasks, surpassing few-shot ICL and domain-specific models and tunings.

Example Usage

We show examples of using Vector-ICL at text-based tasks over here:

For cross-modal Vector-ICL, the scripts are provided in this folder with training and inferencing pipelines.

Questions?

If you have any questions related to the code or the paper, feel free to reach out to us at y5zhuang@ucsd.edu.

Citation

If you find our paper and code useful, please cite us:

@article{zhuang2024vector,
  title={Vector-ICL: In-context Learning with Continuous Vector Representations},
  author={Zhuang, Yufan and Singh, Chandan and Liu, Liyuan and Shang, Jingbo and Gao, Jianfeng},
  journal={arXiv preprint arXiv:2410.05629},
  year={2024}
}