Anchor Token Guided Prompt Learning Methods for VLMs.

This repo contains a series of anchor token-guided prompt learning methods for Vision-Language Models (CLIP):

• [Arxiv] AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learning.
  Zheng Li, Yibing Song, Xin Zhang, Lei Luo, Xiang Li, Jian Yang.
  [Paper]

• [ICCV 25] Advancing Textual Prompt Learning with Anchored Attributes.
  Zheng Li, Yibing Song, Ming-Ming Cheng, Xiang Li, Jian Yang.
  [Paper] [Project Page] [Poster] [PPT] [中文解读] [中文翻译]

💡 Helpful Resources:

• If you are interested in prompt learning and want to know more about related work, we also maintain a list of awesome papers for your reference.
• If you attempt to reproduce the results of this implementation on the existing 15 datasets, the links to those datasets may be broken and unusable. For your convenience, we have provided 14 datasets (excluding ImageNet) in the HuggingFace platform. [Download_Links]

🔥 Some other works may interest you:

• [CVPR 24] PromptKD: Unsupervised Prompt Distillation for Vision-Language Models
  Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, Jian Yang.
  [Paper] [Code] [Project Page] [Poster] [中文论文解读] [视频解读] [中文翻译].
  PromptKD is a simple and effective prompt-driven unsupervised distillation framework for VLMs (e.g., CLIP), with state-of-the-art performance.

🧪 Experimental Comparison

🗒 [Arxiv] AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learning.

Abstract

Existing prompt learning methods, which are built upon CLIP models, leverage textual tokens as anchors to guide the learnable soft tokens. This guidance improves CLIP generalizations. However, these anchors—static in both value and position—lack cross-task and stage-adaptive flexibility.

To address this limitation, we propose AnchorOPT, a dynamic anchor-based prompt learning framework. Specifically, AnchorOPT introduces dynamism in two key dimensions: (i) anchor values eschew handcrafted explicit textual tokens (e.g., "shape", "color"), instead learning dynamically from task-specific data; and (ii) the positional relationship between anchor and soft tokens is no longer fixed but adaptively optimized via a learnable position matrix conditioned on the training stage and task context.

Training occurs in two stages: we first learn the anchor tokens, then freeze and transfer them to the second stage for optimization of soft tokens and the position matrix.

Framework

Fig 1. Architectural comparison among classic prompt learning, ATPrompt, and our proposed AnchorOPT.

🚀 Running

Please see the [AnchorOPT Reproduction Guide].

🗒 [ICCV 25] ATPrompt: Advancing Textual Prompt Learning with Anchored Attributes

Abstract

In this work, we introduce an attribute-anchored textual prompt learning method for vision-language models, named ATPrompt.

This method extends the learning space of soft prompts from the original one-dimensional category level to the multi-dimensional attribute level by incorporating multiple universal attribute tokens into the learnable soft prompts.

Guided by these attributes, soft tokens acquire not only category-specific but also attribute-related general representations during training, thereby enhancing the alignment between images and unknown categories compared to the original method.

Framework

Fig 2. Architectural comparison among vanilla CLIP, classic prompt learning, and our proposed attribute-anchored prompt learning.

🚀 Running

Please see the [ATPrompt Reproduction Guide].

✉️ Contact

If you have any questions, you can submit an issue on GitHub, or contact me by email (zhengli97[at]foxmail.com).

⭐ Citation

If you find our paper or repo helpful for your research, please consider citing the following paper and giving this repo a star. Thank you!

@article{li2025anchoropt,
  title={AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learning},
  author={Li, Zheng and Song, Yibing and Zhang, Xin and Luo, Lei and Li, Xiang and Yang, Jian},
  journal={arXiv preprint arXiv:2511.21188},
  year={2025}
}

@inproceedings{li2025advancing,
  title={Advancing textual prompt learning with anchored attributes},
  author={Li, Zheng and Song, Yibing and Cheng, Ming-Ming and Li, Xiang and Yang, Jian},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={3618--3627},
  year={2025}
}

@inproceedings{li2024promptkd,
  title={Promptkd: Unsupervised prompt distillation for vision-language models},
  author={Li, Zheng and Li, Xiang and Fu, Xinyi and Zhang, Xin and Wang, Weiqiang and Chen, Shuo and Yang, Jian},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={26617--26626},
  year={2024}
}