[CVPR 2026] VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection
Python
22
12 commits
updated Aug 12, 2026
This repository is the official implementation of VLM4RSDet, is a novel collaborative training framework that leverages vision-language models to enhance traditional closed-set remote sensing object detectors without introducing any additional deployment overhead.
VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection
Shuohao Shi, Qiang Fang*, Xin Xu* (*Corresponding Author)
Note that this repository is based on the MMDetection. Assume that your environment has satisfied the above requirements, please follow the following steps for installation.
git clone https://github.com/cszzshi/VLM4RSDet.git
cd VLM4RSDet
pip install -v -e .
To verify whether MMDetection is installed correctly, we provide some sample codes to run an inference demo.
Step 1. We need to download config and checkpoint files.
mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest .
Step 2. Verify the inference demo.
python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu
You will see a new image demo.jpg on your ./outputs/vis folder, where bounding boxes are plotted on cars, benches, etc.
🤗Faster R-CNN w/ VLM4RSDet on VisDrone.
The official download link of VisDrone dataset.
trainset (1.44 GB): Baidu Yun | Google Drive
valset (0.07 GB): Baidu Yun | Google Drive
Test the trained weight using 4 GPUs.
CUDA_VISIBLE_DEVICES=0,1,2,3 tools/dist_test.sh configs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone.py work_dirs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone/epoch_12.pth 4
Test the trained weight using a single GPU.
python tools/test.py configs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone.py work_dirs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone/epoch_12.pth
Welcome to cite this project in your research.
@InProceedings{Shi_2026_CVPR,
author = {Shi, Shuohao and Fang, Qiang and Xu, Xin},
title = {VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2026},
pages = {18450-18460}
}
If you are interested in our project or have any questions, please feel free to contact us at any time via email at [email protected] or [email protected].
This project is licensed under the Apache License (Version 2.0).
This work wouldn't be possible without the incredible open-source code of these projects.
[CVPR 2026] VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection
Python
22
12 commits
updated Aug 12, 2026
This repository is the official implementation of VLM4RSDet, is a novel collaborative training framework that leverages vision-language models to enhance traditional closed-set remote sensing object detectors without introducing any additional deployment overhead.
VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection
Shuohao Shi, Qiang Fang*, Xin Xu* (*Corresponding Author)
Note that this repository is based on the MMDetection. Assume that your environment has satisfied the above requirements, please follow the following steps for installation.
git clone https://github.com/cszzshi/VLM4RSDet.git
cd VLM4RSDet
pip install -v -e .
To verify whether MMDetection is installed correctly, we provide some sample codes to run an inference demo.
Step 1. We need to download config and checkpoint files.
mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest .
Step 2. Verify the inference demo.
python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu
You will see a new image demo.jpg on your ./outputs/vis folder, where bounding boxes are plotted on cars, benches, etc.
🤗Faster R-CNN w/ VLM4RSDet on VisDrone.
The official download link of VisDrone dataset.
trainset (1.44 GB): Baidu Yun | Google Drive
valset (0.07 GB): Baidu Yun | Google Drive
Test the trained weight using 4 GPUs.
CUDA_VISIBLE_DEVICES=0,1,2,3 tools/dist_test.sh configs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone.py work_dirs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone/epoch_12.pth 4
Test the trained weight using a single GPU.
python tools/test.py configs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone.py work_dirs/VLM4RSDet/visdrone/faster-rcnn_r50_fpn_1x_visdrone/epoch_12.pth
Welcome to cite this project in your research.
@InProceedings{Shi_2026_CVPR,
author = {Shi, Shuohao and Fang, Qiang and Xu, Xin},
title = {VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2026},
pages = {18450-18460}
}
If you are interested in our project or have any questions, please feel free to contact us at any time via email at [email protected] or [email protected].
This project is licensed under the Apache License (Version 2.0).
This work wouldn't be possible without the incredible open-source code of these projects.