TinyRS-R1: Compact Vision Language Model for Remote Sensing

IEEE Geoscience and Remote Sensing Letters

This paper presents TinyRS, the first 2B-parameter vision language models (VLMs) optimized for RS, and TinyRS-R1, its reasoning-augmented variant. Based on Qwen2-VL-2B, TinyRS is trained via a four-stage pipeline: pretraining on million-scale satellite images, instruction tuning, fine-tuning with chain-of-thought (CoT) annotations from a new reasoning dataset, and group relative policy optimization (GRPO)-based alignment. TinyRS-R1 matches or surpasses recent 7B RS models in classification, visual question answering (VQA), grounding, and open-ended QA—while using one third of the memory and latency. CoT reasoning improves grounding and scene understanding, while TinyRS excels at concise, low-latency VQA. TinyRS-R1 is the first domain-specialized small VLM with GRPO-aligned CoT reasoning for general-purpose RS.

Paper (arXiv) | Paper (IEEExplore)

Dataset

Models

Installation

For best reproducibility, we suggest you to generate three different environments, one each for finetuning, GRPO training and evaluation.

For SFT:

git clone https://github.com/aybora/TinyRS
cd ~/TinyRS/grpo
conda env create -f environment.yaml
conda activate sft
pip install qwen-vl-utils
pip install flash-attn --no-build-isolation

For GRPO:

git clone https://github.com/aybora/TinyRS
conda create -n grpo python=3.10 -y
conda activate grpo
cd ~/TinyRS/grpo
pip3 install -e ".[dev]"
pip3 install wandb==0.18.3
pip3 install flash-attn==2.7.4.post1 --no-build-isolation

For Evaluation:

conda create -n eval python==3.10 -y
conda activate eval
cd ~/TinyRS/eval 
bash basic_env_setup.sh

Training (SFT)

For pre-training, SFT and CoT SFT, you may follow the sample script below, which works on one node with 4 x H100s or A100s. Sample script assumes you are doing a pretraining, change data_path and image_folder otherwise.

For SFT, change model name to aybora/Qwen2-VL-TinyRS-PRETRAIN, image_folder to ../VHM_dataset_sft, data_path to ../VHM_dataset_sft/list_sft_oversampled.json

For CoT SFT, change model name to aybora/Qwen2-VL-TinyRS, image_folder to ../VHM_dataset_sft, data_path to ../VHM_dataset_sft/list_sft_reasoning_oversampled.json

conda activate sft
cd ./TinyRS/sft/

MODEL_NAME="Qwen/Qwen2-VL-2B-Instruct"

GLOBAL_BATCH_SIZE=128
BATCH_PER_DEVICE=16
NUM_DEVICES=4
GRAD_ACCUM_STEPS=$((GLOBAL_BATCH_SIZE / (BATCH_PER_DEVICE * NUM_DEVICES)))

export PYTHONPATH=src:$PYTHONPATH

deepspeed --master_port 29400 src/training/train.py \
    --deepspeed scripts/zero3_offload.json \
    --model_id $MODEL_NAME \
    --data_path ../VHM_VersaD/list_pretrain_corrected.json \
    --image_folder ../VHM_VersaD \
    --remove_unused_columns False \
    --freeze_vision_tower False \
    --freeze_llm False \
    --tune_merger True \
    --bf16 True \
    --fp16 False \
    --disable_flash_attn2 False \
    --output_dir output/qwen_vhm_pretrain_2b \
    --num_train_epochs 1 \
    --per_device_train_batch_size $BATCH_PER_DEVICE \
    --gradient_accumulation_steps $GRAD_ACCUM_STEPS \
    --image_min_pixels $((512 * 28 * 28)) \
    --image_max_pixels $((1280 * 28 * 28)) \
    --learning_rate 1e-5 \
    --merger_lr 1e-5 \
    --vision_lr 2e-6 \
    --weight_decay 0.1 \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 True \
    --gradient_checkpointing True \
    --report_to tensorboard \
    --lazy_preprocess True \
    --save_strategy "steps" \
    --save_steps 200 \
    --save_total_limit 5 \
    --dataloader_num_workers 4

Training (GRPO)

Below script works on at least one node with 4 x H100s or A100s (65-80 GB). Please note that, this script requires OPENAI_API_KEY for exact reproduction of the model, which may cost ~10-15$ per full training. You may skip open ended questions by adding a flag or removing questions with answer_type=0 from the dataset, if you don't have an API key.

export WANDB_RUN_NAME=Qwen-VL-2B-GRPO-$(date +%Y-%m-%d-%H-%M-%S)
export OPENAI_API_KEY=#ENTER YOUR API KEY

torchrun \
    --nproc_per_node="$GPUS_PER_NODE" \
    --nnodes="$SLURM_NNODES" \
    --node_rank="$SLURM_NODEID" \
    --rdzv_backend=c10d \
    --rdzv_endpoint ${MASTER_ADDR}:${MASTER_PORT} \
    --rdzv_id $SLURM_JOB_ID \
    src/open_r1/grpo.py \
    --deepspeed local_scripts/zero3.json \
    --output_dir checkpoints/${WANDB_RUN_NAME} \
    --model_name_or_path aybora/Qwen2-VL-TinyRS-CoT \
    --dataset_name aybora/VHM_dataset_grpo \
    --max_prompt_length 8192 \
    --max_completion_length 8192 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 1 \
    --logging_steps 1 \
    --bf16 true \
    --beta 0.001 \
    --report_to wandb \
    --gradient_checkpointing true \
    --attn_implementation flash_attention_2 \
    --max_pixels 2359296 \
    --save_total_limit 6 \
    --num_train_epochs 64 \
    --num_generations 4 \
    --save_steps 100 \
    --run_name $WANDB_RUN_NAME

You may need to adjust some of the parameters (MASTER_ADDR, GPUS_PER_NODE etc.) depending on your multi-gpu, multi-node setting.

Evaluation

First download datasets eval folder from our Huggingface Repo, which is forked from ScoreRS HF Repo.

To evaluate our, or your reproduced model, you may use the script below:

SCRIPT_PATH=./TinyRS/eval/python_script/evaluation/rs_evaluation.py
DATA_ROOT="Your path to datasets eval folder"
OUTPUT_DIR="Your path to eval log file"
model_type=lmdeploy
MODEL_PATH=aybora/Qwen2-VL-TinyRS-R1
REASONING_CONFIG=./TinyRS/eval/config/qwen2_thinking_template.json

PYTHONPATH=$(pwd) CUDA_VISIBLE_DEVICES=0 accelerate launch --num_processes 1 --mixed_precision bf16 $SCRIPT_PATH \
    --data_root $DATA_ROOT \
    --output_dir $OUTPUT_DIR \
    --model_type $model_type \
    --model_path $MODEL_PATH \
    --force_inference true \
    --task all \
    --reasoning_config $REASONING_CONFIG

Acknowledgements

Our work is derived from Qwen2-VL for the base model, Qwen-VL-Series-Finetune for the forked main sft code, open-r1-multimodal for the forked main grpo code, ScoreRS for the forked main evaluation code and VHM for the dataset and the base captions. We appreciate all of these great works.

Citation

If you find this code useful for your research, consider citing our works:

@article{koksal2025tinyrs,
  title={Tinyrs-r1: Compact vision language model for remote sensing},
  author={K{\"o}ksal, Aybora and Alatan, A Ayd{\i}n},
  journal={IEEE Geoscience and Remote Sensing Letters},
  year={2025},
  publisher={IEEE}
}

@article{koksal2025samchat,
  title={SAMChat: Introducing Chain-of-Thought Reasoning and GRPO to a Multimodal Small Language Model for Small-Scale Remote Sensing},
  author={K{\"o}ksal, Aybora and Alatan, A Ayd{\i}n},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  volume={19},
  pages={795--804},
  year={2025},
  publisher={IEEE}
}

If you are interested in this work, you may find the following work also useful:

@inproceedings{koksal2025few,
  title={Few-Shot Vision-Language Reasoning for Satellite Imagery via Verifiable Rewards},
  author={K{\"o}ksal, Aybora and Alatan, A Ayd{\i}n},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={6901--6910},
  year={2025}
}