News

• 2025/11/5 Our paper "RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow" is accepted by AAAI 2026!
• 2025/08/16 Welcome to RemoteReasoner. This is the first Reinforcement Learning-based reasoning framework in remote sensing.

Introduction

Remote sensing imagery presents vast, inherently unstructured spatial data, necessitating sophisticated reasoning to interpret complex user intents and contextual relationships beyond simple recognition tasks. In this paper, we aim to construct an Earth observation workflow to handle complex queries by reasoning about spatial context and user intent. As a reasoning workflow, it should autonomously explore and construct its own inference paths, rather than being confined to predefined ground‑truth sequences. Ideally, its architecture ought to be unified yet generalized, possessing capabilities to perform diverse reasoning tasks through one model without requiring additional fine-tuning. Existing remote sensing approaches rely on supervised fine-tuning paradigms and task‑specific heads, limiting both autonomous reasoning and unified generalization. To this end, we propose RemoteReasoner, a unified workflow for geospatial reasoning. The design of RemoteReasoner integrates a multi-modal large language model (MLLM) for interpreting user instructions and localizing targets, together with task transformation strategies that enable multi-granularity tasks, including object-, region-, and pixel-level. In contrast to existing methods, our framework is trained with reinforcement learning (RL) to endow the MLLM sufficient reasoning autonomy. At the inference stage, our transformation strategies enable diverse task output formats without requiring task-specific decoders or further fine-tuning. Experiments demonstrated that RemoteReasoner achieves state-of-the-art (SOTA) performance across multi-granularity reasoning tasks. Furthermore, it retains the MLLM's inherent generalization capability, demonstrating robust performance on unseen tasks and out-of-distribution categories.

Quick Start

Prerequisites

• Python >= 3.8
• CUDA >= 11.8 (for GPU support)
• 16GB+ GPU memory recommended

Setting Up

1. Clone this repository:

git clone https://github.com/1e12Leon/RemoteReasoner.git
cd RemoteReasoner

2. Install dependencies:

pip install -e .

3. Download the pre-trained weights:

   • RemoteReasoner Model: Download from HuggingFace
   • SAM2 Weights: Download SAM2 model weights and place them in the root directory:
     • sam2.1_hiera_tiny.pt (149MB) - Download Link
     • sam2.1_hiera_large.pt (857MB) - Optional, for better performance

4. Organize your directory structure:

RemoteReasoner/
├── checkpoints/
│   └── RemoteReasoner-7B-merged-bf16/  # Place downloaded model here
├── sam2.1_hiera_tiny.pt
├── RemoteReasoner.py
└── ...

Training

We provides training scripts to fine-tune Qwen2.5-VL-7B-Instruct with GRPO (Group Relative Policy Optimization) using LoRA. The training leverages multi-GPU distributed training with DeepSpeed ZeRO-3 for efficient memory usage.

bash RemoteReasoner_GRPO.sh

⚙️ Key Arguments

Inference

Initialize the model and load the RemoteReasoner checkpoint:

import argparse
from RemoteReasoner import RemoteReasoner

parser = argparse.ArgumentParser()
parser.add_argument('--model_path', type=str, 
                    default='checkpoints/RemoteReasoner-7B-merged-bf16',
                    help="Path to the model")
args = parser.parse_args()

# Initialize RemoteReasoner
reasoner = RemoteReasoner(args, device=0)
    

• Pixel Reasoning

img_path = "./assets/demo.jpg"
question = "your query."
think, answer, mask = reasoner.Pixel_reasoning(img_path, question)
# Save the mask
mask.save("output_mask.png")

• Region Reasoning

img_path = "./assets/demo.jpg"
queston = "your query."
think, answer = reasoner.Region_reasoning(img_path, question)

• Contour Reasoning

img_path = "./assets/demo.jpg"
queston = "your query."
think, answer, contour = reasoner.Contour_reasoning(img_path, question)

• Visual Queston Answering

img_path = "./assets/demo.jpg"
queston = "your question."
think, answer = reasoner.VQA(img_path, question)

• Image Captioning

img_path = "./assets/demo.jpg"
think, answer, mask = reasoner.Image_captioning(img_path)

Acknowledge

• Thanks to Kaiyu for providing the EarthReason dataset
• Thanks for the MS-SWIFT repo.

Cite

If you find this work useful, please cite our paper as:

@article{yao2025remotereasoner,
  title={RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow},
  author={Yao, Liang and Liu, Fan and Lu, Hongbo and Zhang, Chuanyi and Min, Rui and Xu, Shengxiang and Di, Shimin and Peng, Pai},
  journal={arXiv preprint arXiv:2507.19280},
  year={2025}
}