jankin123/4DThinker-3B

Model

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

9 commits

1 linked in READMEs

updated May 11, 2026

See the code

README

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

Paper | Code

4DThinker is a framework that enables Vision-Language Models (VLMs) to "think with 4D" through dynamic latent mental imagery—internally simulating how scenes evolve within the continuous hidden space. It addresses dynamic spatial reasoning from monocular video by grounding the model in dynamic visual semantics.

This repository contains the trained model checkpoints from Qwen2.5-VL-3B for 4DThinker.

Model Structure

model/
├── dift/
│   ├── checkpoints/          # DIFT-stage model weights
│   │   ├── model-00001-of-00002.safetensors
│   │   ├── model-00002-of-00002.safetensors
│   │   ├── config.json
│   │   ├── tokenizer.json
│   │   └── ...
│   └── tensorboard/          # DIFT training logs
└── 4drl/
    ├── model-00001-of-00002.safetensors
    ├── model-00002-of-00002.safetensors
    ├── config.json
    ├── tokenizer.json
    ├── trainer_state.json
    └── ...

Models

Model	Stage	Base Model	Description
`dift/checkpoints/`	DIFT	Qwen2.5-VL-3B-Instruct	Supervised with cosine similarity loss on latent visual tokens
`4drl/`	4DRL (GRPO)	DIFT checkpoint	Reinforced with answer-based rewards

Special Tokens

Three special tokens are added to the Qwen2.5-VL vocabulary to support latent imagery:

Token	Description
`<	latent_pad
`<	latent_start
`<	latent_end

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "jankin123/4DThinker-3B",
    subfolder="4drl",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("jankin123/4DThinker-3B", subfolder="4drl")

Citation

@article{4dthinker,
  title={4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding},
  author={Zhang, Quanchen and others},
  journal={arXiv preprint arXiv:2605.05997},
  year={2026}
}

Bibtex

If you find 4DThinker helpful for your work, please cite

@article{chen20264dthinker,
  title={4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding},
  author={Chen, Zhangquan and Zhang, Manyuan and Yu, Xinlei and An, Xiang and Li, Bo and Xie, Xin and Wang, ZiDong and Sun, Mingze and Chen, Shuang and Li, Hongyu and others},
  journal={arXiv preprint arXiv:2605.05997},
  year={2026}
}

License

Apache License 2.0

4DThinker

dynamic-spatial-reasoning

endpoints_compatible

latent-reasoning

safetensors

tensorboard

transformers

video-text-to-text

vision-language-model

Contributors

chenzhangquan03

4 commits

jankin123

4 commits

nielsr

1 commits

Similar repos

zhangquanchen/4DThinker

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

TencentARC/DSR_Suite-Model

Model

Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models

internlm/Spatial-SSRL-Qwen3VL-4B

Model

Spatial-SSRL-Qwen3VL-4B

hongxingli/SpatialLadder-3B

Model

SpatialLadder-3B

jankin123/4DThinker-Training-Data

Dataset

4DThinker Training Data

internlm/Spatial-SSRL-3B

Model

Spatial-SSRL-3B

internlm/Spatial-SSRL-7B

Model

Spatial-SSRL-7B

inclusionAI/ViLaSR

Model

This repository contains the ViLaSR-7B model as presented in [Reinforcing Spatial Reasoning in…

Linked in READMEs

zhangquanchen/4DThinker

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

README badge

jankin123/4DThinker-3B

Model

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

9 commits

1 linked in READMEs

updated May 11, 2026

See the code

README

4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

Paper | Code

This repository contains the trained model checkpoints from Qwen2.5-VL-3B for 4DThinker.

Model Structure

model/
├── dift/
│   ├── checkpoints/          # DIFT-stage model weights
│   │   ├── model-00001-of-00002.safetensors
│   │   ├── model-00002-of-00002.safetensors
│   │   ├── config.json
│   │   ├── tokenizer.json
│   │   └── ...
│   └── tensorboard/          # DIFT training logs
└── 4drl/
    ├── model-00001-of-00002.safetensors
    ├── model-00002-of-00002.safetensors
    ├── config.json
    ├── tokenizer.json
    ├── trainer_state.json
    └── ...

Models

Model	Stage	Base Model	Description
`dift/checkpoints/`	DIFT	Qwen2.5-VL-3B-Instruct	Supervised with cosine similarity loss on latent visual tokens
`4drl/`	4DRL (GRPO)	DIFT checkpoint	Reinforced with answer-based rewards

Special Tokens

Three special tokens are added to the Qwen2.5-VL vocabulary to support latent imagery:

Token	Description
`<	latent_pad
`<	latent_start
`<	latent_end

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "jankin123/4DThinker-3B",
    subfolder="4drl",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("jankin123/4DThinker-3B", subfolder="4drl")

Citation

@article{4dthinker,
  title={4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding},
  author={Zhang, Quanchen and others},
  journal={arXiv preprint arXiv:2605.05997},
  year={2026}
}

Bibtex

If you find 4DThinker helpful for your work, please cite

@article{chen20264dthinker,
  title={4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding},
  author={Chen, Zhangquan and Zhang, Manyuan and Yu, Xinlei and An, Xiang and Li, Bo and Xie, Xin and Wang, ZiDong and Sun, Mingze and Chen, Shuang and Li, Hongyu and others},
  journal={arXiv preprint arXiv:2605.05997},
  year={2026}
}