MCG-NJU/TimeLens2-4B

Model

12

stars

18

commits

1

repos using this model

1

linked in READMEs

Jul 21, 2026

updated

endpoints_compatible
image-text-to-text
qwen3-vl
qwen3_vl
safetensors
temporal-grounding
transformers
video
video-text-to-text
Browse cluster: Multimodal Model Compression & Optimization

README

TimeLens2-4B

TimeLens2-4B is a video multimodal large language model for temporal grounding. Given a video and a text query, it returns the time interval containing the relevant visual evidence.

The model is built on Qwen3-VL-4B-Instruct and achieves 47.7 average mIoU across seven temporal grounding benchmarks.

Despite its compact size, TimeLens2-4B delivers state-of-the-art performance among similarly sized models and outperforms substantially larger open and proprietary baselines.

Paper

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Benchmark Results

Temporal grounding benchmark results

Inference

pip install -U torch torchvision "transformers>=4.57.0" accelerate "qwen-vl-utils[decord]>=0.0.14"
pip install -U flash-attn --no-build-isolation
from pathlib import Path

from qwen_vl_utils import process_vision_info
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "MCG-NJU/TimeLens2-4B"
video_path = "/path/to/video.mp4"
query = "A man opens the refrigerator."

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    attn_implementation="flash_attention_2",
)
processor = AutoProcessor.from_pretrained(model_id)

prompt = (
    f'Given the query: "{query}", return ALL time spans (in seconds) where the query is relevant.\n'
    "Output format MUST be a JSON array of [start, end] pairs.\n"
)
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": Path(video_path).resolve().as_uri(),
                "fps": 2.0,
                "min_pixels": 32 * 32,
                "max_pixels": 480 * 480,
                "total_pixels": 128000 * 32 * 32,
            },
            {"type": "text", "text": prompt},
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
images, videos, video_kwargs = process_vision_info(
    messages,
    image_patch_size=16,
    return_video_kwargs=True,
    return_video_metadata=True,
)

if videos is not None:
    videos, video_metadatas = zip(*videos)
    videos, video_metadatas = list(videos), list(video_metadatas)
else:
    video_metadatas = None

inputs = processor(
    text=text,
    images=images,
    videos=videos,
    video_metadata=video_metadatas,
    do_resize=False,
    return_tensors="pt",
    **video_kwargs,
).to(model.device)

output_ids = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.01,
    top_p=0.001,
    top_k=1,
    repetition_penalty=1.0,
)
output_ids = [
    output[len(input_ids) :]
    for input_ids, output in zip(inputs.input_ids, output_ids)
]
response = processor.batch_decode(
    output_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(response[0])

Citation

@misc{zhu2026timelens2,
      title={TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs},
      author={Yuhan Zhu and Changlian Ma and Xiangyu Zeng and Xinhao Li and Zhiqiu Zhang and Songze Li and Jun Zhang and Tianxiang Jiang and Yuandong Yang and Ziang Yan and Zikang Wang and Xinyu Chen and Haoran Chen and Shaowei Zhang and Limin Wang},
      year={2026},
      eprint={2607.17423},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.17423},
}

Contributors

ZhuYuhan

16 commits

ZZQ987

2 commits

MCG-NJU/TimeLens2-4B

Model

12

stars

18

commits

1

repos using this model

1

linked in READMEs

Jul 21, 2026

updated

endpoints_compatible
image-text-to-text
qwen3-vl
qwen3_vl
safetensors
temporal-grounding
transformers
video
video-text-to-text
Browse cluster: Multimodal Model Compression & Optimization

README

TimeLens2-4B

TimeLens2-4B is a video multimodal large language model for temporal grounding. Given a video and a text query, it returns the time interval containing the relevant visual evidence.

The model is built on Qwen3-VL-4B-Instruct and achieves 47.7 average mIoU across seven temporal grounding benchmarks.

Despite its compact size, TimeLens2-4B delivers state-of-the-art performance among similarly sized models and outperforms substantially larger open and proprietary baselines.

Paper

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Benchmark Results

Temporal grounding benchmark results

Inference

pip install -U torch torchvision "transformers>=4.57.0" accelerate "qwen-vl-utils[decord]>=0.0.14"
pip install -U flash-attn --no-build-isolation
from pathlib import Path

from qwen_vl_utils import process_vision_info
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "MCG-NJU/TimeLens2-4B"
video_path = "/path/to/video.mp4"
query = "A man opens the refrigerator."

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    attn_implementation="flash_attention_2",
)
processor = AutoProcessor.from_pretrained(model_id)

prompt = (
    f'Given the query: "{query}", return ALL time spans (in seconds) where the query is relevant.\n'
    "Output format MUST be a JSON array of [start, end] pairs.\n"
)
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": Path(video_path).resolve().as_uri(),
                "fps": 2.0,
                "min_pixels": 32 * 32,
                "max_pixels": 480 * 480,
                "total_pixels": 128000 * 32 * 32,
            },
            {"type": "text", "text": prompt},
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
images, videos, video_kwargs = process_vision_info(
    messages,
    image_patch_size=16,
    return_video_kwargs=True,
    return_video_metadata=True,
)

if videos is not None:
    videos, video_metadatas = zip(*videos)
    videos, video_metadatas = list(videos), list(video_metadatas)
else:
    video_metadatas = None

inputs = processor(
    text=text,
    images=images,
    videos=videos,
    video_metadata=video_metadatas,
    do_resize=False,
    return_tensors="pt",
    **video_kwargs,
).to(model.device)

output_ids = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.01,
    top_p=0.001,
    top_k=1,
    repetition_penalty=1.0,
)
output_ids = [
    output[len(input_ids) :]
    for input_ids, output in zip(inputs.input_ids, output_ids)
]
response = processor.batch_decode(
    output_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(response[0])

Citation

@misc{zhu2026timelens2,
      title={TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs},
      author={Yuhan Zhu and Changlian Ma and Xiangyu Zeng and Xinhao Li and Zhiqiu Zhang and Songze Li and Jun Zhang and Tianxiang Jiang and Yuandong Yang and Ziang Yan and Zikang Wang and Xinyu Chen and Haoran Chen and Shaowei Zhang and Limin Wang},
      year={2026},
      eprint={2607.17423},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.17423},
}

Contributors

ZhuYuhan

16 commits

ZZQ987

2 commits