Timeline Annotations for BONES-SEED Humanoid Motion Dataset

Dataset Description:

This dataset provides additional text description annotations from the BONES-SEED humanoid motion dataset. For each motion, this dataset provides an overview text description of the entire motion at a high level, along with a “timeline” of annotated segments within the motion. Each segment generally contains a single atomic action and is defined by a start time, end time, and text description.

Note that these annotations are already included in the BONES-SEED repo but are released here with additional information for completeness.

This dataset is ready for commercial use.

Dataset Owner:

NVIDIA Corporation

Dataset Creation Date:

March-July 2025

License/Terms of Use:

This dataset is governed by the Creative Commons Attribution 4.0 International License (CC BY 4.0).

Intended Usage:

This dataset is intended for users interested in leveraging the BONES-SEED dataset who could benefit from additional text labels and/or fine-grained temporal text descriptions. Potential applications include training text-to-motion generative models or text-motion retrieval models.

Dataset Characterization

Data Collection Method Human
Labeling Method Hybrid: Human, Automated

Human annotators were first asked to provide text annotations for a subset of the dataset. Then, text labels were automatically propagated from annotated motions to the unlabeled motions of the same action type. While propagating the timeline annotations, variations in motion timing were accounted for using dynamic time warping. The dataset metadata indicates which labels were directly provided by a human annotator, and which ones are from automated label propagation.

Dataset Format

JSONL file with each motion on a separate line.

Each line contains:

• filename - unique name of the motion used to cross reference with the BONES-SEED metadata
• overview_description - high-level text description of the entire motion
• events - list of contained segments within this motion. Each event is labeled with:
  • start_time - segment start timestamp within the full motion (sec)
  • end_time - segment end timestamp within the full motion (sec)
  • description - text description of the motion within this segment
• propagated_from_filename - if this motion annotation was automatically propagated from a different motion, this field contains the filename of that motion. Otherwise, if the motion was human-labeled, this field is null.

Dataset Quantification

• Dataset file size: 80 MB
• Total labeled motions: 142220 (same as BONES-SEED)
• Total number of timeline segments: 352703
• Total unique segment descriptions labeled by human annotators: 37943
• Total segment descriptions automatically propagated: 314760
• Average number of timeline segments per motion: 2.48
• Standard deviation number of segments per motion: 2.61
• Maximum number of timeline segments per motion: 78
• Minimum number of timeline segments per motion: 1

References:

• BONES-SEED dataset
• This dataset is used to train Kimodo

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here.