README
Dataset Card for SafeSora
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Project Homepage]
[🤗 SafeSora Datasets]
[🤗 SafeSora Label]
[🤗 SafeSora Evaluation]
[⭐️ Github Repo]
SafeSora is a human preference dataset designed to support safety alignment research in the text-to-video generation field, aiming to enhance the helpfulness and harmlessness of Large Vision Models (LVMs). It currently contains three types of data:
- A classification dataset (SafeSora-Label) of 57k+ Text-Video pairs, including multi-label classification of 12 harm labels for their text prompts and text-video pairs.
- A human preference dataset (SafeSora) of 51k+ instances in the text-to-video generation task, containing comparative relationships in terms of helpfulness and harmlessness, as well as four sub-dimensions of helpfulness.
- An evaluation dataset (SafeSora-Eval) containing 600 human-written prompts, with 300 being safety-neutral and another 300 constructed according to 12 harm categories as red-team prompts.
In the future, we will also open-source some baseline alignment algorithms that utilize these datasets.
Human Preference Dataset
SafeSora is a human preference dataset of 51k+ instances in the text-to-video generation task, containing comparative relationships in terms of helpfulness and harmlessness, as well as four sub-dimensions of helpfulness.
Each data point comprising a user input and two generated videos. Through a heuristic-based annotation process, human preferences were obtained in terms of helpfulness or harmlessness dimensions.
Additionally, due to a pre-annotation heuristic process, human preferences on four helpfulness sub-dimensions were also included. These sub-dimensions are:
Instruction FollowingCorrectnessInformativenessAesthetics
The specific annotation process is as shown in the figure below:
Visualization example of data points
Similar repos
README
Dataset Card for SafeSora
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Project Homepage]
[🤗 SafeSora Datasets]
[🤗 SafeSora Label]
[🤗 SafeSora Evaluation]
[⭐️ Github Repo]
SafeSora is a human preference dataset designed to support safety alignment research in the text-to-video generation field, aiming to enhance the helpfulness and harmlessness of Large Vision Models (LVMs). It currently contains three types of data:
- A classification dataset (SafeSora-Label) of 57k+ Text-Video pairs, including multi-label classification of 12 harm labels for their text prompts and text-video pairs.
- A human preference dataset (SafeSora) of 51k+ instances in the text-to-video generation task, containing comparative relationships in terms of helpfulness and harmlessness, as well as four sub-dimensions of helpfulness.
- An evaluation dataset (SafeSora-Eval) containing 600 human-written prompts, with 300 being safety-neutral and another 300 constructed according to 12 harm categories as red-team prompts.
In the future, we will also open-source some baseline alignment algorithms that utilize these datasets.
Human Preference Dataset
SafeSora is a human preference dataset of 51k+ instances in the text-to-video generation task, containing comparative relationships in terms of helpfulness and harmlessness, as well as four sub-dimensions of helpfulness.
Each data point comprising a user input and two generated videos. Through a heuristic-based annotation process, human preferences were obtained in terms of helpfulness or harmlessness dimensions.
Additionally, due to a pre-annotation heuristic process, human preferences on four helpfulness sub-dimensions were also included. These sub-dimensions are:
Instruction FollowingCorrectnessInformativenessAesthetics
The specific annotation process is as shown in the figure below:
Visualization example of data points