ibm-research/FailureSensorIQ

Dataset

9

stars

18

commits

1

linked in READMEs

Feb 24, 2026

updated

industry

README

FailureSensorIQ Dataset

FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets.

|Github | 🏆Leaderboard | 📖Paper |

Dataset Summary

FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets. By only leveraging the information found in ISO documents, we developed a data generation pipeline that creates questions in two types: (i) FailureMode2Sensor and (ii) Sensor2FailureMode. Additionally, we designed questions in a selection (select the relevant ones) vs. elimination (eliminate the irrelevant ones) format.

FailureSensorIQ dataset consists of 8,296 questions across 10 assets, with 2,667 single-true multi-choice questions and 5,629 multi-true multi-choice questions. The following is the list of assets with count within 2,667 single-true multi-choice questions:

AssetCount
Electric Motor234
Steam Turbine171
Aero Gas Turbine336
Industrial Gas Turbine240
Pump152
Compressor220
Reciprocating IC Engine336
Electric Generator234
Fan200
Power Transformer544

Please find more statistics of the dataset and the dataset construction process from our Github and Paper.

Load the Dataset

To load 2,667 single-true multi-choice QA,

from datasets import load_dataset

# Login using e.g. `huggingface-cli login` to access this dataset
ds = load_dataset("ibm-research/FailureSensorIQ", "single_true_multi_choice_qa")

To load 5,629 multi-true multi-choice QA,

from datasets import load_dataset

# Login using e.g. `huggingface-cli login` to access this dataset
ds = load_dataset("ibm-research/FailureSensorIQ", "multi_true_multi_choice_qa")

Leaderboard

We have benchmarked both open-source LLMs and frontier LLMs on FailureSensorIQ. Furthermore, we have conducted thoroughout Perturbation–Uncertainty–Complexity analysis for deeper insights of each LLMs. For the most updated leaderboard, please refer to our Leaderboard. You can submit the evaluation there.

If you want to reproduce our results, please check out our Github for the evaluation scripts. We documented all the steps you should take to run the evaluation.

Cite this Dataset

If you use our dataset in your paper, please cite our dataset by

@misc{constantinides2025failuresensoriqmultichoiceqadataset,
      title={FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes}, 
      author={Christodoulos Constantinides and Dhaval Patel and Shuxin Lin and Claudio Guerrero and Sunil Dagajirao Patil and Jayant Kalagnanam},
      year={2025},
      eprint={2506.03278},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.03278}, 
}

Contributors

DhavalPatel

10 commits

shuxinl

5 commits

cc4718

3 commits

ibm-research/FailureSensorIQ

Dataset

9

stars

18

commits

1

linked in READMEs

Feb 24, 2026

updated

industry

README

FailureSensorIQ Dataset

FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets.

|Github | 🏆Leaderboard | 📖Paper |

Dataset Summary

FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets. By only leveraging the information found in ISO documents, we developed a data generation pipeline that creates questions in two types: (i) FailureMode2Sensor and (ii) Sensor2FailureMode. Additionally, we designed questions in a selection (select the relevant ones) vs. elimination (eliminate the irrelevant ones) format.

FailureSensorIQ dataset consists of 8,296 questions across 10 assets, with 2,667 single-true multi-choice questions and 5,629 multi-true multi-choice questions. The following is the list of assets with count within 2,667 single-true multi-choice questions:

AssetCount
Electric Motor234
Steam Turbine171
Aero Gas Turbine336
Industrial Gas Turbine240
Pump152
Compressor220
Reciprocating IC Engine336
Electric Generator234
Fan200
Power Transformer544

Please find more statistics of the dataset and the dataset construction process from our Github and Paper.

Load the Dataset

To load 2,667 single-true multi-choice QA,

from datasets import load_dataset

# Login using e.g. `huggingface-cli login` to access this dataset
ds = load_dataset("ibm-research/FailureSensorIQ", "single_true_multi_choice_qa")

To load 5,629 multi-true multi-choice QA,

from datasets import load_dataset

# Login using e.g. `huggingface-cli login` to access this dataset
ds = load_dataset("ibm-research/FailureSensorIQ", "multi_true_multi_choice_qa")

Leaderboard

We have benchmarked both open-source LLMs and frontier LLMs on FailureSensorIQ. Furthermore, we have conducted thoroughout Perturbation–Uncertainty–Complexity analysis for deeper insights of each LLMs. For the most updated leaderboard, please refer to our Leaderboard. You can submit the evaluation there.

If you want to reproduce our results, please check out our Github for the evaluation scripts. We documented all the steps you should take to run the evaluation.

Cite this Dataset

If you use our dataset in your paper, please cite our dataset by

@misc{constantinides2025failuresensoriqmultichoiceqadataset,
      title={FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes}, 
      author={Christodoulos Constantinides and Dhaval Patel and Shuxin Lin and Claudio Guerrero and Sunil Dagajirao Patil and Jayant Kalagnanam},
      year={2025},
      eprint={2506.03278},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.03278}, 
}

Contributors

DhavalPatel

10 commits

shuxinl

5 commits

cc4718

3 commits