Dataset Card for ECInstruct

Our paper was accepted to ICML 2024.

ECInstruct comprises 10 tasks, including attribute value extraction, product relation prediction, product matching, sentiment analysis, sequential recommendation, multiclass product classification, product substitute identification, query product rank, answerability prediction, and answer generation. ECInstruct is split into training sets, validation sets, in-domain (IND) test sets, and out-of-domain (OOD) test sets.

We also provide the product labels for the test set of query-product ranking task, which can be used for evaluation. Please check https://github.com/amazon-science/esci-data for more details.

Dataset Sources

• Repository: GitHub
• Homepage: eCeLLM

Data Split

The statistic of the ECInstruct Dataset is shown in the table below.

Usage

As detailed in the paper, for each task, we could conduct training and evaluation under multiple settings. For example, setting = IND_Diverse_Instruction, task = Answer_Generation indicates the training set for learning models on the answer generation task with diverse instructions for the IND test set.

Quick Start

Run the following command to get the data:

from datasets import load_dataset

dataset = load_dataset("NingLab/ECInstruct")

For better reproducibility, besides the ECInstruct dataset, we put all the data used in various analyses in Hugging Face. That is, the dataset in Hugging Face comprises the ECInstruct dataset, data samples with the seed instruction and unseen instruction as used in the analysis of Section 6.3 and G.1, and data samples used for 1-shot evaluation. If you are only interested in the ECInstruct dataset, please refer to the following code to extract the data:

from datasets import load_dataset
import pandas as pd

dataset = pd.DataFrame(load_dataset("NingLab/ECInstruct")['train'])
default_dataset = dataset[(dataset['setting'] == 'IND_Diverse_Instruction') | (dataset['setting'] == 'OOD_Diverse_Instruction')]

License

Please check the license of each subset in our curated dataset ECInstruct.

Citation

@inproceedings{
    peng2024ecellm,
    title={eCe{LLM}: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data},
    author={Bo Peng and Xinyi Ling and Ziru Chen and Huan Sun and Xia Ning},
    booktitle={Forty-first International Conference on Machine Learning},
    year={2024},
    url={https://openreview.net/forum?id=LWRI4uPG2X}
}