Evaluating Creative Short Story Generation in Humans and Large Language Models

This repo contains the code and data for the paper "Evaluating Creative Short Story Generation in Humans and Large Language Models". Final data with all the human and model stories and their expert/non-expert ratings can be found in HuggingFace.

Setup

Install dependencies

pip install -r requirements.txt

Install spacy packages

python -m spacy download en_core_web_sm

Citation

@misc{ismayilzada2024evaluatingcreativeshortstory,
      title={Evaluating Creative Short Story Generation in Humans and Large Language Models}, 
      author={Mete Ismayilzada and Claire Stevenson and Lonneke van der Plas},
      year={2024},
      eprint={2411.02316},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.02316}, 
}