---

This repository contains the official implementation of the paper
PRL: Prompts from Reinforcement Learning

Abstract:
Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting them typically demands expert intuition and a nuanced understanding of the task. Moreover, the most impactful prompts often hinge on subtle semantic cues, ones that may elude human perception but are crucial for guiding LLM behavior.
In this paper, we introduce PRL (Prompts from Reinforcement Learning), a novel RL-based approach for automatic prompt generation. Unlike previous methods, PRL can produce novel few-shot examples that were not seen during training. Our approach achieves state-of-the-art performance across a range of benchmarks, including text classification, simplification, and summarization.

---

✨ Highlights

• 🚀 Outperforms prior prompt optimization methods like APE and EvoPrompt
• 🧠 Automatically generates novel few-shot prompts unseen during training
• 📈 Strong gains across multiple NLP tasks:
  • +2.58% over APE and +1.00% over EvoPrompt (Classification)
  • +4.32 ROUGE over APE and +2.12 over EvoPrompt (Summarization)
  • +6.93 SARI over APE and +6.01 over EvoPrompt (Simplification)

---

🛠️ Installation & Setup

PRL is based on the ms-swift framework.
Please follow the environment setup instructions provided in that repository.

You also need to download the benchmark datasets (Classification, Summarization, Simplification) from
👉 https://nlp.cs.princeton.edu/projects/lm-bff/datasets.tar

---

🧪 Benchmarks

🔤 Classification

To train PRL on the MR dataset:

./scripts/mr/mr_qwen_qwen.sh

📚 Summarization

To run PRL on summarization tasks:

./scripts/sum/sum_qwen_qwen.sh

✏️ Simplification

To evaluate PRL on simplification:

./scripts/sim/sim_qwen_qwen.sh

📄 Citation

If you find our work useful, please consider citing:

@article{batorski2025prl,
  title     = {PRL: Prompts from Reinforcement Learning},
  author    = {Batorski, Pawe{\l} and Kosmala, Adrian and Swoboda, Paul},
  journal   = {arXiv preprint arXiv:2505.14412},
  year      = {2025}
}

🙏 Acknowledgments

This work builds on the ms-swift framework.
We thank the authors of APE, EvoPrompt, and APO for their inspiring contributions to the field of automated prompt generation.