Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs

This is the official implementation for the paper Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs.

Setup

To run the experiments, please create a conda environment and install the required dependencies.

Environment Creation

Create a new conda environment named expected-harm using Python 3.12:

conda create -n expected-harm python=3.12
conda activate expected-harm

Install Dependencies

Install the required packages using requirements.txt:

pip install -r requirements.txt

Directory Structure

This repository contains the following experiment directories. Each directory includes its own README.md with detailed running instructions.

• jailbreak_experiment/: Contains the code for the main experiment presented in the paper.
• cost_severity_ablations/: Contains the code for the cost and severity ablations.
• linear_probing_experiment/: Contains the code for the linear probing experiments.