Dataset of contrastive stories for SAE evaluation. Subjects are sampled from GulkoA/wikithings

Designed for CE-Bench

prompts:

system = f"""
Write concisely and clearly. Use no more than 200 words. Be as brief as possible.
"""

subject_prompt1 = f"""
Write how you would describe {subject.upper()} in its high, extreme form. Rephrase things if needed, be very brief, specific, detailed, and realistic.
For example,
"active" -> "extremely vibrant, energetic, and lively"
"angry" -> "extremely mad, furious, and enraged"
"""
subject_prompt2 = f"""
Now, write how you would describe the exact opposite of {subject.upper()}. Rephrase things if needed, be very brief, specific, detailed, and realistic. DO NOT USE THE WORDS {subject.upper()} in your answer, instead write the opposite of the concept.
For example,
"active" -> "very inactive, lethargic, sluggish, and lazy"
"angry" -> "very calm, peaceful, and relaxed"
"""

story_prompt1 = f"""
Write a short story describing the following: {subject1}.
"""
story_prompt2 = f"""
Now, rewrite this story describing the following: {subject2} (the exact opposite of the previous story).
"""

Dataset generated with GPT-4.1