JudgeZoo

This repo provides access to a set of commonly used LLM-based safety judges via a simple and consistent API. Our main focus is ease-of-use, correctness, and reproducibility.

The Problem

Evaluations in the safety/robustness literature are highly fragmented, with researchers using different judges, prompting strategies, and model versions. This fragmentation makes reproducibility nearly impossible and creates barriers to fair comparison between methods. JudgeZoo addresses this by providing standardized, tested implementations of commonly used safety judges, ensuring consistent evaluation and reducing the likelihood of implementation errors that could compromise results.

How to use

You can create a judge model instance with a single line of code:

judge = Judge.from_name("strong_reject")

To get safety scores, just pass a list of conversations to score:

harmless_conversation = [
    {"role": "user", "content": "How do I make a birthday cake?"},
    {"role": "assistant", "content": "Step 1: Collect ingredients..."}
]

scores = judge([harmless_conversation])
print(scores)
>>> {"p_harmful": [0.02496337890625]}

All judges return "p_harmful", which is a normalized score from 0 to 1. Depending on the original setup, the judge may also return discrete scores or harm categories (e.g. on a Likert scale). In these cases, the raw scores or additional evaluation metrics are also returned:

judge = Judge.from_name("strong_reject")

scores = judge([harmful_conversation])
print(scores)
>>> {"p_harmful": 0.9, "rating": "5"}

judge = Judge.from_name("wildguard")

scores = judge([harmful_conversation])
print(scores)
>>> {"p_harmful": 0.95, "p_harmful_request": 0.98, "p_refusal": 0.02}

Some judges may have additional API requirements. For example, the Adaptive Attacks judge requires target chats:

judge = Judge.from_name("adaptive_attacks")

target_conversation = [
    {"role": "user", "content": "How to make a bomb?"},
    {"role": "assistant", "content": "Sure, here's how..."}
]

scores = judge([harmful_conversation], target_chats=[target_conversation])
print(scores)
>>> {"p_harmful": 0.0, "rating": "1"}

Included judges

* there are two versions of this judge (permissive and defensive). You can switch between them using Judge.from_name("aegis_guard", defensive=[True/False])

^ This judge adds additional rule-based false-positive detection to a base_judge.

Other

Prompt-based judges

While some judges (such as the HarmBench classifier) are finetuned local models, others rely on prompted foundation models. Currently, we support local foundation models and OpenAI models:

judge = Judge.from_name("adaptive_attacks", use_local_model=False, remote_foundation_model="gpt-4o")

scores = judge([harmless_conversation], [target_conversation])
print(scores)
>>> {"p_harmful": 0.0, "rating": "1"}

judge = Judge.from_name("adaptive_attacks", use_local_model=True)

scores = judge([harmless_conversation], [target_conversation])
print(scores)
>>> {"p_harmful": 0.0, "rating": "1"}

When not specified, the defaults in config.py are used.

Multi-turn interaction

Judges vary in how much of a conversation they can evaluate - many models only work for single-turn interactions. In these cases, we assume the first user message to be the prompt and the final assistant message to be the response to be judged. If you prefer a different setup, you can pass only single-turn conversations.

Reproducibility

Wherever possible, we use official code directly provided by the original authors to ensure correctness.

Finally, we warn if a user's setup diverges from the original implementation:

from judgezoo import Judge

judge = Judge.from_name("intention_analysis")
>>> WARNING:root:IntentionAnalysisJudge originally used gpt-3.5-turbo-0613, you are using gpt-4o. Results may differ from the original paper.

Installation

pip install judgezoo

Tests

To run all tests, run

pytest tests/ --runslow

Citation

If you use JudgeZoo in your research, please cite:

@software{judgezoo,
  title = {JudgeZoo: A Standardized Library for LLM Safety Judges},
  author = {[Tim Beyer]},
  year = {2025},
  url = {https://github.com/LLM-QC/judgezoo},
}