Llama-Prompt-Guard-2-22M-onnx

This repository provides a ONNX converted and quantized version of meta-llama/Llama-Prompt-Guard-2-22M

🧠 Built With

• Meta LLaMA – Foundation model powering the classifier
  • meta-llama/Llama-Prompt-Guard-2-22M
  • meta-llama/Llama-Prompt-Guard-2-86M
• 🤗 Hugging Face Transformers – Model and tokenizer loading
• ONNX – Model export and runtime format
• ONNX Runtime – Efficient inference backend

📥 Evaluation Dataset

We use jackhhao/jailbreak-classification for the evaluation (train+test)

🧪 Evaluation Results

🤗 Usage

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification
import numpy as np

# Load model and tokenizer using optimum
model = ORTModelForSequenceClassification.from_pretrained("gravitee-io/Llama-Prompt-Guard-2-22M-onnx", file_name="model.quant.onnx")
tokenizer = AutoTokenizer.from_pretrained("gravitee-io/Llama-Prompt-Guard-2-22M-onnx")

# Tokenize input
text = "Your comment here"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)

# Run inference
outputs = model(**inputs)
logits = outputs.logits

# Optional: convert to probabilities
probs = 1 / (1 + np.exp(-logits))
print(probs)

🐙 GitHub Repository:

You can find the full source code, CLI tools, and evaluation scripts in the official GitHub repository.