llm-security

A research and experimentation repository for evaluating LLM security workflows.

Overview

This project appears to focus on:

• generating and preparing datasets for LLM security experiments
• running classification-based and validation pipelines
• evaluating attack, refinement, and inference outputs
• organizing results and experiment artifacts

The repository contains notebooks, Python utilities, configuration files, and saved evaluation outputs.

Repository structure

• src/ — core Python package code
  • classifier/ — classification-related logic
  • data_generation/ — dataset creation and transformation helpers
  • pipeline/ — pipeline code
  • rag/ — retrieval-augmented generation utilities
  • scripts/ — runnable scripts
  • tools/ — reusable tools
  • utils/ — shared utilities
• data/ — raw, translated, external, and final datasets
• training/ — training assets and classifier pipelines
• evaluation/ — evaluation scripts and CSV outputs
• results/ — experiment result files
• configs/ — pipeline configuration files

Notebooks and artifacts

The repository includes several notebooks for experimentation and analysis, such as:

• classification_based.ipynb
• test_mci_detection.ipynb
• test_wiki.ipynb
• data/create_final_datasets.ipynb

These notebooks likely support dataset creation, detection experiments, and analysis workflows.

Requirements

The project targets Python 3.9+ and uses the following main dependencies:

• requests
• bitsandbytes
• pandas
• openai
• numpy
• tqdm
• openpyxl
• yaml
• tavily-python

Install them with:

pip install -r requirements.txt

Getting started

1. Clone the repository.
2. Create and activate a Python 3.9+ environment.
3. Install dependencies.
4. Review configs/pipeline.yaml for pipeline settings.
5. Explore the notebooks or scripts in src/, training/, and evaluation/.

Usage

Because the repository is research-oriented, the exact entry points may vary by experiment. A typical workflow is:

1. Prepare or transform data in data/
2. Train or run classifier pipelines in training/
3. Execute validation or evaluation scripts in evaluation/
4. Review generated outputs in results/

Notes

• The repository currently does not include a detailed public API description.
• Some folders contain generated artifacts and notebooks rather than production application code.
• If you want, this README can be expanded with concrete commands once the main scripts and notebook entry points are finalized.