📌 Overview

AutoPYara is a Python framework for automated YARA rule generation from collections of malware samples. It combines:

• Variational Bayesian Gaussian Mixture Models (VBGMM)
• Augmented DBSCAN with centroid refinement
• Malicious/benign Bloom filter isolation
• Byte-level n-gram feature extraction

The result: cluster-aware, precision-engineered YARA signatures with minimal manual effort.

🧠 How it works

flowchart TD
    A[Malware Samples] --> B[Byte n-gram Extraction]
    B --> C["Bloom Filter Isolation<br/>(benign removal + malicious focus)"]
    C --> D["Clustering Engine<br/>(VBGMM or Augmented DBSCAN)"]
    D --> E[Cluster-Specific Signature Construction]
    E --> F[High-Quality YARA Rules]

✨ Features

• Automated Clustering — group similar malware samples together automatically to create concise, targeted rules.
• Two Core Presets — the standard AutoYara (VBGMM) approach, or the enhanced AutoPYara (Augmented DBSCAN) pipeline.
• Built-in Bloom Filters — ships with pre-trained EMBER and AutoPYara filters to efficiently filter out benign n-grams.
• Multiple Output Formats — raw strings, compiled yara-python objects, or yaramod parsed objects.
• Custom Training — train your own Bloom filters on proprietary datasets.

🚀 Installation

Requirements

• Python >= 3.9
• A Java Runtime Environment (JRE 11+) on PATH or pointed to by JAVA_HOME. AutoPYara's clustering/rule-generation backend runs inside a JVM. pip install itself doesn't need Java, but AutoPYara() will raise a clear error the first time you construct it without one — install a JRE before you actually use the tool. On Debian/Ubuntu: sudo apt install default-jre.

pip install autopyara

python -m build
pip install dist/autopyara-*.whl

autopyara-download

⚡ Quick Start

Generating your first YARA rule is as simple as pointing the tool at a directory of malware samples.

from autopyara import AutoPYara

# 1. Initialize the tool
tool = AutoPYara()

# 2. Generate a rule using the AutoPYara preset
results = tool.generate(
    input_files="/path/to/malware/directory",
    preset="AutoPYara",
    rule_name="my_custom_rule",
    output_format="string"
)

# 3. Print the results
print(f"Discovered {results['k_clusters']} distinct malware clusters.")
print("\nGenerated YARA Rule:")
print(results['rule_string'])

# Force exactly 4 clusters using the AutoPYara augmented pipeline
results = tool.generate(
    input_files="/path/to/malware",
    preset="AutoPYara",
    augmented_target_k=4  # Forces the optimizer to find 4 clusters
)

# Returns a compiled yara-python object ready for immediate scanning
results = tool.generate(
    input_files="/path/to/malware",
    output_format="yara-python"
)

compiled_rule = results["output"]
matches = compiled_rule.match("/path/to/suspicious/file.exe")

results = tool.generate(
    input_files="/path/to/malware",
    bloom_malicious="/absolute/path/to/custom/malicious_bloom",
    bloom_benign="/absolute/path/to/custom/benign_bloom",
)

tool = AutoPYara()

# Extract 8-grams from a directory of benign software
tool.train(
    input_dir="/path/to/benign/software",
    output_dir="/path/to/save/new/bloom",
    ngram_size=8
)

📖 Documentation

Full documentation lives at botacin-s-lab.github.io/AutoPYaraPyPI. It's intentionally basic for now — installation, quick start, and the API reference — with more material (including the accompanying paper, once published) landing there over time.

🧪 Development

pip install -e ".[test]"
pytest tests/

tests/test_core_helpers.py and tests/test_augmented_dbscan.py are pure-Python unit tests (no JVM/network needed). tests/test_smoke_generate.py runs the real pipeline end-to-end against small synthetic dummy files (not real malware) using the built-in Bloom filters.

See RELEASING.md for how versioning and PyPI publishing work.

🤝 Contributing

We're accepting contributions — if you run into an issue or have a fix, fork the repo, open a PR against main, and we'll take a look. PRs are automatically built and tested; once checks pass and a maintainer approves, it gets merged. main itself isn't open to direct pushes from anyone (including maintainers) — everything goes through review. See CONTRIBUTING.md for details.

📄 Citation

If you use AutoPYara in academic work, please cite:

Mabon Ninan*, Nhat Minh Nguyen*, Soumyajyoti Dutta, Sidharth Anil, and Marcus Botacin. "AutoPYara: Next-Gen YARA Rule Generator for Malware Family Clustering." Annual Computer Security Applications Conference (ACSAC 2026), to appear. *Equal contribution. Texas A&M University — {ninanmm, nmnguy29, soumyajyoti1998, sid.anil, botacin}@tamu.edu

@inproceedings{autopyara2026,
  title     = {AutoPYara: Next-Gen YARA Rule Generator for Malware Family Clustering},
  author    = {Ninan, Mabon and Nguyen, Nhat Minh and Dutta, Soumyajyoti and Anil, Sidharth and Botacin, Marcus},
  booktitle = {Proceedings of the Annual Computer Security Applications Conference (ACSAC)},
  year      = {2026},
  note      = {To appear}
}

The AutoYara preset builds on the original AutoYara project (Apache 2.0); if you use it, please also cite:

Edward Raff, Richard Zak, Gary Lopez Munoz, William Fleming, Hyrum S. Anderson, Bobby Filar, Charles Nicholas, and James Holt. "Automatic Yara Rule Generation Using Biclustering." 13th ACM Workshop on Artificial Intelligence and Security (AISec '20), 2020. doi:10.1145/3411508.3421372 · arXiv:2009.03779

See the Architecture page for the full attribution.

License

MIT — see LICENSE.