Diffusion Data Tools

A collection of scripts for scraping, downloading, and processing data for diffusion models.

Setup Instructions

1. Create Virtual Environment

python -m venv venv

2. Activate Virtual Environment

.\venv\Scripts\activate

3. Install Core Dependencies (CUDA Support)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

4. Install Remaining Requirements

pip install -r requirements.txt

Available Scripts

• caption_promptgen.py: Generate captions and tags for images using Florence-2 models.
• fancap_scraper.py: Scrape images from fancap.
• downloader.py: Download images from gallery pages.
• single_img_folder.py: Organize images into single-image folders.
• playwright_scraper.py: Advanced scraper using Playwright and stealth mode.