PromptForge

Local-first prompt quality scoring and optimization.

Published on PyPI as tuneprompt.

PromptForge scores LLM prompts across seven quality dimensions, then rewrites weak prompts into clear, intent-preserving instructions — runnable on your machine via Python API, CLI, or Gradio.

"Make an app about social media like facebook"
                    │
                    ▼
         ┌─────────────────────┐
         │  Quality Scorer     │  ModernBERT · ~150M
         │  41.5 → issues…     │
         └──────────┬──────────┘
                    ▼
         ┌─────────────────────┐
         │  Prompt Optimizer   │  Qwen2.5-1.5B + LoRA
         └──────────┬──────────┘
                    ▼
   Build a social media app similar to Facebook…
   profiles · feed · likes · constraints · output format
                    │
                    ▼
              41.5 → 94.0

PyPI · Docs · Product plan · Contributing · License

---

Why PromptForge

Most prompt tools either judge quality or rewrite text. PromptForge does both in one local pipeline:

No API key required for inference once models are on disk.

---

Example

Input

Make an app about social media like facebook and stuff

Output (optimizer)

Build a social media app similar to Facebook for product managers.
This is for a portfolio demo.

Core features:
- User profiles and friend connections
- News feed with posts, likes, and comments
- Basic notifications

Requirements:
- Use Python and Flask.
- Keep the first version simple and usable
- Include error handling and clear project structure

Include short examples.

Quality: 41.5 → 94.0 (Δ +52.5) · topic preserved · no fallback

---

Models

Weights are not stored in git. Train locally or download from Hugging Face:

pip install tuneprompt

python -m promptforge download \
  --quality-repo ArjunShukla/PromptForge-Quality \
  --optimizer-repo ArjunShukla/PromptForge-Optimizer

• Quality: https://huggingface.co/ArjunShukla/PromptForge-Quality
• Optimizer: https://huggingface.co/ArjunShukla/PromptForge-Optimizer

---

Results

Quality scorer (held-out):

---

Quickstart

Install

pip install tuneprompt

Package: tuneprompt on PyPI
Import module: promptforge · CLI: tuneprompt or promptforge

Download models & run

python -m promptforge download \
  --quality-repo ArjunShukla/PromptForge-Quality \
  --optimizer-repo ArjunShukla/PromptForge-Optimizer

python -m promptforge init
python -m promptforge doctor
python -m promptforge run "Build me a website for a startup"
python -m promptforge analyze "Make an app." --json

Same via CLI entrypoints:

tuneprompt run "Build me a website for a startup"
# or
promptforge run "Build me a website for a startup"

On some Windows setups, Application Control blocks .venv\Scripts\*.exe. Prefer python -m promptforge ….

Python API

from promptforge import PromptForge

# After download + init, or pass Hub / local paths:
pf = PromptForge(
    quality_model_path="ArjunShukla/PromptForge-Quality",
    optimizer_model_path="ArjunShukla/PromptForge-Optimizer",
)

print(pf.analyze("Make an app."))
result = pf.run("Make an app about social media like facebook and stuff")
print(result["optimized_prompt"])
print(result["delta"]["quality_score"])

Install from source (optional)

git clone https://github.com/arjun988/promptModel.git
cd promptModel

python -m venv .venv
# Windows:  .venv\Scripts\activate
# Unix:     source .venv/bin/activate

pip install -U pip
pip install -e ".[demo,dev]"

GPU tip: default pip install torch is often CPU-only. For NVIDIA (incl. RTX 50-series):

pip uninstall -y torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Full local guide: docs/LOCAL.md

---

Train your own

Anyone can improve the models with their own data:

# Phase 1 — quality scorer
python scripts/train_quality.py --require-gpu --regenerate

# Phase 2 — optimizer (recommended on 8GB GPUs)
python scripts/train_optimizer.py --require-gpu --fast --regenerate

---

CLI

---

Project layout

promptModel/
├── src/promptforge/     # Package: scorer, optimizer, pipeline, CLI
├── configs/             # Training + local defaults
├── scripts/             # Train / eval / Hub export
├── demo/                # Gradio app
├── notebooks/
│   ├── colab/           # Self-contained experiments
│   └── package/         # Thin package drivers
├── docs/                # PRD + local setup
├── tests/
└── pyproject.toml

Notebooks: notebooks/README.md

---

Roadmap

---

Contributing

Issues and PRs welcome. See CONTRIBUTING.md for setup, style, and PR expectations.

pip install -e ".[dev]"
pytest -q

---

License

MIT © PromptForge contributors