⚡ Qwen DFlash2 Ghost Writer Autocomplete

A fast local autocomplete tool for prose and code powered by speculative decoding with lightweight draft models.

---

🚀 Quick Setup & Installation

Follow these standard commands in your terminal to set up the environment and launch the app:

# 1. Install required Python packages
pip install -r requirements.txt

# 2. Pre-download target & draft model weights via single Python CLI command
python -c "from transformers import AutoModelForCausalLM, AutoTokenizer; TARGET='Qwen/Qwen3.8-27B-Instruct'; DRAFT='incoai/Qwen3.8-27B-DFlash2'; print('Fetching Tokenizer...'); AutoTokenizer.from_pretrained(TARGET); print('Fetching Target 27B...'); AutoModelForCausalLM.from_pretrained(TARGET, torch_dtype='auto'); print('Fetching Draft Model...'); AutoModelForCausalLM.from_pretrained(DRAFT, torch_dtype='auto'); print('Download Complete!')"

# 3. Launch the application
streamlit run app.py

---

🛠️ Technology Stack Breakdown

• Primary Interface: Streamlit responsive dark dashboard.
• Language Model Backbone: Hugging Face Transformers (AutoModelForCausalLM).
• Speculative Acceleration: assistant_model parameter pairing target models with draft models.
• Runtime Environment: PyTorch with automatic CUDA device mapping.

---

📁 Repository Structure

qwen-dflash2-ghost-writer/
├── app.py
├── requirements.txt
└── README.md

---

🎯 5 Core Use Cases

1. Local Prose Autocomplete: Write long-form articles and documentation locally without active cloud subscriptions.
2. Real-Time Code Completion: Fast single-pass token drafting for Python scripts, functions, and shell automation.
3. Low-Latency Chat Interface: Eliminate waiting times during streaming response generations.
4. Offline Markdown Synthesis: Draft technical reports with instant prose summaries and bullet points.
5. Speculative Decoding Research: Test and benchmark draft model acceptance rates across different domain tasks.

---

🔮 5 Future Enhancement Features

1. Multi-Draft Selection: Integrate dynamic beam candidate selection across multiple draft tokens.
2. Custom Shortcut Hotkeys: Bind custom keyboard triggers like Tab and Ctrl+Space for instant completion.
3. IDE Plugin Extension: Package the engine as a lightweight VS Code extension backend.
4. Quantized Draft Weights: Support GGUF and AWQ draft model weights for minimal VRAM usage.
5. Context History Cache: Add persistent local context caching for long document drafting sessions.

---

🏷️ Keywords & SEO

Qwen 3.8 DFlash2 Speculative Decoding 27B Model 236 TOK/S Local AI Streamlit Transformers Hugging Face PyTorch Fast Inference Code Completion Ghost Writer