This repository demonstrates how to use outlines and llama-cpp-python for structured JSON generation with streaming output, integrating llama.cpp for local model inference and outlines for schema-based text generation.
Python
0
2 commits
updated Mar 4, 2025
Install uv: Follow the installation guide.
Sync dependencies: Run uv sync to install the necessary dependencies.
Download models: Execute uv run src/download_models.py to download the required models.
Test streaming output: Run uv run src/generate_exam.py to test the streaming output.
This repository demonstrates how to use outlines and llama-cpp-python for structured JSON generation with streaming output, integrating llama.cpp for local model inference and outlines for schema-based text generation.
Python
0
2 commits
updated Mar 4, 2025
Install uv: Follow the installation guide.
Sync dependencies: Run uv sync to install the necessary dependencies.
Download models: Execute uv run src/download_models.py to download the required models.
Test streaming output: Run uv run src/generate_exam.py to test the streaming output.
