Book Reader with Local LLM

A C# application that uses local language models to summarize books and answer questions about them. This project demonstrates how to combine EPUB book parsing with local language models for intelligent document analysis.

Features

• Load EPUB Books: Parse and extract text content from EPUB files
• Chapter Summarization: Generate concise summaries of individual chapters
• Full Book Summarization: Create comprehensive summaries of entire books
• Interactive Q&A: Ask questions about specific chapters or the entire book
• Local Processing: All processing happens locally using LlamaSharp with no external API calls

Prerequisites

• .NET 10.0 SDK
• A Llama model file (e.g., Llama-3.2-3B-Instruct-Q4_K_M.gguf)
• EPUB book files

Setup

1. Place your EPUB books in the books/ directory
2. Ensure you have a Llama model in the ../models/ directory
3. The default configuration uses:
   • Book: books/meditiations.epub
   • Model: ../models/Llama-3.2-3B-Instruct-Q4_K_M.gguf

Running the Application

cd BookReader
dotnet run

Usage

The application presents an interactive menu with the following options:

1. Summarize a specific chapter: Get a concise summary of any chapter
2. Summarize the entire book: Generate a comprehensive book summary (processes first 10 chapters)
3. Ask a question about a chapter: Query specific chapter content
4. Ask a question about the entire book: Query the book using context from multiple chapters
5. List chapters: View all available chapters/sections
6. Exit: Close the application

Configuration

You can modify the following settings in Program.cs:

• bookPath: Path to your EPUB file
• modelPath: Path to your Llama model file
• Context size: Adjust in LlamaService.cs (default: 8192 tokens)
• GPU layers: Set GpuLayerCount in LlamaService.cs if GPU acceleration is available

How It Works

EPUB Processing

• Uses the VersOne.Epub library to parse EPUB files
• Extracts text content from HTML chapters
• Cleans HTML tags and decodes entities
• Preserves chapter structure for navigation

LLM Integration

• Uses LLamaSharp for local model inference
• Implements Llama 3.2 chat format with proper system/user/assistant tags
• Supports both summarization and question-answering tasks
• Manages context windows to fit within model limits

Text Processing

• Chunks large texts to fit within context windows
• Combines chapter summaries for book-level understanding
• Optimizes prompts for better model responses

Dependencies

• LLamaSharp: Interface to llama.cpp for running local models
• LLamaSharp.Backend.Cpu: CPU backend for model inference
• VersOne.Epub: EPUB file parsing library

Performance Notes

• Model loading takes 10-30 seconds depending on model size and hardware
• Chapter summarization: ~10-30 seconds per chapter
• Full book summarization processes the first 10 chapters for demonstration
• Context size is set to 8192 tokens (adjust based on your RAM availability)
• CPU-only mode by default (configure GPU layers for faster inference if available)

Extending the Application

Consider adding:

• Book library management
• Save/load summaries and Q&A sessions
• Export functionality for summaries
• Support for other document formats (PDF, DOCX)
• Streaming output for real-time response generation
• RAG (Retrieval Augmented Generation) for better question answering
• GPU acceleration support

Troubleshooting

Model loading fails: Ensure the model path is correct and the file is a valid GGUF format

Out of memory errors: Reduce the context size in LlamaService.cs

Slow performance: Consider using a smaller model or enabling GPU acceleration

Book not found: Verify the EPUB file is in the books/ directory

License

This is a demonstration project for educational purposes.