Dillhaven Assistant

An open-source, customizable in-home voice assistant with a client-server architecture, powered by large language models.

Overview

Dillhaven Assistant is a modular, extensible voice assistant designed to run on a variety of hardware configurations. It features a client-server architecture that allows the resource-intensive components (like speech recognition and language models) to run on more powerful hardware, while lightweight clients can run on devices like Raspberry Pi.

The assistant provides natural voice interaction with configurable personalities, voice synthesis, and integration with language models for intelligent responses.

Features

• Client-Server Architecture: Run the server on powerful hardware and clients on lightweight devices
• Voice Interaction: Speech-to-text and text-to-speech capabilities
• Customizable Personalities: Configure the assistant's name, voice, and personality
• LLM Integration: Connect to language models like Google's Gemma for intelligent responses
• Cross-Platform: Runs on x86_64 and ARM architectures (including Raspberry Pi)
• Modular Design: Composed of multiple specialized crates for different functionalities

Requirements

Server Requirements

• Rust toolchain (latest stable version recommended)
• ONNX Runtime (for machine learning models)
• Audio libraries: libasound2-dev (on Linux)
• Protobuf compiler
• Docker (for building ARM binaries)

Client Requirements

• Rust toolchain
• Audio libraries: libasound2-dev (on Linux)
• Microphone and speakers

Installation

From Source

1. Clone the repository:

   git clone https://github.com/pdylanross/dillhaven-assistant.git
   cd dillhaven-assistant
   

2. Install ONNX Runtime:

   # Example for Linux (adjust for your platform)
   mkdir -p /usr/local/onnxruntime/lib
   # Download and extract ONNX Runtime to the directory above
   

3. Build the server:

   just build-release-server
   

4. Build the client:

   just build-release-client
   

For Raspberry Pi

1. Build the ARM client using Docker:

   just build-release-client-arm
   

2. Deploy to your Raspberry Pi:

   ./hack/local-rpi-deploy/deploy.sh pi@your-raspberry-pi-hostname
   

Configuration

Create a configuration file based on the examples in the hack directory:

personality:
  name: "your-assistant-name"
  system_prompt: |
    System prompt that defines your assistant's personality
  location: "your-location"
  voice_settings:
    local:
      path: "/path/to/voice/model.onnx.json"
llm_client:
  vllm:
    model: "google/gemma-3-4b-it"  # or another compatible model
    base_url: "http://localhost:8000/v1"
    api_key: ""
server:
  addr: "0.0.0.0:4567"
client:
  endpoint: http://your-server-address:4567

Usage

Running the Server

# Development mode
just run-server

# Release mode
just run-release-server

Running the Client

# Development mode
just run-client

# Release mode
just run-release-client

Command-line Options

# Run the server with a specific config file
./target/server/release/dillhaven-assistant server -c /path/to/config.yaml

# Run the client with a specific config file
./target/client/release/dillhaven-assistant client -c /path/to/config.yaml

Project Structure

The project is organized as a Rust workspace with multiple crates:

• dillhaven-assistant: Main application crate with CLI interface
• dillhaven-assistant-api: API definitions and protocol buffers
• dillhaven-assistant-audio: Audio processing utilities
• dillhaven-assistant-client: Client implementation
• dillhaven-assistant-llm: Large Language Model integration
• dillhaven-assistant-observe: Observability and monitoring
• dillhaven-assistant-server: Server implementation
• dillhaven-assistant-speech-to-text: Speech recognition
• dillhaven-assistant-sync: Synchronization utilities
• dillhaven-assistant-text-processing: Text analysis and processing
• dillhaven-assistant-text-to-speech: Voice synthesis
• dillhaven-assistant-types: Common type definitions
• dillhaven-assistant-util: Utility functions

Development

Building

# Format code
just fmt

# Run linter
just clippy

# Run tests
just test

# Build all components in release mode
just build-releases

Contributing

Contributions are welcome! Here's how you can contribute:

1. Fork the repository
2. Create a feature branch: git checkout -b my-new-feature
3. Make your changes
4. Run tests: just test
5. Commit your changes: git commit -am 'Add some feature'
6. Push to the branch: git push origin my-new-feature
7. Submit a pull request

Please make sure your code passes all tests and follows the project's coding style.

License

This project is licensed under the Apache License 2.0—see the LICENSE file for details.