Intel® AI for Enterprise RAG

[!IMPORTANT] 📣 Intel® AI for Enterprise RAG is moving to a new home! Version 2.3.0 is the final release published from this repository. Starting with the next release, Intel® AI for Enterprise RAG will continue its development in a new repository - details will be shared soon.

The project isn't going anywhere - it's growing. The team, the roadmap, and our commitment to production-grade enterprise RAG remain fully in place. The move sets us up for the next phase of the project and a stronger foundation for what's ahead.

Stay tuned:

• Watch this repository for the announcement of the new location
• Continue using 2.3.0 with confidence - it's a stable, supported release
• Issues and discussions remain open here until the transition is complete

Intel® AI for Enterprise RAG simplifies transforming your enterprise data into actionable insights. Powered by Intel® Xeon® processors and Intel® Gaudi® AI accelerators, it integrates components from industry partners to offer a streamlined approach to deploying enterprise solutions.

Why use Intel® AI for Enterprise RAG?

Enable intelligent AI experiences that understand your business context:

• Domain-Specific Intelligence - Enrich conversations and document processing with your organizational knowledge without training or fine-tuning models
• Multiple Use Cases - Support for ChatQ&A for conversational AI, Document Summarization for extracting key insights from documents, and AudioQnA for voice-based question answering
• Rapid Deployment - Transform enterprise documents into AI-powered experiences in minutes, not months
• Enterprise-Ready Scale - Deploy secure, compliant AI solutions that grow with your business needs

Core Features

• One-Click Enterprise Deployment - Fully automated Kubernetes cluster provisioning with Ansible playbooks, supporting both single-node and multi-node configurations with comprehensive infrastructure setup.
• Optimized AI Hardware Support - Native support for Intel® Xeon® processors and Intel® Gaudi® AI accelerators with Horizontal Pod Autoscaling (HPA), balloons policy for CPU pinning on NUMA architectures, and performance-tuned configurations.
• Enterprise-Grade Security & Compliance - Integrated Identity and Access Management (IAM) with Keycloak, programmable guardrails for fine-grained control, Pod Security Standards (PSS) enforcement for secure enterprise operations, role-based access control for vector databases, and Intel® Trust Domain Extensions (TDX) support for confidential computing.
• Comprehensive Monitoring & Observability - Integrated telemetry stack with Prometheus, Grafana dashboards, distributed tracing with Tempo, and centralized logging with Loki for full pipeline visibility.

If you're interested in getting a glimpse of how Intel® AI for Enterprise RAG works, check out the following demo.

 

[!NOTE] The video provided below showcases the beta release of our project. As we've transitioned to next releases, users can anticipate an improved UI design, improved installation process along with other enhancements.

Pipeline Architecture

Our system consists of two primary processing pipelines, each built on top of a shared microservices architecture. However, only one pipeline can be deployed at once.

• ChatQnA – enabling retrieval-augmented question answering through conversational interaction.
• Document Summarization (DocSum) – responsible for generating concise summaries from input documents.

The pipeline architecture for ChatQnA is shown below. For the detailed microservices architecture, refer here.

Document Summarization's pipeline architecture is available here.

Table of Contents

1. Intel® AI for Enterprise RAG
   • Why use Intel® AI for Enterprise RAG?
   • Core Features
     • Pipeline Architecture
2. Requirements
   • System Requirements
   • Software Prerequisites
   • Hardware Requirements
     • Deployment on Xeon only
     • Deployment on Xeon + Gaudi Accelerator
3. Getting Started
   • Validate Hardware Requirements
   • Intel AI Enterprise RAG Deployment - Standalone
     • Install a Kubernetes cluster (optional - if you don't have one)
     • Install infrastructure components (storage, operators, backup tools)
     • Deploy the Intel® AI for Enterprise RAG application on top of the prepared infrastructure
4. Intel AI Enterprise RAG Deployment - Partner Solutions
   • Nutanix Enterprise AI & Intel® AI for Enterprise RAG
   • VMware & Intel® AI for Enterprise RAG
5. Documentation
6. Support
7. Publications
8. License
9. Security
10. Intel's Human Rights Principles
11. Model Card Guidance
12. Contributing
13. Trademark Information

Requirements

System Requirements

Software Prerequisites

• Hugging Face Model Access: Ensure you have the necessary access to download and use the chosen Hugging Face model. Default models can be inspected in config.yaml.
• For multi-node clusters CSI driver with StorageClass supporting accessMode ReadWriteMany (RWX) is required. NFS server with CSI driver that supports RWX can be installed via deployment guide.

Hardware Requirements

These are minimal requirements to run Intel® AI for Enterprise RAG with default settings. In case of more(or less) resources available, feel free to adjust the parameters in the resource configuration files for your chosen pipeline:

• ChatQnA: resources-reference-cpu.yaml or resources-reference-hpu.yaml
• Docsum: resources-reference-cpu.yaml or resources-reference-hpu.yaml

Deployment on Xeon only

To deploy the solution using Xeon only, you will need access to any platform with Intel® Xeon® Scalable processor that meet below requirements:

• logical cores: A minimum of 60 logical cores
• RAM memory: A minimum of 128GB of RAM
• Disk Space: 200GB of disk space is generally recommended, though this is highly dependent on the model size

[!NOTE] A limited single-user deployment is also possible on 32 logical cores / 64 GB RAM. See docs/minimum_requirements.md for the required configuration changes.

[!NOTE] By default, Intel® AI for Enterprise RAG uses the NRI plugin for performance optimization. For more info: NRI plugin

Deployment on Xeon + Gaudi Accelerator

To deploy the solution on a platform with Gaudi® AI Accelerator you need to have access to instance with minimal requirements:

• logical cores: A minimum of 56 logical cores
• RAM memory: A minimum of 128GB of RAM though this is highly dependent on database size
• Disk Space: 500GB of disk space is generally recommended, though this is highly dependent on the model size and database size
• Gaudi cards: 8
• Gaudi driver: 1.24.0

Deployment on Xeon + Intel® Arc™ B-Series GPU (Experimental)

[!WARNING] Experimental Support: Intel® Arc™ Pro B-Series GPU (Battlemage/XPU) support is experimental and recommended for evaluation purposes only.

To deploy the solution on a platform with Intel® Arc™ B-Series GPU (XPU) you need to have access to an instance with minimal requirements:

• logical cores: A minimum of 48 logical cores
• RAM memory: A minimum of 128GB of RAM
• Disk Space: 500GB of disk space is generally recommended, though this is highly dependent on the model size
• Operating System: Ubuntu 25.10 (Plucky) for preliminary evaluation

[!NOTE] A limited single-user deployment is also possible on 32 logical cores / 64 GB RAM with minimal_configuration: true. See deployment/README.md for XPU-specific configuration details.

Getting Started

Install the prerequisites.

cd deployment/
sudo apt-get install python3-venv
python3 -m venv erag-venv
source erag-venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
ansible-galaxy collection install -r requirements.yaml --upgrade

Validate Hardware Requirements

Before proceeding with the deployment, it's recommended to validate that your hardware meets the requirements for Intel® AI for Enterprise RAG. To perform hardware validation, you need to create an inventory.ini file first.

An example inventory.ini file structure and detailed instructions are provided in the Cluster Deployment Guide.

Once you have created the inventory.ini file, you can validate your hardware resources using the validate playbook located at playbooks/validate.yaml:

ansible-playbook playbooks/validate.yaml --tags hardware -i inventory/test-cluster/inventory.ini

[!NOTE] If this is a Gaudi deployment, add the additional flag -e is_gaudi_platform=true If this is an Intel® Arc™ B-Series (XPU) deployment, add the additional flag -e is_bmg_platform=true

Intel AI Enterprise RAG Deployment - Standalone

Install a Kubernetes cluster (optional - if you don't have one)

Intel® AI for Enterprise RAG offers ansible automation for creating a K8s cluster. If you want to set up a K8s cluster, follow the Cluster Deployment Guide.

Install infrastructure components (storage, operators, backup tools)

The Intel® AI for Enterprise RAG repository offers installation of additional infrastructure components on the deployed K8s cluster:

• Gaudi_operator - dedicated for K8s clusters with nodes that use Gaudi AI accelerators
• Intel GPU device plugins - for Intel® Arc™ B-Series (XPU) deployments
• CSI drivers - need to dynamically provision storage for PODs
• Velero - installing Velero backup tool

If your K8s cluster requires installing any of these tools, please follow the Infrastructure Components Guide.

[!NOTE] For Intel® Arc™ B-Series (XPU) deployments, use -e is_bmg_platform=true with infrastructure and application playbooks. See deployment/README.md for detailed XPU configuration instructions.

Deploy the Intel® AI for Enterprise RAG application on top of the prepared infrastructure

Once you have a K8s cluster with all infrastructure components installed, you can install the Intel® AI for Enterprise RAG application on top of it. Please follow the Application Deployment Guide.

Intel AI Enterprise RAG Deployment - Partner Solutions

Nutanix Enterprise AI & Intel® AI for Enterprise RAG

Nutanix Enterprise AI Deployment Guide.

VMware & Intel® AI for Enterprise RAG

VMware Deployment Guide.

Documentation

Refer to deployment/README.md or docs for more detailed deployment guide or in-depth instructions on Intel® AI for Enterprise RAG components.

Support

Submit questions, feature requests, and bug reports on the GitHub Issues page.

Publications

Feel free to checkout articles about Intel® AI for Enterprise RAG:

• How to Integrate SharePoint Online with a RAG system
• Lenovo Validated Design: AI POD Mini for Enterprise RAG Implementation
• Give Your RAG a Voice: Building an Audio Q&A Experience with Intel® AI for Enterprise RAG
• Accelerate AI Value Creation with Nutanix and Intel® AI for Enterprise RAG
• Converging Paradigms: Architecting a Hybrid and Open Platform for Unified HPC and AI Workloads
• Starting With the End in Mind: Intel and Nutanix’s Blueprint for an Enterprise-Grade RAG Chatbot
• Securing Enterprise RAG Deployments
• Document Summarization: Transforming Enterprise Content with Intel® AI for Enterprise RAG
• Scaling Intel® AI for Enterprise RAG Performance: 64-Core vs 96-Core Intel® Xeon®
• Comprehensive Analysis: Intel® AI for Enterprise RAG Performance
• Monitoring and Debugging RAG Systems in Production
• NetApp AIPod Mini – Deployment Automation
• Multi-node deployments using Intel® AI for Enterprise RAG
• Rethinking AI Infrastructure: How NetApp and Intel Are Unlocking the Future with AIPod Mini
• Deploying Scalable Enterprise RAG on Kubernetes with Ansible Automation
  

---

License

Intel® AI for Enterprise RAG is licensed under the Apache License Version 2.0. Refer to the "LICENSE" file for the full license text and copyright notice.

This distribution includes third-party software governed by separate license terms. This third-party software, even if included with the distribution of the Intel software, may be governed by separate license terms, including without limitation, third-party license terms, other Intel software license terms, and open-source software license terms. These separate license terms govern your use of the third-party programs as set forth in the "THIRD-PARTY-PROGRAMS" file.

Please note: component(s) depend on software subject to non-open source licenses. If you use or redistribute this software, it is your sole responsibility to ensure compliance with such licenses.

Security

The Security Policy outlines our guidelines and procedures for ensuring the highest level of security and trust for our users who consume Intel® AI for Enterprise RAG.

Intel's Human Rights Principles

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model Card Guidance

You, not Intel, are responsible for determining model suitability for your use case. For information regarding model limitations, safety considerations, biases, or other information consult the model cards (if any) for models you use, typically found in the repository where the model is available for download. Contact the model provider with questions. Intel does not provide model cards for third party models.

Contributing

If you want to contribute to the project, please refer to the guide in CONTRIBUTING.md file.

Trademark Information

Intel, the Intel logo, OpenVINO, the OpenVINO logo, Pentium, Xeon, and Gaudi are trademarks of Intel Corporation or its subsidiaries.

Other names and brands may be claimed as the property of others.

© Intel Corporation