NVIDIA AI-Q Blueprint

🏆 BENCHMARK NOTE 🏆

To obtain results consistent with the nvidia-aiq DeepResearch Bench leaderboard and DeepResearch Bench II benchmark repository results, please use the drb1 and drb2 branches, respectively.

Table of Contents

• Overview
• What's New
• Software Components
• Target Audience
• Prerequisites
• Architecture
• Getting Started
  • Clone the Repository
  • Automated Setup
  • Obtain API Keys
  • Set Up Environment Variables
• Configuration Files
• Ways to Run the Agents
  • Command-line interface (CLI)
  • Web UI
  • Async Deep Research Jobs
  • MCP Server
  • Benchmarks
  • Jupyter Notebooks
• Evaluating the Workflow
  • Available Benchmarks
  • Running Evaluations
• Development
• Roadmap
• Security Considerations
• License

Overview

The NVIDIA AI-Q Blueprint is a deployable research backend built on the NVIDIA NeMo Agent Toolkit and LangChain Deep Agents. Teams can self-host the application boundary and connect deployment-owned models, data sources, authentication, policy controls, storage, and observability. It provides both quick, cited answers and in-depth, report-style research, plus benchmarks and evaluation harnesses for measuring quality. AI-Q is focused on governed research workflows; it is not a general-purpose coding-agent harness.

Key features:

• Orchestration node — One node classifies intent (meta vs. research), produces meta responses (for example, greetings, capabilities), and sets research depth (shallow vs. deep).
• Shallow research — Bounded, faster researcher with tool-calling and source citation.
• Structured deep research — Advisory source routing, structured planning, concurrent researcher workers, bounded source-tool batching, and a dedicated writer produce citation-backed reports and other requested output shapes.
• Report follow-up — Ask questions about a completed report, create a child-job cosmetic rewrite, or run delta research with the parent report as context.
• Workflow configuration — YAML configs define agents, tools, LLMs, and routing behavior so you can tune workflows without code changes.
• Modular workflows — All agents (orchestration node, shallow researcher, deep researcher, clarifier) are composable; each can run standalone or as part of the full pipeline.
• Skills, sandbox execution, and durable outputs — Built-in research/synthesis skills are exposed as host-side, read-only definitions. Skills that invoke code use a provider-neutral sandbox contract; Modal and OpenShell create one physical sandbox per deep-research job. Opt-in rich-file capture checkpoints manifest-declared files after successful sandbox commands, finalizes on success/failure, stores bytes in SQL or S3-compatible storage, and delivers metadata to the Files tab live and on replay.
• Portable Agent Skills — aiq-deploy selects, starts, and validates an AI-Q deployment; aiq-research calls routed chat and async research from compatible coding harnesses.
• Data source registry — UI toggles and request payloads can select web, paper, enterprise, collaboration, and knowledge-layer sources per message.
• Expanded sources — Paper search supports Serper, SerpAPI, and SearchAPI; You.com adds web, contents, general-research, and finance-research tools; Nimble adds configurable web search; focused profiles demonstrate DuckDuckGo news, Polymarket, OpenSearch, and Azure AI Search knowledge retrieval.
• Production API and auth — REST endpoints, async job ownership, per-user OAuth-protected MCP sources, token validator entry points, and provider lifecycle hooks support authenticated deployments; a separate public MCP server exposes stateless research tools for trusted networks.
• Opt-in policy controls — NeMo Guardrails middleware covers selected workflow and agent boundaries, and narrow application-level encryption can protect final async output plus selected artifact-event content.
• Observability, profiling, and cost analysis — NeMo Relay preserves task, named-agent, LLM, and tool hierarchy across interactive turns and async researchers. ATOF feeds local debugging and tokenomics reports; OTEL exports traces to external observability backends.
• Evaluation harnesses — Built-in benchmarks (for example, FreshQA, DeepResearch) and evaluation scripts to measure quality and iterate on prompts and agent architecture.
• Frontend options — Run through CLI, web UI, or async jobs. Refer to Getting started and Ways to run the agents.
• Deployment options - Deployment assets for Docker Compose and Helm; the repository source chart honors the Helm release namespace for every namespaced resource.

What's New

Recent changes include:

• Structured, concurrent deep research — Advisory source routing, structured planning, concurrent researcher workers, bounded source-tool batching, and a dedicated writer replace the earlier three-role flow and move plan ownership out of the clarifier.
• Work that continues from a completed report — Users can ask questions against an existing report, create child-job rewrites, or run delta research with the parent report as context.
• Portable skills, sandboxes, and durable files — Provider-neutral sandbox execution, the new aiq-deploy skill, expanded aiq-research workflows, opt-in artifact capture, SQL or S3-compatible storage, and live or replayed Files-tab access turn generated files into durable outputs.
• Sources, integrations, and policy controls — OpenSearch and Azure AI Search join the knowledge backends; You.com adds four search and research tools, Nimble adds configurable web search, and the standalone public MCP server exposes submit/poll/report operations; per-user MCP OAuth, opt-in NeMo Guardrails middleware, and narrowly scoped async-content encryption add deployment controls without making them universal defaults.
• Operations and user experience — Async traces preserve the agent hierarchy, the source Helm chart honors the selected release namespace, and the UI improves concurrent-research activity, session recovery, and WebSocket reliability.

AI-Q v2.2.0 is published on NVIDIA NGC as the aiq-agent backend container, aiq-frontend web container, and aiq2-web Helm chart. Each artifact uses version 2.2.0.

See the changelog for detailed release history; the linked feature docs describe configuration and current limitations.

Software Components

The checked-in default CLI and web profiles use these core components:

• NVIDIA NeMo Agent Toolkit 1.8.0
• LangChain Deep Agents 0.6.5 or newer
• NVIDIA Nemotron 3.5 Lightning for intent classification and shallow research in the default profiles
• NVIDIA Nemotron 3 Ultra for clarification and every deep-research role
• Google Gemma 4 31B IT (document summary, if used)
• NVIDIA Nemotron 3 Embed 1B (embedding model for the knowledge layer, if used)
• NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning (vision-language model for the LlamaIndex knowledge layer, if used)
• Tavily Search API for web search
• Serper, SerpAPI, or SearchAPI for Google Scholar paper search

Known hosted-serving limitation: Nemotron 3.5 Lightning can intermittently produce citation-incomplete or malformed shallow drafts when served through NVIDIA API Catalog. AI-Q fails closed instead of publishing those drafts. The Brev getting-started launchable uses Nemotron Ultra for shallow research; the general-purpose shipped profiles retain Lightning. See Troubleshooting for details and the self-hosted Lightning option.

The shipped frontier profile, configs/config_frontier_models.yml, uses GPT-5.6 Luna for intent classification, shallow research, source routing, and deep-research execution, with GPT-5.6 Sol for clarification, orchestration, planning, and writing. Treat any bring-your-own model or modified profile as an experimental customization until the complete workflow is evaluated with that exact model, prompt, hyperparameter, tool-calling, and structured-output configuration. Refer to Configuration Files; there is no single all-features profile.

Target Audience

This project is for:

• AI researchers and developers: People building or extending agentic research workflows
• Enterprise teams: Organizations needing tool-augmented research with citation-backed research
• NeMo Agent Toolkit users: Developers looking to understand advanced multi-agent patterns

Prerequisites

• Python 3.11–3.13
• uv package manager
• NVIDIA API key from NVIDIA AI (for NIM models)
• Node.js 22+ and npm (optional, for web UI mode)

Optional requirements:

• A web-search API key for the configured provider: Tavily, Exa, Nimble, or You.com
• A paper search API key for one of the supported providers: Serper (SERPER_API_KEY), SerpAPI (SERPAPI_API_KEY), or SearchAPI (SEARCHAPI_API_KEY)

Note: Configure at least one data source (web search, paper search, or knowledge layer) to enable research functionality.

If these optional API keys are not provided, the agent continues to operate without the corresponding search capabilities. Refer to Obtain API Keys for details.

Hardware Requirements

When using NVIDIA API Catalog (the default), inference runs on NVIDIA-hosted infrastructure and there are no local GPU requirements. The hardware references below apply only when self-hosting models via NVIDIA NIM.

For detailed installation instructions, refer to Installation -- Hardware Requirements.

Architecture

AI-Q uses a LangGraph-based state machine with the following key components:

• Orchestration node: Classifies intent (meta vs. research), produces meta responses when needed, and sets depth (shallow vs. deep) in one step
• Shallow research agent: Bounded tool-augmented research optimized for speed
• Deep research agent: Multi-phase research with planning, iteration, and citation management

Each agent can be run individually or as part of the orchestrated workflow. For detailed architecture documentation, refer to Architecture.

Getting Started

Clone the Repository

git clone https://github.com/NVIDIA-AI-Blueprints/aiq.git && cd aiq

Automated Setup

Run the setup script to initialize the environment:

./scripts/setup.sh

This script:

• Creates a Python virtual environment with uv
• Installs all Python dependencies (core, frontends, benchmarks, data sources)
• Installs UI dependencies (if Node.js is available)

Manual Installation

For selective installation, install packages individually:

# Create and activate virtual environment
uv venv --python 3.13 .venv
source .venv/bin/activate

# Install core with development dependencies
uv pip install -e ".[dev]"

# Install frontends (pick what you need)
uv pip install -e ./frontends/cli          # CLI frontend
uv pip install -e ./frontends/debug        # Debug console
uv pip install -e ./frontends/aiq_api      # Unified API (includes debug)

# Install benchmarks (pick what you need)
uv pip install -e ./frontends/benchmarks/freshqa

# Install data sources (pick what you need)
uv pip install -e ./sources/tavily_web_search
uv pip install -e ./sources/exa_web_search
uv pip install -e ./sources/google_scholar_paper_search
uv pip install -e ./sources/nimble_web_search
uv pip install -e ./sources/you_com
uv pip install -e "./sources/knowledge_layer[llamaindex,foundational_rag]"

Obtain API Keys

Obtain an NVIDIA API Key

1. Sign in to NVIDIA Build
2. Click on any model, then select "Deploy" > "Get API Key" > "Generate Key"

Obtain a Tavily API Key

1. Sign in to Tavily
2. Navigate to your dashboard
3. Generate an API key

Obtain an Exa API Key

1. Sign in to Exa
2. Create an API key from the dashboard
3. Add it to deploy/.env as EXA_API_KEY

Refer to the exa_web_search section in the Configuration Reference for workflow usage.

Obtain a You.com API Key

Follow the You.com quickstart to create an API key and add it to deploy/.env as YDC_API_KEY. Refer to You.com API Suite for tool configuration.

Obtain a Paper Search API Key

Paper search supports three interchangeable providers. Set the provider field on the paper_search function in your workflow config (defaults to serper):

Refer to sources/google_scholar_paper_search/README.md for configuration details.

Set Up Environment Variables

Create a .env file in deploy/ directory:

cp deploy/.env.example deploy/.env

Replace your API keys.

Note: Depending on your usecase, deep research report quality can be enhanced by enabling searching across academic research papers. We use Serper for this. If you want to use paper search, follow the steps in the Customization guide to enable it.

Configuration Files

The configs/ directory holds YAML workflow configs that define agents, tools, LLMs, and routing. Use the one that matches your run mode and data sources:

Ways to Run the Agents

The frontends/ directory contains different interfaces for interacting with the agents. You can also run agents directly through the NeMo Agent Toolkit CLI.

Command-line interface (CLI)

The CLI provides an interactive research assistant in your terminal:

# Activate the virtual environment
source .venv/bin/activate

# Run with the convenience script
./scripts/start_cli.sh

# Verbose logging
./scripts/start_cli.sh --verbose

# Or run directly with the NeMo Agent Toolkit CLI (dotenv loads deploy/.env into the environment)
dotenv -f deploy/.env run nat run --config_file configs/config_cli_default.yml --input "How do I install CUDA?"

The CLI frontend source is in frontends/cli/.

Web UI

For a full web-based experience:

./scripts/start_e2e.sh

This starts:

• Backend API server at http://localhost:8000
• Frontend UI at http://localhost:3000

The web UI source is in frontends/ui/. Refer to frontends/ui/README.md for more details.

Web UI with Docker Compose

You can also run the backend and UI with Docker Compose:

cd deploy/compose

# No-auth local setup (LlamaIndex default)
docker compose --env-file ../.env -f docker-compose.yaml up -d --build

# To select a different backend config, set BACKEND_CONFIG in deploy/.env, for example:
# BACKEND_CONFIG=/app/configs/config_web_frag.yml

For more details, refer to:

• deploy/compose/README.md

Async Deep Research Jobs

For public endpoints, SSE replay, report follow-up, and durable artifact access, refer to the REST API documentation.

MCP Server

Expose AI-Q to MCP clients through the standalone, stateless Streamable HTTP server:

: "${NVIDIA_API_KEY:?Set NVIDIA_API_KEY}"
: "${TAVILY_API_KEY:?Set TAVILY_API_KEY}"
uv sync --project mcp --frozen
AIQ_CHECKPOINT_DB=postgresql://localhost/aiq_jobs \
  uv run --project mcp --frozen aiq-mcp-server

The endpoint defaults to http://localhost:9001/mcp and advertises exactly submit_query, poll_query, and get_final_report. This public server intentionally has no authentication; job UUIDs are bearer capabilities and the endpoint must not be exposed directly to an untrusted network. See Expose AI-Q as an MCP Server for the exact JSON protocol, health contracts, security model, and container deployment.

MCP is an independent uv project with its own mcp/uv.lock. The root lock remains compatible with NAT's cryptography<47 constraint, while the frozen MCP release/container profile pins cryptography==50.0.0 as a security hardening measure. The release-supported platform is Linux x86_64 with CPython 3.13, through either the frozen source project or the release container. Other 64-bit source hosts are development-only; x86_64 macOS and 32-bit Windows are unsupported because cryptography 50 does not publish those wheels. Run the release container on a supported 64-bit Linux/container host. The MCP package's local path dependency closure is not published as a standalone wheel. See the MCP security policy for the full platform contract.

Benchmarks

To run agents in evaluation mode, refer to the Evaluating the Workflow section.

Jupyter Notebooks

The docs/notebooks/ directory contains a three-part series that walks through the blueprint from first run to full customization. Run them in order:

Evaluating the Workflow

The frontends/benchmarks/ directory contains evaluation pipelines for assessing agent performance.

Available Benchmarks

Running Evaluations

Step 1: Install the dataset

The dataset files are not included in the repository. We have included a script to retrieve them from the Deep Research Bench Github Repository and format them for the NeMo Agent Toolkit evaluator.

To download the dataset files, run the following script:

python frontends/benchmarks/deepresearch_bench/scripts/download_drb_dataset.py

Step 2: Generate reports using NAT evaluation harness

dotenv -f deploy/.env run nat eval --config_file frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.yml

Step 3: Convert the output into a compatible format

python frontends/benchmarks/deepresearch_bench/scripts/export_drb_jsonl.py --input <path to your workflow_output.json> --output <path to the output file you want to create with .jsonl extension>

Step 4: Run evaluation

Follow instructions in the Deep Research Bench Github Repository to run evaluation and obtain scores.

Optional: Phoenix Tracing

If your config enables Phoenix tracing, start the Phoenix server before running nat eval.

Start server (separate terminal):

uvx --from arize-phoenix phoenix serve

For detailed benchmark documentation, refer to:

• Deep Research Bench README
• FreshQA README

Development

For development, contribution, and documentation, refer to:

• Development and Contributing: Setup, testing, PR workflow, sign-off/DCO
• Tutorial Notebooks: Getting started overview, deeper dive, and customization notebooks
• Architecture: Component details and data flow
• Customization: Configuration and customization options
• Knowledge Layer Setup: RAG backends and document ingestion
• Agent Skills: Install the portable AI-Q research skill in compatible coding harnesses
• Skills and Sandbox Example: Run deep research with built-in skills and Modal sandbox execution
• Profiling and Cost Analysis: Generate tokenomics and latency reports from Relay ATOF traces
• Docs index: Full documentation list and component docs
• Changelog: Version history and changes

Roadmap

The checkboxes below track implementation in the current branch; a checked item does not by itself indicate availability in a published release.

• NeMo Guardrails Integration: Opt-in middleware for selected workflow, shallow-researcher, and deep-researcher boundaries.
• NVIDIA Dynamo Integration: Reduce latency via priority scheduling at scale.
• Per-user MCP OAuth: Connect each signed-in user to protected MCP data sources through the UI.
• Skills & Sandboxing: Support host-side, role-assigned deep-research skills and provider-neutral execution. Modal and OpenShell create one physical sandbox per deep-research job.
• Report Follow-up and Rewriting: Answer against a completed report, submit cosmetic child rewrites, and run delta research with parent-report context.
• Durable Sandbox Artifacts: Opt-in manifest checkpoints after successful sandbox commands, terminal harvesting on success/failure, SQL or S3-compatible byte storage, and live/replayed Files-tab access.
• Custom Skill Management: Add UI and lifecycle controls for user-provided skill bundles.
• Dynamic Model Routing: Allow sub-agents to automatically select the optimal model per task.
• Resource Management: Implement configurable token caps and tool-call budgets.
• Expanded Web Search: Tavily, Exa, You.com, and Nimble integrations with configurable source selection.
• Additional Search Providers: Add integration examples including Perplexity.
• Multimedia Output: Embed audio, video, and images directly into reports.
• Voice-to-Text Input: Integrate NVIDIA Riva for hands-free accessibility.

Security Considerations

• The AI-Q Blueprint is shared as a reference and is provided "as is". The security in the production environment is the responsibility of the end users deploying it. When deploying in a production environment, please have security experts review any potential risks and threats; define the trust boundaries, implement logging and monitoring capabilities, secure the communication channels, integrate AuthN & AuthZ with appropriate access controls, keep the deployment up to date, ensure the containers/source code are secure and free of known vulnerabilities.
• A robust frontend that handles AuthN & AuthZ is highly recommended. Missing AuthN & AuthZ will result in ungated access to customer models if directly exposed e.g. the internet, resulting in either cost to the customer, resource exhaustion, or denial of service.
• AI-Q includes opt-in NeMo Guardrails middleware for selected workflow and agent boundaries. Guardrails are not enabled universally; operators must attach and test the policies required for their deployment.
• Optional async job content encryption protects final job output and selected artifact-event content only. It is off by default and is not full database-level job-content encryption.
• The AI-Q Blueprint doesn't require any privileged access to the system.
• Deep research skills can invoke sandboxed code execution for analysis workflows. Keep sandbox credentials, quotas, lifecycle cleanup, attestation, and network policy aligned with your deployment's trust boundaries. Shared OpenShell attachment is an explicit debug-only mode and is not job-isolated.
• End users are responsible for ensuring the availability of their deployment.
• End users are responsible for building, and patching, the container images to keep them up to date.
• The end users are responsible for ensuring that OSS packages used by the developer blueprint are current.
• The built-in workflow and agent logger paths redact raw prompts, tool payloads, and model completions. Enabled source adapters, external middleware, model and tool providers, tracing exporters, and reverse proxies can have separate request-logging and retention behavior; audit and configure every enabled component independently, and keep production logs access-controlled.

License

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use, found in LICENSE-THIRD-PARTY.

GOVERNING TERMS: AIQ blueprint software and materials are governed by the Apache License, Version 2.0