Conversational AI Evaluation Tool - v2.0.1

1. Overview

AIEvaluationTool is an end-to-end platform for evaluating conversational AI systems across API, web, and WhatsApp-style interfaces.

It combines:

• TDMS for managing test data (test cases, plans, prompts, strategies)
• Test Case Execution Dashboard for running evaluations, tracking runs, and reviewing results
• CLI workflow for importer, execution, analysis, and report generation

For full details, refer to the official documentation.

2. Architecture And Design

TDMS and the Dashboard run as a unified application layer behind a single Docker gateway.

3. Configuration

Docker reads runtime values from .env and application-level configuration from config.json.

3.1 Create Environment File

cp .env.example .env

3.2 Required Files

• Root .env
• Root config.json
• src/app/interface_manager/config.json (for browser automation settings)

3.3 XPath Configuration For Web App Automation

For web/WhatsApp targets, configure element selectors in:

• src/app/interface_manager/xpaths.json
• src/app/interface_manager/credentials.json

Use stable relative XPath values for login fields, prompt input areas, response containers, and logout elements.

Detailed configuration guide: Docker Setup and Configuration

4. Getting Started With Docker

4.1 Build And Start

docker compose up -d --build nginx

For the Docker development stack, use:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build

4.2 Open The Application

• TCE UI: http://localhost:${NGINX_PORT:-80}/
• TDMS UI: http://localhost:${NGINX_PORT:-80}/tdms/
• Selenium live view: http://localhost:${NGINX_PORT:-80}/selenium/
• Health: http://localhost:${NGINX_PORT:-80}/healthz

4.3 Stop Or Reset

docker compose down
# full reset
docker compose down -v

4.4 Other Run Modes

• Docker CLI flow: Docker Run CLI
• Docker UI flow: Docker Run UI
• GPU model setup: Docker GPU Setup
• Non-Docker/local setup: UI Setup
• Full docs portal: AIEvaluationTool Documentation

5. How The Project Came To Life

Made with Gource