Character Classification System

An AI-powered image recognition system designed to identify characters from games and anime.

✨ Features

• Multi-format Recognition: Supports images and videos
• Multi-role Detection: Identify multiple characters in single image (YOLOv8/v10 integration)
• High Accuracy: Powered by MobileNetV2, EfficientNet-B0/B3, ResNet50
• DeepDanbooru Integration: Enhanced tagging capabilities
• Attribute Prediction: Hair color, eye color, clothing attributes
• RESTful API: Batch processing support
• Log Fusion: Build new models from classification logs
• Layered Architecture: Distributed deployment ready
• Model Warm-up: Reduced first-request latency
• Request Debouncing/Throttling: Prevent duplicate submissions
• Image Compression: Optimized upload bandwidth
• Redis Cache: Reduce redundant computations
• Feishu Notifications: Real-time progress updates
• Token Auto-refresh: Seamless authentication

📐 System Architecture

Layered topology: Access (API Gateway) → Business services (API / Model / Multimedia / Search) → Async workers (Inference / Search Worker) → Core AI (Classification / Detection / Recognition / Tagging / Keypoint) → Infrastructure (Redis / MySQL / RabbitMQ / Fluent-bit / Grafana) → Deployment (Supervisord / Docker Compose / Kubernetes).

🚀 Quick Start

Prerequisites

• Python 3.9+
• 16GB+ RAM (required for model loading)
• NVIDIA GPU (recommended for inference speed)
• Redis Server (for caching)
• Docker & Docker Compose (for containerized deployment)

Installation

# Clone the repository
git clone https://github.com/ard-team/anime_role_detect.git
cd anime_role_detect

# Create virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -r requirements-base.txt
pip install -r requirements-ml.txt  # For model training/inference
pip install -r requirements-dev.txt  # For development
pip install supervisor  # For process management

# Configure environment
cp .env.example .env
# Edit .env with your configuration

Running with Supervisor (Recommended)

# Start Redis (required for caching)
redis-server &

# Start all services using supervisord
supervisord -c supervisord.conf

# Check service status
supervisorctl status

# Stop all services
supervisorctl stop all

# linux
nohup ./src/run/sh/run_with_supervisor.sh start >/dev/null 2>&1 &

Docker Deployment

# Build and start all services
docker-compose up --build -d

# Check container status
docker-compose ps

# View logs
docker-compose logs -f <service_name>

# Stop services
docker-compose down

# k8s deployment

# 1. 构建所有镜像
./scripts/k8s/build_k8s_images.sh

# 2. 部署到 K8s（权威源：k8s/base/，详见 k8s/README.md）
kubectl apply -k k8s/base/

# 3. 查看部署状态
kubectl get pods -n anime-role-detect

Service Access

Infrastructure ports: Redis 6379, MySQL 3306, RabbitMQ 5672, fluent-bit 2020 (Docker Compose only).

API Documentation

• Swagger Docs: http://localhost:8080/docs
• Redoc Docs: http://localhost:8080/redoc

Default Credentials

• Username: admin / user
• Password: Set via environment variables ADMIN_PASSWORD and USER_PASSWORD
• Note: Default passwords are auto-generated on first startup if not set

📁 Project Structure

anime_role_detect/
├── src/                    # Source code (editable install: pip install -e .)
│   ├── api/                # Backend API service (FastAPI, port 8001)
│   │   └── routes/         # API routes (classification, auth, collector, search, video,
│   │                       #   cleaning, history, models, onnx_inference, async_inference,
│   │                       #   tracing, version, health, misc)
│   ├── services/           # Microservices
│   │   ├── api_gateway/    # API Gateway (port 8080, aggregates Swagger docs)
│   │   ├── model_service/  # Model Service (port 8000, includes keypoint_worker)
│   │   ├── multimedia_service/  # Multimedia Service (port 8002, video rendering)
│   │   ├── search_service/ # Search Service + worker (port 8003, CLIP+FAISS)
│   │   ├── inference_worker/   # CLIP inference worker
│   │   ├── inference_queue/    # Inference queue manager (Redis/Memory fallback)
│   │   ├── cache_service/  # Redis Cache Service
│   │   ├── model/          # Business model services (classify/recognize/NSFW/multi-model/version)
│   │   ├── processor/      # Model loaders / image processors / preprocessors
│   │   ├── support/        # Database service and support layer
│   │   ├── training/       # Training-related services
│   │   └── notification_service.py  # Feishu notifications
│   ├── core/               # Core capabilities
│   │   ├── classification/ # EfficientNet/MobileNet/DeepDanbooru classification
│   │   ├── detection/      # YOLO multi-role detection + anime_face_detector
│   │   ├── recognition/    # CLIP/ArcFace open-set recognition + feature store
│   │   ├── tagging/        # WD-ViT-Tagger + DeepDanbooru tagging
│   │   ├── keypoint/       # MediaPipe keypoints
│   │   ├── ocr/            # EasyOCR
│   │   ├── feature_extraction/  # Feature extraction (incl. CoreML)
│   │   ├── log_fusion/     # Log fusion
│   │   ├── preprocessing/  # Image/data preprocessors
│   │   ├── config/         # Configuration (ServiceConfig / DeviceManager)
│   │   ├── cache/          # Cache abstractions
│   │   ├── logging/        # Structured logging (loguru JSON)
│   │   └── ...             # error / exception / feedback / version / utils
│   ├── data/               # Data collection / cleaning / augmentation / search index
│   ├── data_pipeline/      # Data cleaning pipeline + active_learning + Streamlit webui
│   ├── data_collection/    # (Legacy) keyword-based collector entry
│   ├── models/             # Database models + training / evaluation / prediction / deployment
│   ├── tasks/              # Celery tasks (classify/image/video/model/cleanup)
│   ├── utils/              # Shared utilities (image, http, concurrency, memory, monitoring, config)
│   ├── middleware/         # HTTP middleware (auth_enhanced / monitoring / tracing)
│   ├── frontend/           # Frontend (Next.js 15 + React 18 + TypeScript App Router)
│   ├── run/                # Service management / monitor dashboard / launch scripts
│   ├── cache/              # HuggingFace / Keras model cache directories
│   └── static/             # Static assets
├── models/                 # Model weights (git-ignored)
├── tests/                  # Test suites (unit / integration / model / workflow / regression / performance / benchmark)
├── docs/                   # Documentation (architecture / deployment / training / blog / testing / technical_challenges)
├── scripts/                # Utility scripts (k8s, monitoring, data_*, model_evaluation, coreml, detection, ...)
│   └── skillhub/           # ⚠️ Archived experiment sub-project (88MB, not referenced)
├── archived/               # Historical / broken modules (spider_image_system, arona, ...)
├── deployment/             # Docker deployment files (11 Dockerfile.* + nginx + grafana)
├── k8s/                    # Kustomize (base/ + overlays/ci/)
├── config/                 # Config templates (config.ini / config.py)
├── supervisord.conf        # Process manager configuration (12 programs)
├── docker-compose.yml      # Docker Compose configuration (13 services)
├── Dockerfile              # Backend Dockerfile (root)
├── Dockerfile.model        # Model Service Dockerfile (root)
├── requirements.txt        # Full dependencies
├── requirements-base.txt   # Base dependencies (for base image)
├── requirements-ml.txt     # ML dependencies
├── requirements-model-service.txt
├── requirements-scripts.txt
├── requirements-dev.txt    # Development dependencies
├── pyproject.toml          # Project configuration (v2.3.0, authoritative version source)
└── .env.example            # Environment template

🌐 API Endpoints

Full route definitions in src/api/routes/. Gateway aggregated docs: http://localhost:8080/docs.

🔧 Configuration

Environment Variables

Docker Configuration

The project includes comprehensive Docker support:

• docker-compose.yml: Multi-service orchestration (13 services) with Redis, MySQL, RabbitMQ, fluent-bit, and all application services
• Root Dockerfile: Backend service image
• Root Dockerfile.model: Model service image
• deployment/: 11 Dockerfiles (base / ml-base / api-service / api-gateway / model-service / multimedia-service / search-service / search-worker / inference-worker / monitoring / frontend) + nginx.conf + grafana dashboard
• Resource limits: model-service 4G/4cpu, others 256M-1.5G (compressed in compose file)

📊 Model Performance

Current Classification Model (v9, EfficientNet-B3)

efficientnet_b3_v9 — EfficientNet-B3 backbone, 167 character classes, canonical preprocessing Resize(288)→CenterCrop(256), evaluated on the held-out test split (post_id-grouped, no train/test overlap). Trained 2026-08-12.

Deployment note: the running backend currently loads efficientnet_b3_v4 (174 classes) by default; v9 is the latest honestly-evaluated checkpoint and is pending promotion to default. Full per-class report: deliverables/gstack/model-baseline-v9-honest-2026-08-12.md.

Deprecated metrics (do not use)

Earlier documentation reported efficientnet_b3 at 51 classes with Top-1 84.00% / 82.65% and Macro-F1 0.8401. These came from a train/test leakage (same images used for both training and testing) and were superseded by the honest 167-class evaluation above. See docs/training/DATA_LEAKAGE_STATUS.md for the leakage analysis.

Multi-role Detection (YOLOv8n)

yolov8n.pt is the COCO-pretrained baseline (6.25 MB, 3.15M params). It is not fine-tuned on anime characters — avg confidence 0.444, ~4 FPS on MPS. Fine-tuning is pending.

Model Comparison (reference)

🔒 Security

• JWT authentication with secret key rotation
• Password hashing with bcrypt/sha256
• Rate limiting to prevent abuse
• Input validation and sanitization
• HttpOnly Cookie storage
• Content Security Policy (CSP) for XSS protection
• Automatic token refresh mechanism

🧪 Testing

Automated Testing

# Run unit tests
python -m pytest tests/ -v

# Run integration tests
python -m pytest tests/integration/ -v

# Run model benchmark (produces scripts/model_evaluation/benchmark_results.json)
python scripts/model_evaluation/run_benchmark.py

📚 Documentation

For detailed documentation:

• docs/architecture/ - Project structure & architecture design
• docs/deployment/ - Deployment guides (Kubernetes, Ubuntu)
• docs/training/ - Model training guides + data leakage analysis
• docs/blog/ - Technical blog posts
• docs/testing/ - Testing guides
• docs/technical_challenges/ - Technical challenges & solutions
• docs/system_design.md / docs/system_design_perf.md - System design & performance optimization plan

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for details on:

• How to submit bug reports and feature requests
• Code style guidelines
• Pull request process

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Version: v2.4.0 | Last Updated: 2026-08-12 | Maintainer: ARD Team

---

Topics: anime, character-recognition, image-classification, deep-learning, python-api, computer-vision, yolov8, nextjs, docker, microservices