Korean AI Multi-Model Inference (K-AI)

Note: This is a toy project designed to explore and understand the model inference structure of the kai library. It focuses on architectural mapping and experimentation rather than production use.

This project provides a unified inference engine and CLI for South Korea's top Sovereign AI LLM models:

1. K-EXAONE-236B (LG AI Research)
2. Solar-Open-100B (Upstage)
3. A.X-K1 (SK Telecom)
4. VAETKI (NC AI)
5. HyperCLOVAX-SEED-Omni-8B (Naver)
6. HyperCLOVAX-SEED-Think-32B (Naver)

The codebase allows for checking architecture compatibility, creating random weights for testing, and running inference (text generation/chat) through a unified interface.

Supported Models

Architecture Specifications

MoE Models

Dense Models (HyperCLOVA)

Installation

# Install dependencies
pip install -r requirements.txt

# Install the package in editable mode
pip install -e .

Usage

1. Interactive CLI

Run the CLI to interact with a model. You can list available model types:

python cli.py --list-models

To run with a specific model path (e.g., a mini model created for testing):

python cli.py --model-path ./solar_open_mini --model-type solar_open

2. Creating Random Weights (Testing)

Since these models are massive, you can create "mini" versions with random weights to test the architecture and memory flow:

# Create Solar-Open mini model
python create_random_weights.py --model-type solar_open

# Create K-EXAONE mini model
python create_random_weights.py --model-type k_exaone

# Create A.X-K1 mini model
python create_random_weights.py --model-type ax_k1

# Create VAETKI mini model
python create_random_weights.py --model-type vaetki

# Create HyperCLOVA Omni mini model
python create_random_weights.py --model-type hyperclova_omni

# Create HyperCLOVA Think mini model
python create_random_weights.py --model-type hyperclova_think

These will create directories like ./solar_open_mini containing config.json, model.safetensors, and tokenizer files downloaded from HuggingFace.

To skip tokenizer download:

python create_random_weights.py --model-type hyperclova_omni --no-tokenizer

3. Library Usage

You can use the KAIInferenceEngine in your Python code:

from kai.inference import KAIInferenceEngine

# Load engine
engine = KAIInferenceEngine(
    model_path="./k_exaone_mini",
    model_type="k_exaone",
    device="cuda"  # or "cpu", "mps"
)

# Chat interface
response = engine.chat([
    {"role": "user", "content": "안녕하세요! K-AI에 대해 알려주세요."}
])
print(response)

Project Structure

• kai/: Main package
  • core/: Shared components (Attention, MLP, MoE, Norm, RoPE)
  • models/: Model-specific implementations
    • k_exaone/: K-EXAONE 236B (LG)
    • solar_open/: Solar Open 100B (Upstage)
    • ax_k1/: A.X-K1 519B (SKT)
    • vaetki/: VAETKI 112B (NC)
    • hyperclova_omni/: HyperCLOVAX SEED Omni 8B (Naver)
    • hyperclova_think/: HyperCLOVAX SEED Think 32B (Naver)
  • inference/: Unified inference engine
• cli.py: Command-line interface
• create_random_weights.py: Utility to generate test weights