alakmar344/youAI-2B-From-Scratch-Transformer-Implementation

A complete, end-to-end framework for training a 2-billion parameter Transformer model from scratch. Includes custom architecture, data preparation pipelines, distributed training scripts, and a Flask-based web interface.

0

stars

16

commits

Python

primary language

Jul 15, 2026

updated

README

YouAI

Load, fine-tune, align, merge, evaluate, and serve any open-source LLM — in a few lines of Python.

A professional, batteries-included toolkit: 15 model families, 18 datasets, 12 export formats, DPO/ORPO alignment, LoRA/QLoRA fine-tuning, model merging (DARE/TIES/SLERP), evaluation tools, multi-GPU training, and a production inference server with streaming and batching.


What you can do in 5 lines

import youai

# Load ANY pretrained model — Llama, Qwen, Mistral, DeepSeek, Gemma, Phi, Falcon...
model = youai.from_pretrained("meta-llama/Llama-2-7b-hf")

# Fine-tune with LoRA
youai.train(model, "my_data.txt", epochs=1, lora=True)

# Align with DPO (human preference optimization)
youai.align(model, "preferences.json", algorithm="dpo")

# Merge two models (DARE, TIES, SLERP, linear)
merged = youai.merge_models(base_model, finetuned, method="dare", alpha=0.5)

# Evaluate
results = youai.run_benchmark(model, tokenizer, data_path="test.txt")

# Serve
youai.serve(pretrained="gpt2", port=8000)

New in v3.0 — Alignment, Merging, Evaluation

DPO/ORPO/SimPO alignment training

# Direct Preference Optimization — align with human preferences
youai.align(model, "preference_data.json", algorithm="dpo", beta=0.1)
youai.align(model, "prefs.json", algorithm="orpo")  # no reference model
youai.align(model, "prefs.json", algorithm="simpo")  # length-normalised

Model merging (4 methods)

# Linear merge
merged = youai.merge_models(model_a, model_b, method="linear", alpha=0.7)

# SLERP (spherical interpolation — better for language models)
merged = youai.merge_models(model_a, model_b, method="slerp", alpha=0.5)

# DARE (randomly prune delta, rescale — good for multi-model merge)
merged = youai.merge_models(model_a, model_b, method="dare", density=0.2)

# TIES (trim, elect sign, merge — handles conflicting updates)
merged = youai.merge_models(model_a, model_b, method="ties", density=0.2)

# Model soup (average multiple checkpoints)
merged = youai.model_soup([ckpt1, ckpt2, ckpt3])

Evaluation tools

# Perplexity
results = youai.evaluate_perplexity(model, tokenizer, "test.txt")

# Generation quality (diversity, repetition, coherence)
results = youai.evaluate_generation(model, tokenizer, prompts=["What is AI?"])

# BLEU / ROUGE
bleu = youai.compute_bleu(references, hypotheses)
rouge = youai.compute_rouge_l(references, hypotheses)

# Comprehensive benchmark
results = youai.run_benchmark(model, tokenizer, data_path="test.txt")

Model card generation

youai.generate_model_card(model, "README.md", model_name="my-llama",
                          base_model="meta-llama/Llama-2-7b-hf",
                          metrics={"perplexity": 12.3})

Supported model families (15)

FamilyModelsArchitecture
LlamaLlama-2 7/13/70B, Llama-3 8/70B, CodeLlamaRoPE + RMSNorm + SwiGLU + GQA
QwenQwen2 0.5B/1.5B/7B/72B, CodeQwenRoPE + RMSNorm + SwiGLU + GQA
MistralMistral-7B, Mixtral-8x7BRoPE + RMSNorm + SwiGLU + GQA
DeepSeekDeepSeek 7B/67BRoPE + RMSNorm + SwiGLU
GemmaGemma 2B/7BRoPE + RMSNorm + GELU
PhiPhi-2, Phi-3 MiniRoPE + RMSNorm + SwiGLU
FalconFalcon 7B/40BRoPE + LayerNorm + GQA
YiYi 6B/34BRoPE + RMSNorm + SwiGLU
BaichuanBaichuan 7B, Baichuan2 7BRoPE + RMSNorm + SiLU
InternLMInternLM 7B, InternLM2 7BRoPE + RMSNorm + SwiGLU
MPTMPT 7BALiBi + LayerNorm
StableLMStableLM 3BRoPE + RMSNorm + SwiGLU
StarCoderStarCoder2 3B/7BLearned + LayerNorm + GELU
GPT-2gpt2/medium/large/xl, distilgpt2Learned + LayerNorm + GELU
OpenELMAOpenELMRoPE + RMSNorm + SwiGLU

Why YouAI?

YouAInanoGPTHF TransformersvLLM
Load any model (15 families)1 line5-10 lines
LoRA fine-tune3 lines50+ lines
DPO/ORPO alignment1 lineseparate lib
Model merging (DARE/TIES/SLERP)1 lineseparate lib
Evaluation suitebuilt-inpartial
18 dataset presetspartial
12 export formatspartial
Inference server
Readable source
175 tests

Installation

pip install -e .              # core (torch + transformers)
pip install -e ".[server]"    # + FastAPI inference server
pip install -e ".[accelerate]"# + multi-GPU training
pip install -e ".[all]"       # everything

Testing

pip install -e ".[dev]"
pytest    # 175 tests, runs in seconds on CPU

Documentation

See DOCUMENTATION.md for the full API reference and CHANGELOG.md for the version history.

License

MIT — see LICENSE.

Contributors

alakmar344

16 commits

alakmar344/youAI-2B-From-Scratch-Transformer-Implementation

A complete, end-to-end framework for training a 2-billion parameter Transformer model from scratch. Includes custom architecture, data preparation pipelines, distributed training scripts, and a Flask-based web interface.

0

stars

16

commits

Python

primary language

Jul 15, 2026

updated

README

YouAI

Load, fine-tune, align, merge, evaluate, and serve any open-source LLM — in a few lines of Python.

A professional, batteries-included toolkit: 15 model families, 18 datasets, 12 export formats, DPO/ORPO alignment, LoRA/QLoRA fine-tuning, model merging (DARE/TIES/SLERP), evaluation tools, multi-GPU training, and a production inference server with streaming and batching.


What you can do in 5 lines

import youai

# Load ANY pretrained model — Llama, Qwen, Mistral, DeepSeek, Gemma, Phi, Falcon...
model = youai.from_pretrained("meta-llama/Llama-2-7b-hf")

# Fine-tune with LoRA
youai.train(model, "my_data.txt", epochs=1, lora=True)

# Align with DPO (human preference optimization)
youai.align(model, "preferences.json", algorithm="dpo")

# Merge two models (DARE, TIES, SLERP, linear)
merged = youai.merge_models(base_model, finetuned, method="dare", alpha=0.5)

# Evaluate
results = youai.run_benchmark(model, tokenizer, data_path="test.txt")

# Serve
youai.serve(pretrained="gpt2", port=8000)

New in v3.0 — Alignment, Merging, Evaluation

DPO/ORPO/SimPO alignment training

# Direct Preference Optimization — align with human preferences
youai.align(model, "preference_data.json", algorithm="dpo", beta=0.1)
youai.align(model, "prefs.json", algorithm="orpo")  # no reference model
youai.align(model, "prefs.json", algorithm="simpo")  # length-normalised

Model merging (4 methods)

# Linear merge
merged = youai.merge_models(model_a, model_b, method="linear", alpha=0.7)

# SLERP (spherical interpolation — better for language models)
merged = youai.merge_models(model_a, model_b, method="slerp", alpha=0.5)

# DARE (randomly prune delta, rescale — good for multi-model merge)
merged = youai.merge_models(model_a, model_b, method="dare", density=0.2)

# TIES (trim, elect sign, merge — handles conflicting updates)
merged = youai.merge_models(model_a, model_b, method="ties", density=0.2)

# Model soup (average multiple checkpoints)
merged = youai.model_soup([ckpt1, ckpt2, ckpt3])

Evaluation tools

# Perplexity
results = youai.evaluate_perplexity(model, tokenizer, "test.txt")

# Generation quality (diversity, repetition, coherence)
results = youai.evaluate_generation(model, tokenizer, prompts=["What is AI?"])

# BLEU / ROUGE
bleu = youai.compute_bleu(references, hypotheses)
rouge = youai.compute_rouge_l(references, hypotheses)

# Comprehensive benchmark
results = youai.run_benchmark(model, tokenizer, data_path="test.txt")

Model card generation

youai.generate_model_card(model, "README.md", model_name="my-llama",
                          base_model="meta-llama/Llama-2-7b-hf",
                          metrics={"perplexity": 12.3})

Supported model families (15)

FamilyModelsArchitecture
LlamaLlama-2 7/13/70B, Llama-3 8/70B, CodeLlamaRoPE + RMSNorm + SwiGLU + GQA
QwenQwen2 0.5B/1.5B/7B/72B, CodeQwenRoPE + RMSNorm + SwiGLU + GQA
MistralMistral-7B, Mixtral-8x7BRoPE + RMSNorm + SwiGLU + GQA
DeepSeekDeepSeek 7B/67BRoPE + RMSNorm + SwiGLU
GemmaGemma 2B/7BRoPE + RMSNorm + GELU
PhiPhi-2, Phi-3 MiniRoPE + RMSNorm + SwiGLU
FalconFalcon 7B/40BRoPE + LayerNorm + GQA
YiYi 6B/34BRoPE + RMSNorm + SwiGLU
BaichuanBaichuan 7B, Baichuan2 7BRoPE + RMSNorm + SiLU
InternLMInternLM 7B, InternLM2 7BRoPE + RMSNorm + SwiGLU
MPTMPT 7BALiBi + LayerNorm
StableLMStableLM 3BRoPE + RMSNorm + SwiGLU
StarCoderStarCoder2 3B/7BLearned + LayerNorm + GELU
GPT-2gpt2/medium/large/xl, distilgpt2Learned + LayerNorm + GELU
OpenELMAOpenELMRoPE + RMSNorm + SwiGLU

Why YouAI?

YouAInanoGPTHF TransformersvLLM
Load any model (15 families)1 line5-10 lines
LoRA fine-tune3 lines50+ lines
DPO/ORPO alignment1 lineseparate lib
Model merging (DARE/TIES/SLERP)1 lineseparate lib
Evaluation suitebuilt-inpartial
18 dataset presetspartial
12 export formatspartial
Inference server
Readable source
175 tests

Installation

pip install -e .              # core (torch + transformers)
pip install -e ".[server]"    # + FastAPI inference server
pip install -e ".[accelerate]"# + multi-GPU training
pip install -e ".[all]"       # everything

Testing

pip install -e ".[dev]"
pytest    # 175 tests, runs in seconds on CPU

Documentation

See DOCUMENTATION.md for the full API reference and CHANGELOG.md for the version history.

License

MIT — see LICENSE.

Contributors

alakmar344

16 commits

Languages

Python

100.0%