Learning AI

This repository contains notes and code examples related to AI/ML, with a focus on understanding the fundamentals of large language models, inference engines, and hardware acceleration.

In-progress

• Hailo Hailo-10H AI accelerator (NPU), Raspberry Pi AI HAT+
• Parakeet — Supporting Parakeet in whisper.cpp
• Kimi-Linear
• CUDA FA exploration

---

Table of Contents

Notes

• Model Architectures
• Model Formats & Quantization
• Attention & Embeddings
• Inference & Decoding
• Training & Fine-tuning
• Hardware & Acceleration
• Audio & Speech
• Vision & Multimodal
• Agents & Applications

Code

• Fundamentals
• GPU Code
• NPU Code
• Vector Databases
• Embeddings
• Audio Code
• Agents Code

---

Notes

Model Architectures

• Architectures Overview
  • Transformers
  • RNN
  • LSTM
  • CNN
  • State Space Models (SSM)
  • Mamba / Mamba-2
  • RWKV
  • Linear Attention
  • MatFormer
• Mixture of Experts (MoE)
• KAN
• BitNet
• Autoencoder

Model Formats & Quantization

• GGML
• Llama.cpp
• Model Formats
• Quantization
• GGUF / GPTQ / AWQ / AWQ
• LoRA / QLoRA / QLoRA

Attention & Embeddings

• Attention
• Position Embeddings
• Tokenization
• Word Embeddings
• Normalization
• Softmax / Logits / Logits
• Residual Connections
• Activation Functions
• Loss Functions
• One-Hot Encoding
• Control Vectors

Inference & Decoding

• Sampling
• Speculative Decoding
• Infill
• Grammars
• llguidance
• ChatML / Chat Templates / Chat Templates

Training & Fine-tuning

• Fine-tuning
• DPO
• Reinforcement Learning
• Optimization Algorithms
• LBFGS
• Linear Regression
• Markov Chains
• Flow Matching

Hardware & Acceleration

GPU

• CUDA
• ROCm
• Metal
• Vulkan
• OpenCL
• WebGPU
• WebNN
• MUSA
• eGPU
• Mesa
• HIP
• NCCL
• GPU Overview

NPU / Other

• Hailo
• OpenVINO
• SYCL
• CANN
• CoreML
• WASM / WASI-NN / WASI-NN

Audio & Speech

• Whisper
• Parakeet
• VAD

Vision & Multimodal

• CLIP
• ViT
• LLaVA
• Image Preprocessing

Agents & Applications

• Agents Overview
• RAG
• LangChain
• LLM Chain
• MRKL
• ReAct
• MCP

---

Code

Fundamentals

Exploration code for core AI/ML concepts, libraries, and frameworks.

GPU Code

GPU compute exploration across multiple APIs.

NPU Code

Neural Processing Unit exploration (Hailo).

Audio Code

Audio processing and speech-to-text.

---

---

Notes Index

For a complete list of all notes, see the notes directory.