---

The Goal

Master AI inference, AI agent harness systems, and hardware engineering — then design a physical AI chip.

The endpoint is a single die that runs production AI workloads, hosts a real agent runtime, and talks to the physical world over Wi-Fi/BLE/Thread — a Jetson-class AI brain fused with an ESP32-class wireless stack on one chip. Designing it takes three working skill sets at once: inference engineering (Qwen-class decode, kernels, quantization, rooflines, multi-GPU serving), agent harness systems (sessions, tools, multi-agent loops, RAG, evals, production observability), and hardware engineering (RTL, embedded Linux, Jetson, ESP32, RF, ASIC flow). This roadmap teaches all three side by side — the chip is the artifact, but the three pillars are the actual work.

---

Who This Is For

• AI/ML engineers who want to stop treating inference as a black box and design the chip that runs it.
• Inference engineers who want to extend up into agent-runtime co-design and down into kernel + silicon.
• Embedded/firmware engineers who want to climb the stack — from boards to runtimes to chip architecture.
• Hardware/RTL/FPGA engineers who need workload and runtime intuition before specing accelerators.
• CS students who want a structured path that ends at "I designed a chip" rather than "I read about chips."

Baseline: comfortable with Python or C++, and basic Linux command-line use. No prior hardware background required.

Already past the basics? Phase 1 and Phase 2 exist to fill specific gaps — digital logic, embedded Linux — not to gate you. If you already know CUDA/C++, go straight to Phase 3 and pick up Phase 1–2 material only where a later module says you need it.

If you only want to call an LLM API, this isn't for you. If you want to design the silicon that calls it, keep reading.

---

Three Ways In

Pick the finish line that matches what you want to be able to do, not a track number.

Each path stands on its own — you don't need the other two to get something real out of it. The Path below is where all three converge into the physical-AI-chip endpoint.

---

The Three Pillars

(Already picked a path above? This section is the deep-dive on why the three connect — skip to The Path if you just want the phase-by-phase breakdown.)

This roadmap is not three unrelated study tracks. It is a co-design loop. The inference workload tells you what the silicon must accelerate, the agent harness tells you what product behavior the runtime must support, and the hardware platform tells you what power, memory, I/O, security, and manufacturing constraints are real.

                               TARGET ARTIFACT
┌────────────────────────────────────────────────────────────────────────────┐
│                         Physical AI Agent Chip                             │
│ Linux-capable CPU + NPU/GPU/DLA + SRAM/DRAM fabric + ISP/audio/sensors     │
│ + Wi-Fi/BLE/Thread/Zigbee + secure boot/OTA + local agent runtime support  │
└────────────────────────────────────┬───────────────────────────────────────┘
                                     │
        ┌────────────────────────────┼────────────────────────────┐
        ▼                            ▼                            ▼
┌───────────────────────┐  ┌───────────────────────┐  ┌───────────────────────┐
│  AI Inference Systems │  │ Agent Harness Systems │  │ Physical HW Platform  │
├───────────────────────┤  ├───────────────────────┤  ├───────────────────────┤
│ Transformer execution │  │ Sessions and memory   │  │ Digital logic + RTL   │
│ GEMV/GEMM kernels     │  │ Tool/skill execution  │  │ CPU/NPU/GPU/DLA arch  │
│ Attention + KV cache  │  │ Gateway/RPC protocols │  │ SRAM/DRAM/DMA fabric  │
│ Quantization formats  │  │ Planner/executor loop │  │ MIPI CSI-2 + ISP      │
│ Tensor/pipeline para. │  │ Multi-agent control   │  │ Audio/GPIO/CAN/I2C    │
│ Serving schedulers    │  │ Evals and guardrails  │  │ Embedded Linux/L4T    │
│ CUDA/Triton kernels   │  │ Telemetry and tracing │  │ ESP32-class wireless  │
│ Roofline profiling    │  │ Product update path   │  │ FPGA -> ASIC flow     │
│ Real models: Qwen etc │  │ OpenClaw/SDKs/APIs    │  │ Board -> SoC thinking │
└───────────┬───────────┘  └───────────┬───────────┘  └───────────┬───────────┘
            │                          │                          │
            └──────────────┬───────────┴───────────┬──────────────┘
                           ▼                       ▼
        ┌────────────────────────────────────────────────────────────────┐
        │ Cross-layer contracts you learn to write                       │
        ├────────────────────────────────────────────────────────────────┤
        │ Workload contract: tokens/s, TTFT, context length, KV memory,  │
        │ batch shape, precision, latency tail, power budget.            │
        │ Runtime contract: APIs, cancellation, scheduling, security,    │
        │ observability, OTA, failure recovery, agent state persistence. │
        │ Hardware contract: MAC array, SRAM size, DMA plan, interconnect│
        │ bandwidth, radio wake events, camera/audio I/O, boot chain.    │
        └────────────────────────────────────────────────────────────────┘

                              CONVERGENCE PATH
┌────────────────────────────────────────────────────────────────────────────┐
│ Jetson-class AI subsystem        -> NPU/GPU/DLA/CPU compute tile           │
│ ESP32-class connectivity         -> Wi-Fi/BLE/Thread/Zigbee radio block    │
│ Camera, audio, and sensors       -> MIPI CSI-2, ISP, codecs, low-power I/O │
│ Agent runtime and serving stack  -> scheduler, memory manager, telemetry   │
│ FPGA/RTL/HLS/MLIR experiments    -> accelerator spec and tape-out target   │
└────────────────────────────────────────────────────────────────────────────┘

1. AI Inference Engineering

The workload your chip will run. This pillar teaches transformer execution from the bottom up: tokenization, embeddings, QKV projection, RoPE, attention, MLP, sampling, KV cache growth, quantization, batching, and serving. You learn why decode is often memory-bandwidth bound, why prefill and decode behave differently, and how GEMV/GEMM kernels, CUDA Graphs, FlashAttention, paged attention, tensor parallelism, and roofline analysis change the system.

Output of this pillar: benchmark reports, kernel experiments, model-memory budgets, quantization choices, and workload contracts precise enough to drive an accelerator architecture.

2. AI Agent Harness Systems

The software stack that lives above your chip. This pillar covers agentic runtimes, session models, gateway RPCs, tool calling, skills, multi-agent loops, RAG, evals, observability, policy controls, and product update flows. A physical AI chip does not just run matmuls. It runs user-facing loops with state, timeouts, cancellations, tool failures, network events, sensor interrupts, and safety constraints.

Output of this pillar: a production-style agent harness with clear runtime interfaces, telemetry, evals, tool boundaries, scheduling requirements, and the operational behavior your chip must support.

3. Physical Hardware Engineering

The substrate itself. This pillar starts with digital design, computer architecture, C/C++ systems work, embedded Linux, Jetson Orin, custom carrier boards, L4T, TensorRT/DLA, ESP32, OpenThread, Zigbee, ESP-Hosted, sensors, camera bring-up, audio, power, thermal, compliance, and manufacturing. Then it moves toward FPGA, HLS, MLIR/compiler work, RTL, SoC architecture, and AI chip design.

Output of this pillar: working boards, bring-up logs, Linux images, wireless integration, sensor pipelines, FPGA/RTL prototypes, and a credible chip spec with compute, memory, I/O, security, and manufacturing constraints.

Why the combination? A chip without a runtime is a brick. A runtime without an agent stack is a benchmark. An agent stack without inference cost discipline is a demo. An inference accelerator that cannot talk to the physical world over wireless, camera, audio, and sensor interfaces is a coprocessor someone else has to integrate. The three pillars are how you build a chip that ships in a real physical product: workload, runtime, radio, sensors, board, compiler, and silicon aligned from the start.

---

What You'll Have at the End

The reason for this roadmap, written as a checklist:

• You can take a transformer model, predict its decode tok/s on a given chip from first principles, and explain where it falls short of the roofline.
• You can hand-tune a CUDA/kernel path for a Qwen-class model — fused QKV, fused gate+up, CUDA Graphs, INT8 KV, speculative decoding — and quote a before/after benchmark.
• You can run a production agent harness end-to-end: gateway, sessions, skills, tool calls, multi-agent supervision, observability dashboards, the lot.
• You can take a Jetson module, design a carrier board for it, bring up custom L4T, flash it in volume, and ship a product against FCC/CE.
• You can bring up an ESP32 wireless coprocessor over SPI, expose it to a Linux host as a Wi-Fi/BLE/Thread/Zigbee radio, and integrate it into the same product.
• You can write RTL, drive timing closure on a real FPGA, and lower a small transformer block through HLS or a custom MLIR dialect.
• You can write the architecture spec for a physical AI agent chip — one die containing an NPU tile (Qwen-class decode at edge power), a wireless subsystem (Wi-Fi 6/BLE 5/Thread/Zigbee), MIPI CSI-2 camera input, ISP, audio I/O, and a Linux-capable CPU — with realistic numbers for tile size, SRAM budget, MAC array, DMA, RF integration, and compiler/runtime interface.

That last bullet is the goal. The first six exist to make it real.

---

The AI Chip Stack

Everything in this roadmap maps onto an 8-layer stack. The point isn't to memorize layers — it's to understand how decisions in one layer ripple through the others.

When you're designing a chip, every layer is a constraint and a degree of freedom. The roadmap teaches you to read the whole column.

---

The Path

Five phases. The first four are foundation; the fifth is where the three pillars converge.

Phase 1 — Digital Foundations (Hardware pillar)

The language of hardware. Logic gates → GPU code.

Phase 2 — Embedded Systems (Hardware pillar)

Get hands-on with real hardware. MCUs, sensors, embedded Linux.

Phase 3 — AI Workloads (Inference & Agent pillars start here)

Understand the workloads your chip must serve. Core + two tracks.

Core (everyone):

• Neural Networks — backprop, CNNs, transformers from first principles
• Transformer Fundamentals — the prerequisite for every inference lecture downstream
• Deep Learning Frameworks — micrograd → PyTorch → tinygrad

Track A — Hardware & Edge AI: Computer vision, sensor fusion, Voice AI, Edge AI & optimization. Feeds Phase 4B and Phase 5C.

Track B — Agentic AI & ML Engineering: 42 lectures on agent harnesses, LangGraph, multi-agent systems, RAG, evaluation, production runtime discipline, OpenClaw, OpenAI Agents SDK, security, plus a Qwen3.5-4B-Base Unsloth fine-tuning course. This is the agent harness pillar in its primary form — read in order if your destination is the chip + runtime + harness story.

Phase 4 — Deployment & Compilation (All three pillars co-exist here)

Take AI to real silicon. Three specialized tracks.

You don't have to do all three. But to land at chip design, you want enough of A to write RTL, enough of B to know what an inference platform looks like, and enough of C to know how a compiler will target your chip.

Phase 5 — Specialization & Convergence

The three pillars converge here. Specialization tracks plus the chip-design endpoint.

The signature path: Phase 1 → Phase 2 → Phase 3 (Core + Track B) → Phase 4 (selected) → Phase 5C + Phase 5F.

The MLSys path: Phase 1 §3/§4 → Phase 3 Core → Phase 4B/4C → Phase 5A/B/C → Phase 5G.

---

Featured Inference Lectures

The deepest, most current technical content lives in these Phase 5 lectures — read them as a single arc:

Read them in order if you're new. Skip to whichever solves your current problem if you're not.

---

How to Use This Roadmap

Don't read this like a book. Treat it like a build-and-measure curriculum.

For every block:

1. Read the theory.
2. Build the subsystem or implement the technique.
3. Measure something — latency, throughput, occupancy, bandwidth, power, accuracy, area, perplexity.
4. Ship one reusable artifact (benchmark, kernel, board, dashboard, RTL block, eval report).

Each artifact is a brick in the chip you're going to design.

Before you start, decide three things:

1. Where you're entering the stack. (See "Who This Is For" above.)
2. What hardware you can actually use. Jetson Orin Nano is the cheapest end-to-end inference target; an RTX or rented L40S/H100 covers most of the datacenter path; a Xilinx Zynq dev board covers FPGA; an ESP32 + sensor breakout covers embedded.
3. How you'll track outputs. A notebook, a benchmarks repo, a project log — any system you actually use.

---

Course Quality Bar

Every serious module in this roadmap should end with evidence, not vibes.

Use this standard for each course block:

Weak completion:

I read about CUDA, TensorRT, and FPGAs.

Strong completion:

I built a TensorRT INT8 benchmark on Orin Nano, captured latency/RAM/power,
compared it to FP16, and explained why one layer stayed memory-bound.

The roadmap is intentionally broad, but the completion standard is narrow: build something real, measure it, and explain the tradeoff.

---

Reference Projects

These projects exist for you to study, not just read about:

---

Target Roles This Enables

The roadmap is full-stack on purpose, but it produces several well-paid specialist roles along the way:

→ See Roles & Market Analysis for salary data, 23 sub-roles, remote percentages, and hiring signals.

---

Why This Roadmap Exists

A physical AI chip — Jetson-class brain + ESP32-class radio + sensors + Linux on one die — is one of the most demanding engineering projects a small team can attempt. It needs:

• Workload truth. You can't design an NPU tile or a memory hierarchy without knowing what bytes-per-token a Qwen-class decode will throw at you. That's the inference pillar.
• System truth. Your chip is going to host runtimes that host agent harnesses in a battery-powered product. Wrong access pattern (batch=1 chat vs always-on wake word vs long-context retrieval), wrong wake-on-radio policy, wrong boot ROM, you've shipped the wrong chip. That's the agent harness pillar.
• Engineering truth — both halves. Silicon doesn't care about your intentions. RTL, timing, power, embedded software, board, antenna, FCC, manufacturing — there's no shortcut. You need the AI-compute side (Jetson stack) and the wireless side (ESP32 stack) on the same die and on the same Linux. That's the hardware pillar.

Most people learn one pillar. Some learn two. This roadmap is for the people who want to learn all three, and then build the thing that puts an AI agent in a real product — talking to sensors, talking to networks, talking to humans, off a single SoC.

---

Star History

---