⚡ Compilers and Runtimes for AI: From Prompts to Accelerators

EECS 598 · Fall 2025

Cutting Edge and Emerging Technologies from the Programming Interface down to Hardware Acceleration of AI

---

📖 Course Summary

The science and art of creating efficient AI systems spans the entire computing stack—from high-level language abstractions down to specialized hardware accelerators. This course provides a comprehensive exploration of AI compiler and runtime techniques, covering everything from language-level AI compiler systems (DSPy, SGLang, MTP, Guidance, LMQL) to hardware-level acceleration (GPU kernels, TPU compilation, custom ASICs, and emerging AI chips).

Students will learn how modern AI compilers and runtime systems like PyTorch, JAX, TVM, TensorRT, VLLM, and specialized LLM compilers orchestrate the full pipeline from prompt engineering and program synthesis down to optimized execution on heterogeneous hardware. The course covers the complete spectrum: prompt-level optimizations, graph-level transformations, kernel-level tuning, memory hierarchy optimization, and distributed system coordination.

What You’ll Do

• Explore language-level AI compiler techniques (prompt optimization, program synthesis, declarative AI programming) and traditional compiler optimizations (graph-level transformations, kernel tuning, memory management, distributed training)
• Work hands-on with cutting-edge AI compiler ecosystems (DSPy, SGLang, MTP, Guidance, LMQL, MLIR, TVM, PyTorch, CUDA) and heterogeneous hardware platforms (GPUs, TPUs, custom accelerators, emerging AI chips)
• Identify and focus on a specific research project within the scope of these technologies, demonstrating novel compiler/runtime optimizations for targeted AI workloads
• Present a capstone project that delves deep into a particular aspect of AI compilation, from language-level innovations to hardware-level breakthroughs

Projects teams will be on the smaller side (~2–3 students) and will include selecting a focused research direction, designing targeted optimization approaches, building specialized compiler/runtime components, and benchmarking performance improvements.

What You’ll Learn

• The comprehensive landscape of AI systems: from language-level AI compiler techniques (DSPy, SGLang, MTP) to hardware-level acceleration (GPU kernels, TPU compilation, custom ASICs)
• State-of-the-art techniques: prompt-level optimization, program synthesis, graph-level transformation, auto-tuning, quantization, inference acceleration, and emerging AI chip architectures
• Critical research skills: interpreting papers, evaluating cutting-edge systems, presenting technical ideas, and bridging the gap between high-level AI programming and low-level hardware optimization

Grading is research project-centric. You’ll showcase your project’s evolution through presentations, paper reviews, and final demos.

---

👨‍🏫 Instruction Team

• Instructor: Jason Mars (📧 profmars@umich.edu)
• GSI: TBD

---

🗓 Logistics

• Lecture: TBD
• Credits: 4
• Office Hours: On Demand
• GSI Office Hours: TBA
• Course Discussion: Piazza (TBD)
• Canvas: TBD
• Recorded Lectures: Available on Canvas

---

📅 Schedule

---

📂 Tentative List of Papers

Need to add some compound AI papers, Graphine, and a few others

---

📊 Grading

This is a very 'do based' course, we'll be learning, creating, innovating and sharing. A significant portion of the grade is allocated to the research project. Most (if not all) does very well in this class as long as you stay engaged on the journey. Lets create some amazing stuff! 😊

Research Project: 80%

• Project Pitch: 10%
• Project Update: 10%
• Video Lightning Talk: 15%
• Final Presentation: 20%
• Paper Write-Up: 25%

Participation, Impact, and Engagement: 20%

• Paper Presentations: 10%
• Repo Deconstructions: 5% (Project Details)
• Paper Vibe Logs: 5% / #No of Papers

---

🎯 How We'll Work

This course emphasizes collaborative learning and knowledge sharing. Students will actively participate in presentations that showcase their understanding and discoveries from the course materials.

Grading and Logistics

Grading Basis

• Grading will be based on each student's GitHub repository for this class. Your repo is the official record of your work and deliverables.

Repository Registration

• Each student must enter their U-M uniquename and GitHub repository link in the "GitHub repos" tab of the same Google Sheet used for student presentations (https://docs.google.com/spreadsheets/d/1y7yw2zQt6hjsVg0bTg0fLS1cplkk9_-1nd-qYPybwfE/edit?usp=sharing).

Paper Vibe Logs

• Upload your paper vibe logs to your GitHub repository.
• Vibe logs are due the same day as the paper being presented.
• Each student submits their own vibe log (no group submissions).
• Each vibe log should contain about 5–10 questions you asked to GPT about the paper being presented.

"Vibe Learning" Log

A key component of our learning methodology is the "vibe learning" presentation. In these sessions, students will:

• Walk through AI conversation logs: Present a curated log of their conversations with AI systems where they learned interesting concepts from the course papers
• Teach back to the class: Use their AI interaction logs as a foundation to explain complex compiler and runtime concepts to their peers
• Demonstrate understanding: Show how they've internalized and can communicate technical concepts through their AI-assisted learning journey
• Share insights: Highlight unexpected discoveries, connections, and "aha moments" that emerged from their AI conversations

This approach leverages the power of AI as a learning companion while ensuring students develop deep understanding through the process of teaching others. Students will learn not just from the papers themselves, but from each other's unique learning paths and AI-assisted discoveries.

Student Presentations

Students should sign up for a paper here (in the paper signup tab): https://docs.google.com/spreadsheets/d/1y7yw2zQt6hjsVg0bTg0fLS1cplkk9_-1nd-qYPybwfE/edit?usp=sharing

Presentation Slides

• Aim for 15–20 slides
• Cover the paper’s motivation, problem, key ideas/methodology, system/architecture, evaluation/results, limitations/trade-offs, and key takeaways
• Include relevant figures/diagrams/tables from the paper (or simplified redraws) to clearly tell the story
• Use a clear narrative: problem → idea → how it works → why it’s better → evidence → implications
• For style and pacing, watch recent paper talks on YouTube from top architecture/systems conferences such as ISCA, MICRO, ASPLOS, and HPCA

---

📝 Final Paper

Your research project culminates in a conference-style paper that documents your work, findings, and contributions.

Requirements

• Format: Standard CS conference paper format (e.g., ACM or IEEE style)
• Length: 5–8 pages (excluding references)
• Tools: Use Overleaf (recommended for collaboration) or local LaTeX
• Due Date: December 15th, 2025
• Submission: Push final PDF to your GitHub repository

Recommended Templates

• ACM: Use the acmart template with sigconf option — Overleaf ACM Template
• IEEE: Use the IEEE Conference template — Overleaf IEEE Template

Paper Structure

Your paper should include:

1. Abstract (~150–250 words): Concise summary of problem, approach, and key results
2. Introduction: Motivation, problem statement, and contributions
3. Background/Related Work: Context and how your work relates to existing research
4. Approach/Methodology: Technical details of your solution
5. Implementation: System architecture, tools used, key design decisions
6. Evaluation: Experiments, benchmarks, and results with analysis
7. Discussion: Limitations, lessons learned, future work
8. Conclusion: Summary of contributions and impact
9. References: Properly formatted citations

Tips for First-Time Paper Writers

Writing your first research paper can feel daunting. Here are practical tips to help you succeed:

Getting Started

• Start early: Don't wait until the last week. Good papers require multiple drafts and revisions
• Read papers in your area: Notice how they structure arguments and present results — mimic what works
• Write the easy parts first: Start with methodology or implementation sections where you describe what you built

Writing Advice

• Be precise: Avoid vague language like "very fast" or "much better" — use specific numbers and comparisons
• One idea per paragraph: Each paragraph should have a clear topic sentence and support one main point
• Use active voice: "We implemented X" is clearer than "X was implemented"
• Define terms before using them: Don't assume readers know your acronyms or jargon
• Figures tell stories: A good diagram can replace paragraphs of text — invest time in clear visuals

Common Mistakes to Avoid

• Don't oversell: Be honest about limitations; reviewers appreciate intellectual honesty
• Don't skip related work: Show you understand the landscape and where your work fits
• Don't bury the lede: State your main contribution clearly in the introduction
• Don't forget baselines: Always compare against something — a naive approach, prior work, or standard benchmarks
• Don't submit without proofreading: Typos and grammar errors undermine credibility

The Revision Process

1. First draft: Get ideas on paper without worrying about perfection
2. Self-review: Read your paper aloud — awkward phrasing becomes obvious
3. Peer review: Have teammates or classmates read it and provide feedback
4. Final polish: Check formatting, fix citations, ensure figures are readable

Useful Resources

• How to Write a Great Research Paper — Simon Peyton Jones (Microsoft Research)
• Writing Technical Articles — Henning Schulzrinne (Columbia)

---

🛠 Notes

• https://medium.com/geekculture/ai-compilers-ae28afbc4907
• https://developer.nvidia.com/blog/understanding-ptx-the-assembly-language-of-cuda-gpu-computing/
• https://rocm.blogs.amd.com/software-tools-optimization/amdgcn-isa/README.html

---

⭐ Prepare to build the next generation of compilers and runtimes for AI.