CSCI5540 Machine Learning Systems, Fall 2026

Administrivia

Schedule

• Lectures:
  • Mon 10:30am – 1:15pm, WMY 408
• Tutorials:
  • Mon 1:30pm - 2:15pm, WMY 408

Team

[NOTE]: Due to the large class size, please do not email us individually. Ed should be used for all Q&A.

Ed

The Ed page for this course is here. All communication about this course is done over Ed. This includes questions, discussions, announcements, as well as private messages.

Course outline

This graduate course will introduce you to the key concepts and the state-of-the-art in large-scale software systems for LLMs and agents, and encourage you to think about either building new tools or how to apply existing ones in various domains.

Since datacenters and cloud computing form the backbone of modern computing, we will start with an overview of the two. We will then take a deep dive into three key aspects of the landscape: systems for LLM, systems for agents, and the common infra. Our topics will include: basics on generative models and agentic AI from a systems perspective; systems for LLMs such as pre-training, inference; systems for agents such as post-training, sandbox; and common infra issues such as reliability, simulation; etc. We will cover topics primarily from top conferences that take a systems view to the relevant challenges.

Note that this course is NOT focused on ML/AI algorithms or methods. Instead, we will focus on how one can build software systems so that existing AI methods can be used in practice and new AI methods can emerge.

Prerequisites

Students are expected to have good programming skills and must have taken at least one undergraduate-level systems-related course (from operating systems, databases, distributed systems, and networking). This is not formally enforced but without some background students may find this course quite challenging. Having an undergraduate ML/AI course may be helpful, but not required or necessary.

Textbook

This course has no textbooks. We will read recent papers from top venues to understand trends in scalable GenAI and agentic systems, and their applications.

Tentative Schedule and Reading List

This is an evolving list and subject to changes due to the breakneck pace of agentic and generative AI innovations.

Tutorials

Policies

The format of this course is heavily borrowed from Prof. Mosharaf Chowdhury's CSE 585 from U. Michigan with his consent.

Adverse weather: The University has approved revised arrangements regarding suspension of face-to-face class meetings under adverse weather conditions. Details of the revised arrangements have been included in the updated Postgraduate Student Handbook. In light of this, this course adopts the following weather policy by default: when a lecture or tutorial is affected as a result of adverse weather, we will switch to online teaching using Zoom for that lecture or tutorial at the scheduled time; the paper presentations and discussions will be done in the Zoom meeting. The Zoom detail will be emailed to all students well before the scheduled time. The Zoom session will also be recorded in full so students can review the materials. By registering the course students are aware of and agree to this policy.

Academic honesty: CUHK policy; Engineering faculty policy

Groups

All activities of this course will be performed in groups of 5 students. Remaining students will be grouped randomly. We will take it into consideration during grading for groups with fewer students.

Register your group information here: https://forms.gle/QfSWSvF7NHtutVC56, by 23:59 Sep 13. After the deadline we will add remaining students to the groups randomly. Final grouping information: https://docs.google.com/spreadsheets/d/1SXECiKS-mpcFCk9n-cZX8UQtq69UT6HkRK8D4z_xIhA/edit?usp=sharing.

Online bidding for papers will start on Sep 14 following the finalization of groups.

Required Reading

Each lecture will have two/three required readings that everyone must read.
There will be two or more optional related reading(s) that only the presenter(s) should be familiar with. They are optional for the rest of the class.

Student Lectures

The course will be conducted as a seminar. Only one group will present in each class. Each group will be assigned at least one lecture over the course of the semester. Presentations should succinctly cover all required papers for that lecture. The duration of the presentation should be at most 40 minutes for each paper with short clarifying questions and interruptions. The rest of the lecture time will be dedicated toward discussion on the papers and the broader topic(s) covered by the papers.

In the presentation, you should:

• Provide necessary background and motivate the problem (not restricted by the paper itself).
• Present the high level idea, approach, and/or insight (using examples, whenever appropriate) in the required reading.
• Discuss technical details so that one can understand key details without carefully reading.
• Explain the differences between related works.
• Identify strengths and weaknesses of the required reading and propose directions of future research.

The instructor team will review and suggest improvements for the presentations before each lecture. Therefore, the slides for a presentation must be emailed to the instructor team at least 24 hours prior to the corresponding class. To enable suggestions, use Google Slides and allow the instructor team give in-line comments.

Lecture Summaries

Each group will also be assigned to write summaries for at least one student lecture. The summary assigned to a group will not be the reading they gave the lecture on. The group will write a summary for all presented papers (required readings) for that lecture.

A paper summary must address the following questions in sufficient details (2-3 pages):

• What is the problem addressed in the lecture, and why is this problem important?
• What is the state of related works in this topic?
• What is the proposed solution, and what key insight guides their solution?
• What is one (or more) drawback or limitation of the proposal?
• What are potential directions for future research?
• The discussions between presenters and reviewers during the lecture.

The paper summary must be submitted to https://docs.google.com/forms/d/e/1FAIpQLScedf8xuxkDfueS7x7s3-uSYwtBvu9iBLPAsKB3ax364nvaKw/viewform?usp=publish-editor within 24 hours after the class ends at 1:15pm. Late summaries will not be counted. You should use this format for writing your summary.

Allocate enough time for your reading, discuss as a group, write the summary carefully, and finally, include key observations from the class discussion.

Post-Presentation Panel Discussion

To foster a deeper understanding of the papers and encourage critical thinking, each lecture will be followed by a panel discussion. This discussion will involve three distinct roles played by different student groups (presenters, reviewers, and summarizers), simulating an interactive and dynamic scholarly exchange.

Roles and Responsibilities

1. The Presenters

• Group Assignment: The group that presents the paper will play the role of the paper's authors.
• Responsibility: As authors, you are expected to defend your paper against critiques, answer questions, and discuss how you might improve or extend your research in the future, akin to writing a rebuttal during the peer-review process.

2. The Reviewers

• Group Assignment: Each group will be assigned to one slot to play the role of reviewers for all presented papers (required readings) of that lecture.
• Responsibility: After the presentation, reviewers critically assess the paper, pose challenging questions for the presenters, and highlight potential weaknesses or areas for further investigation. Your goal is to engage in a constructive critique of the paper, simulating a peer review scenario.

3. The Summarizers

• Group Assignment: Listen and document the discussions between presenters and reviewers.
• Responsibility: Write a summary as mentioned above and submit it after the lecture.

4. Rest of the Class

• Responsibility:
  • You are required to ask one insightful question on ed for each presented (required) paper before each class. We will post a discussion thread on ed for each paper.
  • During the panel discussions, feel free to actively ask questions and engage in the dialogue.

Participation

Given the discussion-based nature of this course, participation is required both for your own understanding and to improve the overall quality of the course. You are expected to attend all lectures (you may skip up to 2 lectures due to legitimate reasons), and more importantly, participate in class discussions. There will be random events to gauge attendance.

A key part of participation will be in the form of discussion in Ed. The group in charge of the summary should initiate the discussion and the rest should participate. Not everyone has to add something every day, but it is expected that everyone has something to say over the semester.

Project

You will have to complete substantive work on an instructor-approved problem and have original contribution. Surveys are not permitted as projects; instead, each project must contain a survey of background and related work.

You must meet the following milestones (unless otherwise specified in future announcements) to ensure a high-quality project at the end of the semester:

• Form a group and declare your group's membership by September 14. After this date, we will form groups from the remaining students.
• Prepare a 2-page PDF draft proposal (including references) by September 30. Remember to include the names and CUHK email addresses of the group members. Submission link: https://docs.google.com/forms/d/e/1FAIpQLSchHjxbAwMSOA_ZHPnkWXiW75rCj5a_EO-qXcXRzY1vmFKeOA/viewform?usp=publish-editor.
• Each group must present mid-semester progress during the tutorial hours on November 2.
• Each group must turn in an 8-page final report and your source code via email on or before 1:00PM EST on December 17. The report must be submitted as a PDF file, with formatting similar to that of the papers you've read in the class. It should point to a git repository with all the code along with a README file with a step-by-step guide on how to compile and run the code. Code that cannot run will be treated as a non-submission without any debugging effort from the teaching team. Make sure your instructions are complete.

GPU resources

• The dept has reserved at least 2 servers each with 8 3090 GPUs for this course. Enrolled students have been given access to them already.
• Usage example of the 3090 GPUs (slurm based; cse IP or vpn required)

ssh linux1-15
export SLURM_CONF=/opt1/slurm/gpu-slurm.conf
srun --account gpu --qos csci5540 -p csci5540 --gres=gpu:1 --pty /bin/bash

• The dept also has much more powerful GPUs shared by all students, and many individual faculty members (i.e. your advisor) have their own GPUs for which their students have highest priority using. More info can be found on the internal website (CSE IP required).

Grading