This course will cover the basics of (1) what LLM-based AI Agents actually are; (2) where they can be useful (and where they are not); and (3) how to safely train and deploy an agent for a given virtual domain.
Students should be familiar with basic CS concepts such as Search (A*, Monte Carlo Tree Search) and Deep Learning concepts such as Transformers (how self-attention works) and the basics of how Large Language Models are (pre-)trained. Students are expected to come into the class with the ability to implement these concepts from scratch (in Python/numpy) and also be able to use popular libraries such as Huggingface. Basic knowledge of Reinforcement Learning (what is a Markov Decision Process, differences between online and offline RL, RL from Human Feedback) is a plus but not required.
Undergrad Intro to AI/RL, and grad level Intro to Deep Learning / NLP types of courses are highly recommended
Office Hours (see the Staff page for office hour locations)
There is no textbook for this course, but you will be required to puchase a varitey of materials including software and API credits. If any of these are prohibitively expensive for your budget, please let the instructor know.
Over the course of the semester, as part of the class participation each final project group must prepare one or more ~15-20 minute presentations on a research paper relevant to the course. Since these presentation will be a substantial component of the learning experience in the class, slides must be prepared and emailed to us at least 72 hours in advance of the lecture they will be presented in (e.g., by 3PM on the Monday before the presenation date), so that we can provide feedback on them. Failure to send us the slides ahead of time will result in a grade penalty on the presentation.
Homeworks are expected to be done individually, and teams of ~5 are needed for the final project.
You may use LLMs for code completion on coding assignments but must submit a CREDITS.txt file noting the exact LLM you used along with prompt. All homeworks will also require explanations written of the code which must be done solely without an LLM. Writing on final projects can be edited but not entirely written with an LLM. Failure to comply with any of these policies will result in a 0 on that particular assignment.
Each student has five free “late days”. Homeworks can be submitted at most two days late. If you are out of late days, then you will not be able to get credit for subsequent late assignments. One “day” is defined as anytime between 1 second and 24 hours after the homework deadline. The intent of the late day policy it to allow you to take extra time due to unforseen circumstances like illnesses or family emergencies, and for forseeable interruptions like on campus interviewing and religious holidays. You do not need to ask permission to use your late days. No additional late days are granted. Late days only apply to the homeworks. They cannot be used on the final project, which must be finished by the final day of class. Late days may not be used for paper presentations.