CS 396 Reasoning and Planning in the Foundation Model Era
Winter 2025
Time: Monday and Wednesday 9:30am-10:50am
Location: Technological Institute M164, Over zoom for some external talks, project presentations and discussions
Instructor: Prof. Manling Li (Email: manling.li@northwestern.edu)
TA: Shang Wu (Email: ShangWu2028@u.northwestern.edu)
Instructor and TA Office Hours: Instructor Monday 11:00am-11:30am in-person, Wednesday 11:00am-11:30am in-person, may change to zoom due to travel schedules. TA office hours are on Monday and Wednesday over zoom (Please contact TA ShangWu2028@u.northwestern.edu about it)
Course Google Folder: announced on Canvas.
Assignment Submission: on Canvas: https://canvas.northwestern.edu/courses/225677
Course Summary: This course will explore the core challenge of reasoning and planning in AI, and how foundation models (such as large language models) build up approaches to problem-solving, decision-making, and automated planning. The course bridges theoretical foundations with cutting-edge applications. Topics include chain-of-thought reasoning for complex problem-solving, agent model (including both digital agent and embodied agent), reasoning under uncertainty, and ethical considerations in automated decision-making. Through a combination of lectures, hands-on laboratories, and project work, students will gain experience in:
- Understand fundamental principles of reasoning and planning in AI
- Master agent architectures and their implementation using foundation models
- Develop practical skills in designing and implementing web and embodied agents
- Critically evaluate different agent paradigms and their applications
- Design solutions combining classical planning with modern foundation models
Course Syllabus:
| Week | Date | Lecture |
|---|---|---|
| Week 1 | Topic: | Introduction to Foundation Models and Agent Models |
| 01/06 | From Foundation Models to Agent Models: Environment Interaction + Goal-Driven Decision Making | |
| 01/08 | Long Horizon Reasoning and Planning | |
| Week 2 | Topic: | Chain-of-Thought, Tree-of-Thought and Systematic Reasoning |
| 01/13 | Chain-of-Thought, Instruction Tuning, Tree-of-Thought, Graph-of-Thought | |
| 01/15 | Casual Reasoning | |
| Week 3 | Topic: | Reinforcement Learning |
| 01/20 | No Class due to holiday | |
| 01/22 | Introduction to Reinforcement Learning, RL with feedback | |
| Week 4 | Topic: | Introduction to Scaling Law |
| 01/27 | o1: Time Scaling Law | |
| 01/29 | Project lightning talks |
|
| Week 5 | Topic: | Knowledge Memorization and Reasoning |
| 02/03 | Continued Project lightning talks Deepseek-R1 |
|
| 02/05 | Information Extraction, Knowledge Graph Representation | |
| Week 6 | Topic: | Knowledge-Augmented LLMs |
| 02/10 | Schema Induction, Retrieval-Augmented LMs, Knowledge Editing | |
| 02/12 | Project Pitch and Discussions | |
| Week 7 | Topic: | Advanced Reasoning Topics |
| 02/17 | Knowledge-Driven Reasoning in Embodied Agents (Guest Lecture by Weiyu Liu from Stanford) |
|
| 02/19 | Deepseek R1 Training (Guest Lecture by Weihao Zeng, Junxian He from HKUST) |
|
| Week 8 | Topic: | Agent Models - Fundamentals |
| 02/24 | Reasoning in VLMs (Guest Lecture by Ziqiao Ma from UMich) |
|
| 02/26 | First mid-term exam (Proctored by TAs) in-person |
|
| Week 9 | Topic: | Advanced Topics on Planning and Reasoning |
| 03/03 | LLM Planning in a MDP formulation |
|
| 03/05 | Search-R1 (RL for search), Simple Test Time Scaling, Large Concept Model, Chain-of-Continunous Thought, Future Directions | |
| Week 10 | Topic: | Future Directions |
| 03/10 | Final Project Presentations | |
| 03/12 | Final Project Presentations |
Grading:
- Mid-Term Exams (30pts in total):
- Each exam will be about open-end questions regarding three papers.
- Term Project (70 pts in total, 5pts project proposal, 3pts lightning talk, 12pts mid-term project report, 50pts final project report):
- The instructor will give 10 topics for the students to choose from. Students are expected to do self-teaming and each team should consist of 3-6 students. Everyone is encouraged to submit papers based on the term projects. Project score will by default be the same for all team members, but some team members can get a higher or lower score than the team score based on individual performance that is assessed in two ways: (1) checking contribution to final deliverables (e.g., Git commits and Final Project Report), and (2) Instructor and TAs’ opinion from project presentations.
