DS677.005 - Deep Learning (FALL · 2024)

• Description
• Teaching Team
• Logistics
• Materials
• Schedule

Description (DS677.005)

Machine Learning develops computer programs that can improve their performance by tapping into existing data and taking feedback from the environment. Systems based on ML have already exceeded human performance in several tasks, including image classification and games like Chess and Go. ML has also made leaps in even more complicated tasks, like Natural Language Processing or self-driving vehicles, and it has even produced art that imitates the style of human artists! This course offers an intense introduction to the fundamental ML concepts and algorithms that constitute the core of these spectacular developments. It takes you on a tour from the basic mathematical notions and algorithms to some of the recent developments, e.g., deep convolutional neural networks, recurrent neural networks, and transformers. You will gain exposure to cutting-edge ML development tools such as Jupyter notebooks, scikit-learn, and PyTorch via hands-on assignments and projects that will instill a working and immediately applicable knowledge of ML methods and will prepare you for more advanced ML courses.

Teaching Team

Logistics

• Time: Friday, 1:00 PM - 3:50 PM EST (FALL 2024）
• Location: ECEC 100
• Mode: Face-to-Face
• Discussions: Feel free to participate in different discussion forums on Canvas
• Quizzes & Assignments: submit and post on Canvas
• Prerequisites: CS675 or DS675 or instructor permission.
• Contact: For external inquiries, personal matters, or emergencies, please email the instructor or TA directly.

Materials

There is no required course textbook. The course will draw material from several sources, including the instructor’s own notes. Some open resources include:

• Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola, Dive into Deep Learning
• Prince, S.J. Understanding Deep Learning. MIT press. (Dec 5, 2023)

Grading

• Quizzes [10%]: Weekly Canvas quizzes reinforcing the material of each module, will help you keep up with the most important theoretical concepts. These quizzes are not proctored.
• Class Participation [10%]: You are expected to attend classes and participate in classes by listening and understanding class contents and asking related questions. You are also expected to participate in weekly Canvas discussions prompted by the instructor, with meaningful questions and answers related to the week’s topics or assignments.
• Homeworks [30%]: Assignments will be given biweekly to give you an opportunity to apply course concepts for that week. Four homework assignments of equal grading weight.
• Mid-term Exam [15%]: In-person exam, 90 minutes. Students are expected to bring a fully charged laptop, as the exam will be on Canvas with LockDown browser. Each student is allowed to bring at most 5 pages of notes. In the event the exam has to take place online, Respondus Monitor will be used for proctoring.
• Project [35%]: The project will consist of three milestones, with weights [10%, 5%, 20%]. You will have opportunities to iterate and revise your work based on peer, TA, and instructor’s feedback.

Schedule

The lecture slides are available for download in the Google Drive folder, while all weekly quizzes and biweekly assignments can be accessed on Canvas.

Week	Date	Topic	Textbook Readings	Due Work
Week 1	09/06	Introduction and course overview	chapter 2, chapter 3
Week 2	09/13	Basic Building blocks in Deep Learning (Softmax Regression, Regularization, Dropout, Initialization, optimization, etc.)	chapters 4, 5, chapters 3, 5	hwk #1
Week 3	09/20	Convolutional Neural Networks (CNNs)	chapter 7	project initiation
Week 4	09/27	Modern CNNs	chapter 8	hwk #2
Week 5	10/04	RNNs and Language Modeling	chapter 9
Week 6	10/11	Modern RNNs and Language Translation	chapter 10	hwk #3
Week 7	10/18	Transformers	chapter 11	project milestone #1
Week 8	10/25	Midterm exam
Week 9	11/01	Vision Transformers	chapter 12	project milestone #2
Week 10	11/08	Selective State Space Models
Week 11	11/15	Graph Neural Networks	online tutorial;chapter 14
Week 12	11/22	Reinforcement Learning	chapter 17	hwk #4
Week 13	11/27	Deep Generative Modeling; New frontiers in Deep Learning	chapter 20
Week 14	12/06	Project Presentation	Requirements for the presentation and report (Canvas)	Project milestone #3
Week 15	TBD	Project Presentation	Requirements for the presentation and report (Canvas)	Project milestone #3

* The schedule is subject to change. Please check Canvas for the most up-to-date information.

For more detailed course information, please refer to the syllabus and the Canvas modules!