CSC 580: Principles of Machine Learning - Fall 2022

Tentative schedule

Slides credit: built upon CSC 580 Fall 2021 lecture slides by Kwang-Sung Jun, which in turn builds upon Daniel Hsu, Francesco Orabona, and Xiaojin (Jerry) Zhu’s teaching materials.

Date	Topics	Notes	Readings	Homework
Aug 22	Introduction, motivation, course mechanics	slides	CIML Chap. 1	HW0
Aug 24	Supervised learning paradigm; decision trees	slides	CIML Chap. 2
Aug 29	Finish decision trees; limits of learning; practical issues in supervised learning	slides	CIML Chap. 3	HW0 due
Aug 31	Hyperparameter selection; cross-validation; nearest neighbors	slides	CIML Chap. 4	HW1
Sep 5	Labor day holiday
Sep 7	Linear classifiers; Perceptron	slides	CIML 5.1-5.6
Sep 12	Practical considerations: features; performance metric beyond error; ROC curves		CIML 5.7, 7.1-7.3
Sep 14	Precision-recall curves; confidence intervals and hypothesis testing	slides	CIML 7.4-7.6
Sep 19	Finish hypothesis testing; Linear models for regression	slides	CIML 7.7
Sep 21	Introduction to convex optimization; Linear models for classification		CIML 11.1-11.2
Sep 26	SVM; stochastic gradient descent; dual formulations	slides	CIML 11.4, 11.5
Sep 28	Finish SVM dual formulation; geometric interpretation; begin kernel methods	slides	CIML 3.4, 11.3
Oct 3	Finish kernel methods: kernel Perceptron; kernel ridge regression; begin unsupervised learning: clustering and k-means		CIML Chap 15	HW2
Oct 5	Class canceled (please review your HW1 & soln guide)		My old notes on SVM and kernels;
Oct 10	Finish clustering; Principal component analysis (PCA)	slides	CIML 9.1-9.3
Oct 12	Finish PCA; begin probabilistic graphical models		CIML 9.4-9.7
Oct 17	Finish Naive Bayes; Midterm review	slides	CIML 16.1-16.2
Oct 19	Midterm exam
Oct 24	Expectation-maximization and Gaussian mixture models (GMMs)	slides	CIML 16.3
Oct 26	Finish GMM; A closer look at Bayes Nets and Conditional Independence		Jason Pacheco’s PGM note	Project proposal due
Oct 31	d-separation; Begin HMMs: the forward algorithm		Jason Pacheco’s Dynamical Systems note
Nov 2	Finish HMMs: Viterbi algorithm for decoding and EM algorithm for learning	slides	CIML 10.1-10.2	HW3
Nov 7	Neural Networks; begin backpropagation		CIML 10.3-10.6
Nov 9	Finish backpropagation; tips for training neural networks	slides	Berkeley CS 285 PyTorch tutorial (see links in slides)
Nov 14	Convolutional neural networks (CNNs)	slides	Stanford CS 231n notes on CNNs
Nov 16	CNNs cont’d; begin generative models: autoencoders
Nov 21	Generative models: VAEs		Jaan Altosaar’s VAE tutorial
Nov 23	Generative models: GANs	slides	The original GAN paper	HW4
Nov 28	Reinforcement learning: policy evaluation and planning in MDPs
Nov 30	Reinforcement learning: learning in MDPs	slides	Andrej Karpathy’s blogpost on Deep RL
Dec 5	Learning Theory: the PAC and agnostic PAC framework
Dec 7	Learning theory: uniform convergence and structural risk minimization; Final review	slides	``Deep double descent’’ paper by Nakkiran et al