You can choose to do one of the following types of projects:
Literature survey. You need to choose a topic (see below for some ideas) and read at least 6 papers on this topic. Be sure to select papers judiciously, so that both classical and state-of-the-art perspectives are covered. Sometimes the Introduction and Related Work sections of recent papers will help you find most relevant papers. When you are in doubt whether a paper is relevant, come to my office hour and we can figure this out. It is important that you read the papers critically, and form your own opinions on the papers you read: What are the major open problems in this area? What are the pros and cons of existing approaches? (The latter can be summarized with a table, for example.)
Implementation. You are asked to conduct experiments on deploying theoretically-principled learning algorithms onto synthetic or real data. If you choose to implement an algorithm from a paper that does not have theorem statements (or ‘trivial’ theorems with straightforward proofs), this may not be a good fit for this course. In addition, we ask that your experiment must be used to support theoretical results obtained in the paper. You are expected to conduct critical analyses on your experimental results, by answering questions such as: how well do the experimental results agree with the proposed theory? If the results do not agree well with the theory, which assumption in the theory are violated? You should also provide a list of datasets you are using.
Research. Research projects, roughly speaking, can have two styles: first, attacking an existing open problem in the literature; second, formulating a new (theoretically interesting and practically relevant) problem and proposing a feasible solution. Completing a research project naturally requires a thorough literature survey in the first place - you need to ensure that your approach or model has never been proposed in prior works. Note that a research project may require substantially larger amount of time compared to the first two project types, so I suggest being careful with this choice if you already have a heavy workload this semester. The upside of a research project is that your work may result in publications.
Project Proposal. A 1-page project proposal is due Mar 16 on gradescope. The project proposal should consist of the following parts:
a list of team members,
a brief description of the project topic,
a justification why this is relevant in learning theory,
for literature survey, have a planned reading list; for implementation, have a list of algorithms to implement and a set of driving questions to answer with the experiments; for research, have a list of research questions to attack (It would be good to have one concrete question, and list a few important sub-questions toward solving it.)
If you need help with choosing a project, please schedule an appointment with me and I will help you brainstorm one.
Midterm Progress Report. A 1-page report on your progress is due Apr 6 on gradescope.
For literature survey, give a list of papers you have already read, and how they are related.
For implementation projects, give a list of algorithms that you have already implemented, and provide experimental results (if you have some). How do the results answer the questions you asked in the proposal?
For research projects, provide a list of trials you have made to attack your research problem, regardless of whether they are successful.
Final Presentation. The presentation will be on Apr 29 and May 4, in class. The actual lengths of the presentations will depend on the number of groups in the class. Please send me a copy of your slides after your presentation.
Final Report. A 4-page summary of the project is due May 4 on gradescope. The report will be judged on both clarity and quality. The report needs to be typeset by LaTeX.
For literature survey, provide a critical summary of the papers you have read; discuss the connections among these papers, and their impacts to broader field.
For implementation projects, present your experimental results, and check whether the experimental results agree with theory.
For research projects, your report should have a Related work section discussing why your result is novel compared to existing work. You should also describe your approach (if you have a new algorithm, provide its pseudocode; if you propose a new learning model, define new key concepts in your model; if you show your proposed algorithm has good performance guarantee, write down a theorem statement on it.)
Below are a few example research directions in machine learning theory, each with a few “seed papers”; you can use the related work section in these papers, or use the “cited by” functionality in e.g. google scholar to find more papers on the same topic. Please also refer to proceeding pages of recent machine learning / learning theory conferences and workshops for more project ideas, such as: