CSC 665: Online Learning and Multi-armed Bandits - Spring 2020

This course is a continuing line of topics from CSC 665 Section 2 Machine Learning Theory that delve into online learning and multi-armed bandits (but the knowledge from ML theory is not required). The students will learn, via the lens of mathematical foundations, how and when machines can learn in an online manner. Specifically, the course offers mathematical formulation of learning environments (e.g., stochastic and adversarial worlds with possibly limited feedback), fundamental limits of learning in these environments, various algorithms concerning sample efficiency, computational efficiency, and generality. Throughout, students will not only learn fundamental mathematical tools upholding the current understanding of sequential decision making in the research community but also develop skills of adapting these techniques to their own research needs such as developing new algorithms.

Why online learning / multi-armed bandits?

  • Backbone of stochastic gradient descent algorithms.
  • How can ‘learning’ possible when when the data is arbitrarily manipulated.
  • Learn how companies learn your preferences by interacting with you in recommendation systems and online advertisements.
  • Besides, beautiful mathematical results and algorithms, and some practical algorithms.

Logistics info

  • Monday and Wednesday, 12:30pm-1:45pm
  • Saguaro Room 223
  • Office Hour: Tuesdays 4-5pm or by email appointment
  • Piazza link access code: bandits
  • Gradescope entry code: M46EWY
  • D2L



There is no designated textbook for this course. Much of the course materials will be based on the following materials (in the order of appearance in class schedule):

The following set of surveys and books also provide a good coverage of relevant materials:

Review for prerequisites

Here are some excellent notes for probability review and linear algebra review.

Machine learning courses at UA

Similar courses at other institutions

The following is a far-from-complete list of learning theory courses offered at other institutions: