CS 780: Deep Reinforcement Learning

Units

3-0-0-0 (9)

 

Pre-requisites

Instructor's consent and Must: Solid understanding of Machine Learning (e.g, CS771 or equivalent course), theoretical and practical knowledge of Deep Learning, Proficiency in Linear Algebra, Probability and Statistics, Proficiency in Python Programming.

 

Desirable

Probabilistic Machine Learning (CS772), Topics in Probabilistic Modeling and Inference (CS775), Deep Learning for Computer Vision (CS776)

 

Level of the course

Senior UG and PG (6xx level)

 

Course Description

In this course, we will explore how an agent (via interactions with the environment) can learn by trial and error. This is quite different from supervised machine learning and comes close to how humans learn by interactions. Reinforcement Learning (RL) deals with problems that require sequential decision making. This course will explore the foundations of reinforcement learning. We will study different algorithms for RL and later in the course, we will explore how functional approximation in RL algorithms could be done using neural networks giving rise to deep reinforcement learning.

 

Tentative Topics

1. Introduction to Reinforcement Learning
2. Multi-armed bandits
3. Markov Decision Processes
4. Dynamic Programming
5. Monte Carlo Methods
6. Temporal-Difference Learning
7. Function Approximation Methods
8. Policy-Based and Value-Based Algorithms: REINFORCE, SARSA, DQN,
9. Advantage Actor-Critic (A2C)
10. Proximal Policy Optimization (PPO)

 

References

This course will follow the book given below but it also gleans information from a variety of sources like research papers, other courses, etc. Relevant references would be suggested in the lectures.

1. Reinforcement Learning An Introduction by Richard S. Sutton and Andrew G. Barto, Second Addition