| Lecture No. |
Topics |
Readings/References/Comments |
Videos/Slides/Notes |
| 1 |
Course Logistics, Intro to Probabilistic Modeling and Inference |
[Z15], [B14] (for now, up to sec 3), a brief prob-stats refresher, a basic tutorial on Bayesian inference |
will be posted on mooKIT
|
| 2 |
Basics of Probabilistic/Bayesian Modeling and Parameter Estimation |
Wikipedia entries (to be read in the same order) on Bayesian Inference, Prior, Likelihood, Posterior, Posterior Predictive, Credible Intervals (for now, these articles are meant for cursory reading; may safely skip the parts that seem too advanced to you), Additional Reading: MLAPP Section 3.1-3.3, Conjugate Priors |
will be posted on mooKIT |
| 3 |
Bayesian Inference for Some Basic Models |
Lecture 2 suggested readings + MLAPP 3.3-3.5, Bayesian Inference for Gaussians, Wikipedia entry on Dirichlet distribution |
will be posted on mooKIT |
| 4 |
Bayesian Inference for Gaussians (Contd) and Exponential Family |
MLAPP 4.3-4.6 (it is far more detailed than you probably need at the moment; you may skip very detailed proofs, can focus more on the examples and the standard results on Gaussian properties, inference, etc), PRML 2.3, Bayesian Inference for Gaussians,
MLAPP 9.1-9.2, some notes on exp-family (if further interested, may skim through the Wikipedia article on exp-fam) |
will be posted on mooKIT |
| 5 |
Probabilistic Linear Regression |
PRML 3.3, MLAPP 7.1-7.3, 7.6 (7.6.1-7.6.2), Recommended Readings: Bayesian Inference tutorial
(with Bayesian linear regression as case study; as of now, may read up to section 3) |
will be posted on mooKIT |
| 6 |
Probabilistic Approaches for Sparse Modeling |
Recommended Readings: Section 4 of Bayesian Inference tutorial, The
Relevance Vector Machine paper (don't need to read all of it in detail; can just skim over to see the key ideas at a high level, and the experimental results), and the other references mentioned in the slides |
will be posted on mooKIT |
| 7 |
(1) Probabilistic Models for Classification: Logistic Regression, (2) Laplace Approximation |
MLAPP 8.4 |
will be posted on mooKIT |
8 |
(1) Generalized Linear Models, (2) Generative Models for Supervised Learning |
MLAPP 9.3 (for GLM), 3.5.1.2, 3.5.2, 3.5.5 (for some examples of generative classification, including the Bayesian way) |
will be posted on mooKIT |
| 9,10,11 |
Gaussian Processes |
PRML Sec. 6.4, MLAPP Sections 15.1-15.2.5, (Optional: 15.3-15.4),
Illustration of various kernels for GP,
Some GP software packages: GPFlow (Tensorflow based),
GPyTorch (PyTorch based),
GPML (MATLAB based)
|
will be posted on mooKIT |
| 12 |
Probabilistic Approaches to Active Learning |
Bayesian Active Learning, BALD paper (sections 1-2, rest optional).
An old but classic paper on this topic: Information-Based Objective Functions for Active Data Selection |
will be posted on mooKIT |
| 13 |
Bayesian Optimization |
An introduction to Bayesian Optimization (with some code),
Another Python Notebook on Bayesian Optimization |
will be posted on mooKIT |
| 14 |
Multi-parameter Models, Conditional Posteriors, Local Conjugacy |
Highly recommended: Paper on Bayesian Matrix Factorization,
and Gibbs Sampling for the Uninitiated
(note: we will look at Gibb sampling again in more detail and formally when talking about MCMC but if you want to get a good and
practical overview then this tuutorial is very nice and doesn't require you to understand MCMC in much detail beforehand) |
will be posted on mooKIT |
| 15 |
Latent Variable Models and EM |
PRML Chapter 9 (has examples of EM for Gaussian Mixture Model and Bayesian Linear Regression), MLAPP 11.4, Optional readings: Original EM paper
(technically very dense but lots of interesting insights),
Another classic paper on EM (more accessible),
Online EM (practically oriented),
Online EM (theoretically oriented) |
will be posted on mooKIT |
| 16 |
Introduction to Variational Inference |
Readings: PRML 10.1,10.2,10.3.10.4, Life after EM (shows the connection between EM, variational EM, and variational inference, through several examples),
VI: A Review for Statisticians
|
will be posted on mooKIT |
| 17 |
VI (Contd) and Recent Advances in VI |
VI: A Review for Statisticians
(Sec 4.3 on SVI), SVI paper
(if you are interested in a more in-depth treatment of SVI),
Advances in Variational Inference
(a bit long but I would suggest skimming it over to get a sense of the various recent advances in VI)
|
will be posted on mooKIT |
| 18 |
Approximate Inference via Sampling |
Readings: PRML 11.1-11.3, MLAPP 24.1-24.3, Recommended:
Intro to MCMC for Machine Learning,
Monte Carlo for Absolute Beginners,
Gibbs Sampling for the Uninitiated
|
will be posted on mooKIT |
| 19 |
Gibbs Sampling Examples, Some Aspects about MCMC |
Readings: PRML 11.1-11.3, MLAPP 24.1-24.4,
Gibbs Sampling for the Uninitiated
|
will be posted on mooKIT |
| 20 |
MCMC with Gradient Information, Recent Advances in MCMC |
Recommended: SGLD paper and other papers referenced in the slides
(not required to get into every technical detail
but try skimming through some of these papers to get a high level idea),
Survey paper on SGMCMC methods like SGLD and improvements,
Patterns of Scalable Bayes
(See sec 4.2 for parallel MCMC, though other parts are also useful for a general introduction to approximate
inference methods, including scalable methods),
No U-Turn Sampler (section 2 describes the basics of HMC),
|
will be posted on mooKIT |
| 21 |
Introduction to Nonparametric Bayesian Modeling |
Suggested Readings: For NPBayes modeling general intro - this
tutorial survey paper,
For NPBayes Clustering, this paper on Dirichlet Process
|
will be posted on mooKIT |
| 22 |
Probabilistic Models for Deep Learning |
Suggested Readings: Weight Uncertainty in Neural Networks (Blundell et al, 2015),
and other papers references in the slides; this tutorial paper on Bayesian Deep Learning |
will be posted on mooKIT |
| 23 |
Deep Generative Models |
Suggested Readings (also look at the references in the slides): For PPCA, FA, etc (classical models), see MLAPP Chap 12;
for gamma-Poisson latent factor model and Dirichlet-multinomial PCA, see this paper
and this paper;
for LDA, see this tutorial paper;
for VAE, see the VAE paper, and see this tutorial paper
and this survey;
for GAN, see the GAN paper, and also
see this survey;
|
will be posted on mooKIT |
