Homework 2: Eigenfaces and Isomap

For part I, you are to modify the MATLAB code PCAdemo from Kevin Murphy's Machine Learning: A Probabilistic Perspective, so as to do the following tasks. In part II, you are also to use the ISOMAP code from Isomap .

The homework is due on Thursday 11 September.

Part I. Eigenfaces and hallucinations

In this part, you are to use the dataset OlivettiExpanded.mat. This includes 10 faces for each of you(converted to 92 X 112 Grayscale), 6 in training set and rest 4 in test set in addition to the original data. In case you have NOT uploaded your faces, you really MUST do it by Thursday 4 September 5 PM. The final .mat file will be updated after that.

A. Compute the reconstruction error for your five images in this .mat file, and show the output for TWO of these in terms of the reconstructed image with 10 and 100 eigenvectors, and the reconstruction error plot for 1 to 100 eigenvectors. A sample output image is shown below.

B. Use k-NN with k=1 and report the face it is recognized as. Do the same with k=3, and report a face as recognized only if two of the neighbours are the same person.

C. A second set of test images are in hw2-test.mat. The images in this file are shown below (they are in the directory)

1 2 3 4 5
Show the same outputs and plots for these images.

Part II. Are face spaces nonlinear?

Take these subsets of your own faces
  1. Horizontal motion
  2. Vertical motion
  3. Mix of Horizontal, Vertical and Rotating Faces
  1. Compute the PCA on these images and computer the residual variance for 1 to 100 eigenvectors.

  2. Run isomap for target dimension d=1 to d=10 and compute the residual variance for 10, 100 eigenvectors for i, ii and iii.

    Report a sample of 4-10 original images from your dataset and the residual variance plots.

  3. What would be the expected intrinsic dimension of the variation in this data?
    Give reasons for any differences you observe in the residual variances plots.