CS365: Homework 1

Problem A

• Error in classification vs number of neighbors graph using kNN classifier
  

  Least error obtained for k=5

Problem B

1. 2-D isomap using Euclidean Distance
   

   Isomap is a dimensionality reduction method which calculates geodesic distances between two points calculated using dijkstras or floyd warshall's algorithm and minimizes the difference between geodesic distance & the distance b/w corresponding points in the low dimension.

1. Digits 1 and 7
   

   Cluster for digits 1 and 7

   

   Residual Variance for digits 1 and 7

   
   
2. Digits 4 and 9
   

   Cluster for digits 4 and 9

   

   Residual Variance for digits 4 and 9

   
   
3. All digits together
   

   Cluster for all digits

   

   Residual Variance for all digits

   
   
2. 2-D isomap using tangent distance


1. Digits 1 and 7
   

   Cluster for digits 1 and 7

   

   Residual Variance for digits 1 and 7

   
   
2. Digits 4 and 9
   

   Cluster for digits 4 and 9

   
   

   Residual Variance for digits 4 and 9

   
   
3. All digits together
   

   Cluster for all digits

   

   Residual Variance for all digits

   
3. EXTRA
   

   Results of the isomap using euclidean distance applied to "2"'s images in database

   
   

   Residual Variance for "2"s

   
   

Problem C

Deep networks first try to reconstruct the data using unsupervised RBM or other. The reconstruction process tries to find the features or prior that cest classify or identify the data. The number of hidden unite, learning rate, epochs etc. other parameters effect this classification. After this fine-tuning is done using the supervised data to reduce the error in classification,. Then using the model built, it classifies the unseen data using disriminative approach.

Training done of 30,000 images and testing data of size 10,000 images is used.

1. DBN
   
2. DBN with NN initialized using weights of DBN
   
3. SAE with FFNN initialized using weights of SAE
   
4. CNN
   
5. NN

Increasing the number of Epochs ( rounds of training) reduces the error in classification.
Increasing the batchsize increases the error.
Low learning rate is god for classification.