Nearest Neighbour
Predict based on most similar images
- Memorize all data and labels
- Predict the label of the most similar training image
Training
- O(1)
- Simply remembers all the training data
Predict
- Calculate L1 Distance for each training image
- Predict label of closest image
Problem
Single nearest neighbor will be affected by outliers
- e.g. yellow area in green
K-Nearest Neighbors
Instead of copying from nearest neighbor, take a majority vote of k closest points
Never used for images
- Very slow
- Distance metrics on pixels are not informative
Hyperparameters
- What is the best value of k to use?
- What is the best distance to use?