Loss Function

A loss function tells us how good our current classifier is

Given a dataset of examples

{(x - i, y_{i})}_{i = 1}^{N}

Where $x_{i}$ is image and $y_{i}$ is label
Loss over the dataset is a sum of loss over examples:

L = \frac{1}{N} \sum_{i} L_{i} (f (x_{i}, W), y_{i})

Regularization

Regularization term: $λ R (W)$

L = \frac{1}{N} \sum_{i} L_{i} (f (x_{i}, W), y_{i}) + λ R (W)

Examples of loss functions

Multiclass SVM Loss

Multinomial Logistic Regression