Regularization

Prevents overfitting

Model should be “simple”, so it works on test data

Regularization term: $λ R (W)$

L = \frac{1}{N} \sum_{i} L_{i} (f (x_{i}, W), y_{i}) + λ R (W)

$λ$ = regularization strength (Hyperparameters)

Common use:

Add some kind of randomness

Average out randomness