Softmax Classifier

AKA Multinomial Logistic Regression, Cross-entropy loss

Want to interpret raw classifier scores as probabilities

s = f (x_{i}; W)

P (Y = k | X = x_{i}) = \frac{e^{s} k}{\sum_{j} e^{s} j}

L_{i} = - \log P (Y = y_{i} | X = x_{i})

Maximum Likelihood Estimation: Choose weights to maximize the likelihood of the observed data

Pasted image 20241130142946.png