What is Regularization in Machine Learning and AI?
Regularization - Regularization in machine learning is a process of introducing additional information in order to prevent overfitting.
The green and blue functions both incur zero loss on the given data points.
Regularization will induce a model to prefer the green function, which may generalize better to unseen data.
Use of regularization in classification:
One particular use of regularization is in the field of classification. Empirical learning of classifiers(learning from a finite data set) is always an undermined problem.
because in general, we are trying to infer a function of any X given only some example
x1,x2,x3,x4,x5,x6............xn.
A regularization term(or regularization) R(f) is added to the loss function:
where V is an underlying loss function that describes the cost of predicting f(x) when the label is y, such as the square loss or hinge loss and lamda is a parameter that controls the importance of the regularization term.R(f) is typically chosen to impose a penalty.