L1 and L2 regularization, L1 and L2 norm
What do these concepts have in common and how do they differ?
Do I understand correctly that with L1 regularization, some of the features are completely discarded, and with L2 regularization, the features are smoothed out so that the rest do not "fade" because of one feature? And L1 and L2 are the norm, are they just different options for calculating the distance?
2
1 answers
Norm is a functional defined on a vector space and generalizes the concept of the length of a vector or the absolute value of a number.
The general formula for Lp norm:
L1 norm / Distance of city blocks:
L2 norm / Euclidean metric:
For regularization to the error formula the corresponding norm is added-at L1 regularization is added L1 norm and L2 norm in the case of L2 regulation:
3
Author: MaxU, 2020-05-12 13:26:16