Why normalization or feature scaling is important ?
Why Normalization or in other word why feature scaling is important ?
Explanation: 3 students visit to a data scientist, below how conversation goes
Cameron : Hey , my height is 5.9 ft. and weight is 175 lbs. I wear t-shirt of size L
Sarah: My height is 5.2 ft. and weight is 115 lbs and I wear t-shirt of size S.
Chris : My height is 6.1 ft. and weight is 140 lbs. Can you help us to find my t-shirt size?
Senior Data Scientist to Junior: Hey, can you try, wanna see how you approach?
Junior DS: I will take average of Height + Weight of Sarah (120.2 - S) and Cameron (180.9 - L)
According to this logic Chris t-shirt size should be of M size.
Senior DS to Junior: Your approach is good, but it won't give right answer until you do scaling of height and weight.
Image source : Google image
Another Example: House Price Y = x1 * theta1 + x2*theta2 + c.
- k-nearest neighbors with an Euclidean distance measure is sensitive to magnitudes and hence should be scaled for all features to weigh in equally.
- Scaling is critical, while performing Principal Component Analysis(PCA). PCA tries to get the features with maximum variance and the variance is high for high magnitude features. This skews the PCA towards high magnitude features.
Clustering use of Euclidian distance for finding the correct centeroid for each cluster. So we need to scale the features before applying clustering.
- Tree based models are not distance based models and can handle varying ranges of features. Hence, Scaling is not required while modelling trees.
- Algorithms like Linear Discriminant Analysis(LDA), Naive Bayes are by design equipped to handle this and gives weights to the features accordingly. Performing a features scaling in these algorithms may not have much effect.
