A Silver Lining in the otherwise dull life of a Master's student ...
Machine learning focuses on developing machines that can learn by itself and act on new data once trained.It involves a lot of head banging till you understand how it works. Once you have a decent understanding of it only then will you be able to appreciate the beauty of it and identify what works best and understand the reasoning behind it as well.
I was recently working on a classic classification problem to detect spam and ham images .I wanted to test different classifiers and pick a model that works best for the given dataset. I decided to go with the safest option, SVM; I wanted my model to do the work and give good results
without me applying many preprocessing techniques. I decided to run SVM on my dataset and it ran for about 14 hours without converging so I stopped it and started again, but to no avail ....
The dataset wasn't that big either - it was around 1200 rows and 22 features related to the images like mean, variance, skew among the different channels , and all the features had numerical values. Apparently this is a pretty common problem with SVM.
I looked around at various options, and there came Normalization, my knight in shining armor. When I normalized the data based on the mean and standard deviation the training phase got over in less than 2 min and the score improved as well from 0.56 to a whooping 0.90.
Great that it worked , and the reasoning behind it is as below.
Consider you have 2 dimensions 1 and 2 and the first has values from 1 to 1000 and the second has values from 0 to 1 then dimension 1 will be assumed to have higher importance than the second dimension. To avoid that Normalizing has to be done based on features instead of being based on rows.
Feature-wise normalization can be done in 2 ways:
using mean and standard deviation , or by using min and max. The former is call soft normalization and is found to work better than the latter which is called hard normalization. In hard normalization we will have to depend on the min and max of a value in the training set which is dynamic when
compared to that of using mean and standard deviation. As the features are normalized their range is reduced and hence it becomes easier for the model to determine the distances and helps converge faster.
As standardization/normalization changes the distribution there is no guarantee that the accuracy will improve for the same setting. In some cases it can yield worse results. Whether normalization improves or worsens the result is an open topic but in this case it worked.
Machine learning focuses on developing machines that can learn by itself and act on new data once trained.It involves a lot of head banging till you understand how it works. Once you have a decent understanding of it only then will you be able to appreciate the beauty of it and identify what works best and understand the reasoning behind it as well.
I was recently working on a classic classification problem to detect spam and ham images .I wanted to test different classifiers and pick a model that works best for the given dataset. I decided to go with the safest option, SVM; I wanted my model to do the work and give good results
without me applying many preprocessing techniques. I decided to run SVM on my dataset and it ran for about 14 hours without converging so I stopped it and started again, but to no avail ....
The dataset wasn't that big either - it was around 1200 rows and 22 features related to the images like mean, variance, skew among the different channels , and all the features had numerical values. Apparently this is a pretty common problem with SVM.
I looked around at various options, and there came Normalization, my knight in shining armor. When I normalized the data based on the mean and standard deviation the training phase got over in less than 2 min and the score improved as well from 0.56 to a whooping 0.90.
Great that it worked , and the reasoning behind it is as below.
Consider you have 2 dimensions 1 and 2 and the first has values from 1 to 1000 and the second has values from 0 to 1 then dimension 1 will be assumed to have higher importance than the second dimension. To avoid that Normalizing has to be done based on features instead of being based on rows.
Feature-wise normalization can be done in 2 ways:
using mean and standard deviation , or by using min and max. The former is call soft normalization and is found to work better than the latter which is called hard normalization. In hard normalization we will have to depend on the min and max of a value in the training set which is dynamic when
compared to that of using mean and standard deviation. As the features are normalized their range is reduced and hence it becomes easier for the model to determine the distances and helps converge faster.
As standardization/normalization changes the distribution there is no guarantee that the accuracy will improve for the same setting. In some cases it can yield worse results. Whether normalization improves or worsens the result is an open topic but in this case it worked.
Pointers :- Normalizing the features can yield better accuracy based on the dataset and model setting; and there is no free lunch :)
No comments:
Post a Comment