Hyperparameter Tuning for Beginners — Part One
First of all, I would like to make it clear that I am in no way an expert and for all intents and purposes, I am still a learner. That being said, this post was birthed from my confusion at the concept of hyperparameter tuning. While I have a more or less firm grasp of the concept at the moment, it was a major source of frustration for me at first. Without much delay, let’s jump right in.
Wikipedia defines hyperparameter tuning in machine learning as the problem of choosing a set of optimal hyperparameters for a learning algorithm and further defines a hyperparameter as a parameter whose value is used to control the learning process. This definition might be simple enough for some, however, it can also be a source of headache to others. The latter is the category I fell into, it is also the category of people for which this post was made.
For this, I would like you to imagine a machine learning algorithm as the recipe for a dish. Lolz, I can already feel the puzzled and questioning stares. Please bear with me for a moment.
If you are done imagining a machine algorithm as a recipe, well done, and congratulations, you are as crazy as I am. The next step is to imagine the hyperparameters as the ingredients for the recipe.
This would then mean that hyperparameter optimization is the process of adjusting the ingredients of the recipe for specific situations. An example is reducing the salt content of the recipe when serving a hypertensive person or if you want to get a little bit technical, it is the process of adjusting the hyperparameters of the specific algorithm in use to suit the data and try to get better predictions.
In most cases, hyperparameter tuning heavily involves trial and error to see which of the parameters are better suited for the data and give the best results.
I hope with this short post, you have been able to gain an understanding of hyperparameter tuning. If you have not, feel free to ask questions. In the next post, we are going to talk about the different methods of tuning hyperparameters and their implementation in python.