What is Hyperparameter Tuning
Optimization of ML model settings
Hyperparameter Tuning is the process of finding optimal settings for a machine learning model that are not learned from data but set before training begins.
Examples of Hyperparameters
- Learning rate — training speed
- Number of layers in neural network
- Batch size — examples per iteration
- Regularization — L1, L2, dropout
Tuning Methods
- Grid Search — exhaustive search of all combinations
- Random Search — random sampling
- Bayesian Optimization — intelligent search based on previous results
- AutoML — automatic tuning
Tools
- Optuna, Hyperopt, Ray Tune, Keras Tuner
Importance
Proper hyperparameter tuning can significantly improve model quality without changing the architecture.