Artificial Intelligence
What is Hyperparameters
Model settings defined before training
Hyperparameters are machine learning model parameters that are set before training begins and do not change during the process. Unlike regular parameters, hyperparameters define the architecture and behavior of the model.
Main Hyperparameters
- Learning rate — model learning speed
- Batch size — data batch size for one iteration
- Number of epochs — number of passes through the entire dataset
- Number of layers — number of neural network layers
- Regularization — regularization parameters (L1, L2, dropout)
Tuning Methods
- Grid Search — exhaustive search of all value combinations
- Random Search — random search in parameter space
- Bayesian Optimization — Bayesian optimization
- AutoML — automatic architecture and parameter selection
Impact on Model
- Too high learning rate — model does not converge
- Too low learning rate — slow training
- Large batch size — faster but worse generalization
- Small batch size — better generalization but slower