What is Overfitting
When a model memorizes training data too well
Overfitting is a machine learning problem when a model memorizes training data too well and fails to generalize to new data.
Signs of Overfitting
- High accuracy on training data
- Low accuracy on test data
- Large gap between train and test metrics
- Model memorizes noise in data
Causes
- Model too complex
- Insufficient training data
- Training for too long
- Lack of regularization
Prevention Methods
- Regularization (L1, L2)
- Dropout in neural networks
- Early stopping
- Cross-validation
- Data Augmentation
- Model simplification
Bias-Variance Tradeoff
Overfitting is associated with low bias and high variance. Finding the right balance is crucial.