Evaluating Models and Avoiding Overfitting
Does the Model Generalize?
Evaluation basics
A good model does well on new data, not just training data. Evaluation metrics help measure performance. For classification, common metrics include accuracy, precision, and recall.
Overfitting
Overfitting happens when a model learns the training set too well, including noise or irrelevant details. It may score highly on training data but poorly on new examples.
Common metrics
| Metric | What it tells you |
|---|---|
| Accuracy | Fraction of correct predictions |
| Precision | How many predicted positives were truly positive |
| Recall | How many true positives were found |
| Loss | Overall prediction error |
Watch for Overfitting
If training performance keeps improving while validation performance gets worse, the model may be overfitting.
What is overfitting?
Overfitting is poor generalization due to memorization or noise fitting.
Correct answer: When a model memorizes training data and performs poorly on new data
Why do we care about validation performance?
Validation helps detect overfitting and guide model choices.
Correct answer: It helps estimate how well the model will work on new data.
Which metric is a fraction of correct predictions?
Accuracy measures the proportion of correct predictions.
Correct answer: Accuracy