Model Evaluation and Metrics
How Do We Know a Model Works?
Classification Metrics
Accuracy is the fraction of correct predictions, but it can be misleading on imbalanced datasets. Precision, recall, and score often give a more detailed picture.
Regression Metrics
For numeric prediction tasks, common metrics include mean squared error and mean absolute error.
Metric Summary
| Metric | Best For | Interpretation |
|---|---|---|
| Accuracy | Balanced classification | Fraction correct |
| Precision | False-positive control | How many positive predictions were right |
| Recall | False-negative control | How many actual positives were found |
| MSE | Regression | Average squared error |
Which metric is especially useful when false negatives are costly?
Recall focuses on how many true positives are captured, which matters when missing positives is expensive.
Correct answer: Recall
Why can accuracy be misleading on imbalanced data?
When classes are uneven, accuracy may hide poor performance on the minority class.
Correct answer: Because a model can get high accuracy by predicting the majority class most of the time.
Confusion matrix
A confusion matrix helps you inspect true positives, true negatives, false positives, and false negatives directly.
What does score combine?
is the harmonic mean of precision and recall.
Correct answer: Precision and recall
What is the purpose of cross-validation?
Cross-validation reduces dependence on a single data split.
Correct answer: To estimate performance more reliably by evaluating on multiple train-validation splits.