Machine Learning Foundations
Learning from Examples
Supervised Learning
In supervised learning, a model learns a mapping from inputs to outputs using labeled examples. The goal is to minimize prediction error on unseen data, not just on the training set.
Generalization
Generalization is the ability to perform well on new data. A model that memorizes the training set without learning patterns has poor generalization.
Key Concepts
What is the main purpose of a test set?
The test set provides an unbiased estimate of how the model will perform on new data.
Correct answer: To estimate performance on unseen data
What is overfitting?
Overfitting indicates poor generalization.
Correct answer: When a model learns the training data too closely and performs poorly on new data.
Learning Paradigms
| Paradigm | Signal | Example |
|---|---|---|
| Supervised | Labeled outputs | Spam classification |
| Unsupervised | No labels | Clustering customers |
| Reinforcement | Rewards from actions | Game playing |
| Self-supervised | Labels derived from data | Masked language modeling |
Which situation is most likely to cause overfitting?
Flexible models trained on small datasets can memorize noise and details rather than learn patterns.
Correct answer: A model with too little data and too much flexibility
Why is a validation set used?
Validation helps tune hyperparameters while preserving the test set for final evaluation.
Correct answer: To make model-selection decisions without using the test set.