Resampling, Bootstrap, and Monte Carlo Methods
Simulation as a statistical tool
Monte Carlo approximation
Monte Carlo methods approximate expectations by averages of simulated draws: . The approximation error typically shrinks at rate .
What is the principal idea of the bootstrap?
Bootstrap methods resample the data, usually with replacement, to approximate the distribution of an estimator.
Correct answer: Resample from the observed data to mimic sampling variability
Why is the bootstrap useful in complex settings?
The bootstrap can approximate standard errors, confidence intervals, and bias for complicated estimators.
Correct answer: It estimates uncertainty when analytic sampling distributions are difficult to derive.
Rule of thumb
If the estimator is smooth enough and the sample is representative, bootstrap approximation often works well; if the estimator is highly non-smooth, extra care is needed.
Resampling methods
| Method | Main use |
|---|---|
| Nonparametric bootstrap | Approximate sampling distribution from the empirical distribution |
| Parametric bootstrap | Simulate from a fitted parametric model |
| Permutation test | Assess significance under label exchangeability |
| Cross-validation | Estimate predictive performance |