Nonparametric, Bayesian Nonparametric, and Robust Statistics
Nonparametric estimation
Flexible function estimation
Nonparametric statistics avoids strict finite-dimensional model assumptions. Kernel density estimation, local polynomial regression, splines, and nearest-neighbor methods adapt to unknown smoothness but face the curse of dimensionality.
Bias-variance tradeoff
As flexibility increases, bias often decreases while variance increases. Optimal smoothing balances these terms and is frequently selected by cross-validation or plug-in rules.
Dimensionality curse
Nonparametric methods can degrade rapidly as dimension increases, often requiring structure such as sparsity, additivity, or low intrinsic dimension.
Bayesian nonparametrics
Infinite-dimensional priors
Bayesian nonparametrics places priors on functions or measures rather than finite parameter vectors. Important examples include the Dirichlet process, Gaussian process priors, and Pitman-Yor processes.
Which object is a common prior over functions in Bayesian nonparametrics?
Gaussian processes define distributions over functions and are widely used in Bayesian nonparametrics.
Correct answer: Gaussian process
What is the role of a Gaussian process prior?
The prior specifies beliefs about function values at all inputs jointly.
Correct answer: It induces a flexible prior distribution over functions with controlled smoothness and covariance structure.
Robust statistics and misspecification
Robustness principles
Robust statistics seeks methods that perform well under contamination, heavy tails, and model misspecification. M-estimators, Huber loss, trimmed means, median-based estimators, and sandwich variance estimation are central tools.
Robustness toolkit
Which robust estimator is less sensitive to outliers than the sample mean?
The median has a higher robustness to extreme values than the mean.
Correct answer: Median
What does breakdown point measure?
It is a key robustness metric for finite-sample resistance to outliers.
Correct answer: The largest fraction of contaminated data an estimator can withstand before becoming arbitrarily bad.