Estimation Theory: Bias, Efficiency, and Asymptotics

Manual: General · Subject: Statistics

Study the properties of estimators, including bias, variance, consistency, and asymptotic normality.

What makes a good estimator?

Finite-sample criteria

For an estimator $\hat\theta$ , key properties include bias $\mathbb{E}[\hat\theta]-\theta$ , variance $\mathrm{Var}(\hat\theta)$ , and mean squared error $\mathrm{MSE}(\hat\theta)=\mathbb{E}[(\hat\theta-\theta)^2]$ .

ℹ️

Bias-variance tradeoff

A small amount of bias can reduce variance enough to lower mean squared error, which is central in modern regularized estimation.

Which identity is correct?

\mathrm{MSE}=\mathrm{Bias}+\mathrm{Var}

\mathrm{MSE}=\mathrm{Bias}^2+\mathrm{Var}

\mathrm{MSE}=\mathrm{Var}-\mathrm{Bias}^2

\mathrm{MSE}=\mathbb{E}[\hat\theta]

What does consistency mean for an estimator?

Asymptotic normality

Many estimators satisfy $\sqrt{n}(\hat\theta_n-\theta_0)\Rightarrow N(0, V)$ . This justifies Wald-type intervals, tests, and delta-method approximations.

The delta method is mainly used to approximate the distribution of what?

A transformed estimator A sufficient statistic A prior distribution A random design matrix

What does Fisher efficiency compare?

Estimator taxonomy

Unbiased estimator

\mathbb{E}[\hat\theta]=\theta

Consistent estimator

\hat\theta_n\xrightarrow{P}\theta

Efficient estimator

Achieves minimal variance among relevant estimators

Robust estimator

Insensitive to modest model misspecification or outliers

Neuroanatomy Explorer

Memory Deck

Match the Pairs

Concept Constellation

Estimation Theory: Bias, Efficiency, and Asymptotics

What makes a good estimator?

Finite-sample criteria

Asymptotic normality