Statistical Models, Likelihood, and Sufficiency

Manual: General · Subject: Statistics

Develop the likelihood framework and the role of sufficient statistics in efficient inference.

Parametric and semiparametric models

From data-generating processes to likelihoods

A statistical model is a family $\{P_\theta:\theta\in\Theta\}$ . Given data $x$ , the likelihood is $L(\theta\mid x)$ , regarded as a function of $\theta$ with the observed data fixed.

🔑

Likelihood is not probability

The likelihood function need not integrate to 1 over $\theta$ ; it is proportional to how plausible the observed data are under each parameter value.

Common inferential objects

Object	Meaning
Likelihood	Relative support for parameter values
Log-likelihood	Convenient additive transformation of likelihood
Score	Gradient of log-likelihood
Observed information	Negative Hessian of log-likelihood

Which quantity is the gradient of the log-likelihood with respect to $\theta$ ?

Fisher information Score Sufficient statistic Prior

Define a sufficient statistic in one sentence.

Factorization theorem

A statistic $T(X)$ is sufficient for $\theta$ if and only if the likelihood factors as $f(x\mid\theta)=g(T(x),\theta)h(x)$ , where $h$ does not depend on $\theta$ .

Why is the factorization theorem important?

It identifies orthogonality of vectors It gives a practical test for sufficiency It proves all estimators are unbiased It removes the need for sample data

What is the log-likelihood advantage in optimization?

Neuroanatomy Explorer

Memory Deck

Match the Pairs

Concept Constellation

Statistical Models, Likelihood, and Sufficiency

Parametric and semiparametric models

From data-generating processes to likelihoods

Factorization theorem