Linear Models and Generalized Linear Models

Manual: General · Subject: Statistics

Analyze regression, identifiability, estimation, and inference in linear and non-normal response models.

Linear regression as geometry

Normal equations and projection

In the linear model $y=X\beta+\varepsilon$ with $\mathbb{E}[\varepsilon]=0$ and $\mathrm{Var}(\varepsilon)=\sigma^2 I$ , the least squares estimator minimizes $\|y-X\beta\|^2$ and projects $y$ onto the column space of $X$ .

What does ordinary least squares minimize?

Absolute residual sum Squared residual sum Likelihood under any distribution Prediction variance

What is the role of the design matrix $X$ ?

Generalized linear models

GLMs extend linear modeling to non-Gaussian responses with exponential-family distributions. They use a link function $g(\mu)=X\beta$ to connect the mean $\mu$ to predictors.

Examples of GLMs

Response model	Link function	Typical use
Binomial	Logit	Binary outcomes
Poisson	Log	Count data
Gaussian	Identity	Continuous outcomes

Neuroanatomy Explorer

Memory Deck

Match the Pairs

Concept Constellation

Linear Models and Generalized Linear Models

Linear regression as geometry

Normal equations and projection

Generalized linear models