Bayesian Inference and Decision Theory
Bayesian Updating
Posterior distribution
Bayesian inference combines prior belief with likelihood to obtain the posterior . The posterior is the basis for estimation, prediction, and uncertainty quantification.
Posterior summaries
Common summaries include the posterior mean, median, mode, credible intervals, posterior predictive distributions, and Bayes factors. The choice of summary depends on the decision problem and loss function.
Decision-theoretic framing
A Bayesian decision rule minimizes posterior expected loss, not merely posterior variance or posterior entropy.
Loss functions and optimal decisions
Risk
For action and parameter , the loss is . The posterior expected loss is . The Bayes action minimizes .
Under squared error loss, what posterior summary is Bayes-optimal?
The posterior mean minimizes expected squared error.
Correct answer: Posterior mean
What is a credible interval?
For example, a credible interval has posterior probability .
Correct answer: An interval that contains the parameter with a specified posterior probability under the posterior distribution.
Computation and modern Bayesian methods
Approximate inference
Exact posteriors are often intractable. Markov chain Monte Carlo, variational inference, Laplace approximation, expectation propagation, and sequential Monte Carlo are central computational tools.
Method comparison
Which method typically converts posterior inference into an optimization problem?
Variational inference approximates the posterior by minimizing divergence in a parameterized family.
Correct answer: Variational inference
Why is a Bayes factor used cautiously in practice?
Evidence comparison depends strongly on the chosen priors and model classes.
Correct answer: Because it can be highly sensitive to prior specification and model misspecification.