Part V: Advanced and Research Topics

This part of the guide ventures beyond the classical likelihood toolkit and into the broader landscape of modern statistical inference. We begin with the deep connections between likelihood and Bayesian reasoning, move through the computational machinery that makes Bayesian and high-dimensional inference feasible, survey the principled methods for choosing among competing models, tour the research frontiers where likelihood ideas meet machine learning, and close with the numerical craftsmanship that underpins every reliable implementation.

Each chapter is self-contained but builds on the foundations laid in Parts I–IV. Cross-references point back to earlier material whenever a concept is reused.

Advanced Topics

• Chapter 17 – Bayesian Connections
  • 17.1 Bayes’ Theorem for Parameters
  • 17.2 Prior Specification
  • 17.3 Conjugate Priors
  • 17.4 MAP Estimation
  • 17.5 The Laplace Approximation
  • 17.6 Credible Intervals vs Confidence Intervals
  • 17.7 Bayesian Model Comparison
  • 17.8 Summary
• Chapter 18 – Computational Methods
  • 18.1 Monte Carlo Integration
  • 18.2 The Metropolis–Hastings Algorithm
  • 18.3 Gibbs Sampling
  • 18.4 Hamiltonian Monte Carlo
  • 18.5 Convergence Diagnostics
  • 18.6 Variational Inference
  • 18.7 Integrated Nested Laplace Approximations (INLA)
  • 18.8 Comparison of Methods
  • 18.9 Summary
• Chapter 19 – Model Selection
  • 19.1 The Bias–Variance Trade-off
  • 19.2 Akaike Information Criterion (AIC)
  • 19.3 Bayesian Information Criterion (BIC)
  • 19.4 Deviance Information Criterion (DIC)
  • 19.5 Widely Applicable Information Criterion (WAIC)
  • 19.6 Cross-Validation
  • 19.7 Likelihood Ratio Test for Nested Models
  • 19.8 Model Averaging
  • 19.9 A Practical Decision Guide
  • 19.10 Summary
• Chapter 20 – Modern Research Frontiers
  • 20.1 Automatic Differentiation
  • 20.2 Neural Density Estimation
  • 20.3 Simulation-Based Inference (Likelihood-Free)
  • 20.4 Amortized Inference
  • 20.5 Normalizing Flows
  • 20.6 Variational Autoencoders (VAEs)
  • 20.7 Score Matching
  • 20.8 Connections Back to Classical Likelihood
  • 20.9 Summary
• Chapter 21 – Numerical Considerations
  • 21.1 Floating-Point Arithmetic
  • 21.2 Working in Log-Space
  • 21.3 The Log-Sum-Exp Trick
  • 21.4 Numerical Differentiation
  • 21.5 Numerical Hessians
  • 21.6 Condition Numbers
  • 21.7 Cholesky vs Inverse for Solving Linear Systems
  • 21.8 Parameterization Matters
  • 21.9 Sparse and Structured Hessians
  • 21.10 Parallel and GPU Computation
  • 21.11 Software Libraries
  • 21.12 Summary