📚 Statistical Rethinking

📚 Statistical Rethinking

BOOK INFORMATION

Statistical Rethinking: A Bayesian Course with Examples in R and Stan
Richard McElreath
2020 (2nd Edition)
488 pages
Statistics/Data Science/Bayesian Methods

KEY TAKEAWAYS

HOOK

Discover why everything you learned about statistics is wrong, and how to build models that actually make sense of the world by embracing Bayesian thinking, causal reasoning, and the humble recognition that all models are wrong but some are useful.

ONE-SENTENCE TAKEAWAY

Statistical modeling isn't about finding the "right" test or procedure but about building scientifically grounded models that acknowledge uncertainty, incorporate causal understanding, and provide honest assessments of what we can and cannot learn from data.

SUMMARY

Statistical Rethinking represents a revolutionary approach to teaching statistics that challenges conventional wisdom and reimagines how statistical modeling should be taught and practiced. Richard McElreath, an evolutionary anthropologist, brings a fresh perspective that bridges the gap between statistical theory and scientific practice, creating a book that serves as both an introduction to Bayesian methods and a critique of traditional statistical education.

The book begins by dismantling the conventional approach to statistics, which McElreath characterizes as a "maze of statistical tests" that confuse students and fail in real-world applications. He argues that the emphasis on null hypothesis testing and p-values has created generations of researchers who apply statistical procedures ritualistically without understanding their purpose or limitations. Instead, McElreath advocates for a Bayesian approach that incorporates scientific knowledge, acknowledges uncertainty, and focuses on building models that actually help us understand the world.

What sets this book apart is its narrative style and philosophical depth. McElreath doesn't just present statistical methods; he tells stories, uses memorable analogies (like comparing statistical models to golems from Jewish folklore), and engages with fundamental questions about the nature of scientific inference. Each concept is introduced through intuitive examples and visualizations before moving to mathematical formalism and computational implementation.

The book's structure is carefully designed as a scaffold that builds understanding progressively. It starts with the basics of probability and Bayesian inference, then moves through linear models, generalized linear models, multilevel modeling, and advanced topics like Gaussian processes and measurement error. Throughout this journey, McElreath emphasizes two key themes: the Bayesian approach to quantifying uncertainty and the use of causal Directed Acyclic Graphs (DAGs) to incorporate scientific understanding into statistical models.

A unique aspect of the book is its integration of causal inference from the beginning. Rather than treating causality as an advanced topic, McElreath introduces causal DAGs early and shows how they can inform model specification and interpretation. This causal perspective helps readers avoid common pitfalls like confounding, spurious associations, and masked relationships that plague much statistical practice.

The computational approach is equally innovative. Rather than focusing on mathematical derivations, McElreath emphasizes simulation-based understanding and provides practical code examples using R and Stan. The book teaches readers to think computationally about statistical problems, using techniques like prior predictive simulation, posterior predictive checking, and cross-validation to build and evaluate models.

INSIGHTS

• Statistical models are golems: Models are powerful but dangerous tools that require careful handling and understanding—the "golem" metaphor emphasizes that models can cause harm when used without proper understanding of their limitations
• Bayesian thinking is natural probability: The Bayesian approach represents probability as degree of belief rather than long-run frequency, making it more intuitive for scientific reasoning and decision-making
• All models are wrong but some are useful: The book embraces George Box's famous quote, teaching readers to focus on building models that are useful for their purpose rather than seeking "true" models
• Causal structure matters: Statistical models cannot have causal meaning without additional assumptions about causal structure, which must be explicitly incorporated through DAGs
• Null hypothesis testing is fundamentally flawed: The book presents a compelling critique of NHST, showing how it misrepresents the scientific process and leads to poor research practices
• Multilevel modeling is a superpower: Partial pooling through multilevel models represents one of the most powerful tools in the Bayesian arsenal, allowing for more nuanced and realistic modeling
• Visualization is essential for understanding: The book emphasizes the importance of plotting for model checking, showing how visualizations reveal insights that numerical summaries miss
• Priors are not subjective but structural: Rather than being subjective, priors represent structural assumptions about the problem and should be chosen thoughtfully and evaluated through prior predictive simulation
• Model comparison is dangerous: The book warns against automatic model selection procedures, advocating instead for model averaging and careful consideration of multiple models
• Scientific modeling is iterative: Statistical modeling is presented as an iterative process of model building, checking, and revision rather than a linear procedure of testing and accepting/rejecting hypotheses

FRAMEWORKS & MODELS

The Bayesian Modeling Framework

This is the central framework of the book, providing a comprehensive approach to statistical modeling:

Components:

• Prior Specification: Choosing structural assumptions about parameters before seeing data, evaluated through prior predictive simulation
• Likelihood Construction: Building the relationship between parameters and data based on scientific understanding
• Posterior Computation: Using computational methods (primarily MCMC) to update beliefs given observed data
• Model Checking: Evaluating model fit through posterior predictive checks and other diagnostic tools
• Model Comparison: Comparing multiple models using information criteria and cross-validation rather than automatic selection

Application: Readers learn to build, fit, and evaluate Bayesian models for a wide variety of problems, from simple estimation to complex multilevel structures.

Evidence: Based on decades of Bayesian statistical theory and practice, with examples drawn from real scientific research across multiple disciplines.

Significance: This framework provides a coherent alternative to traditional frequentist methods that better reflects scientific reasoning and provides more honest assessments of uncertainty.

The Causal DAG Framework

This framework integrates causal reasoning into statistical modeling:

Components:

• Graph Representation: Using Directed Acyclic Graphs to represent causal assumptions and relationships
• Identification: Determining which causal effects can be estimated from observational data given the assumed causal structure
• Model Specification: Using the causal graph to inform which variables to include in statistical models and how to interpret their coefficients
• Counterfactual Reasoning: Using the fitted model to answer counterfactual questions about what would happen under different conditions
• Sensitivity Analysis: Assessing how conclusions depend on causal assumptions

Application: Readers learn to draw causal DAGs for their research problems and use these graphs to guide model specification and interpretation.

Evidence: Based on developments in causal inference from Pearl, Greenland, Robins, and others, with practical examples showing how causal assumptions affect statistical conclusions.

Significance: This framework helps researchers avoid common pitfalls like confounding and provides a systematic way to incorporate scientific knowledge into statistical models.

The Model Building Cycle

This framework describes the iterative process of statistical modeling:

Components:

• Scientific Question: Starting with a clear scientific question rather than a statistical procedure
• Causal Assumptions: Making causal assumptions explicit through DAGs or other representations
• Model Specification: Building a model that incorporates both data and scientific understanding
• Computational Implementation: Using computational tools to fit the model and quantify uncertainty
• Model Evaluation: Checking model fit and comparing alternative approaches
• Interpretation and Communication: Interpreting results in the context of the scientific question and communicating uncertainty honestly

Application: Readers learn to approach statistical problems as an iterative cycle rather than a linear procedure, with each step informing the next.

Evidence: Based on best practices in scientific modeling and data analysis, with examples showing how this iterative process leads to better scientific understanding.

Significance: This framework helps researchers avoid the ritualistic application of statistical procedures and instead engage in thoughtful scientific modeling.

KEY THEMES

• Critique of Statistical Rituals: The book consistently critiques the ritualistic application of statistical procedures without understanding. This theme is developed through examples showing how conventional methods fail in real-world applications and through philosophical arguments about the nature of scientific inference.
• Integration of Causal Reasoning: Causal thinking is woven throughout the book rather than treated as a separate topic. This theme is developed through the early introduction of DAGs and their consistent use in model specification and interpretation throughout the text.
• Computational Thinking: The book emphasizes computational approaches over mathematical derivations. This theme is developed through extensive use of simulation, visualization, and practical code examples that help readers build intuition about statistical concepts.
• Scientific Modeling Over Statistical Testing: The book focuses on building models that help answer scientific questions rather than on testing hypotheses. This theme is developed through examples showing how models can be used to explore relationships, make predictions, and quantify uncertainty.
• Honesty About Uncertainty: The book emphasizes honest communication of uncertainty and limitations. This theme is developed through discussions of posterior predictive checking, model comparison, and the importance of acknowledging what we don't know.

COMPARISON TO OTHER WORKS

• vs. "Bayesian Data Analysis" by Gelman et al: While BDA is considered the Bayesian bible, Statistical Rethinking serves as a more accessible on-ramp with better narrative flow and more intuitive explanations. BDA is more comprehensive and mathematical, while Statistical Rethinking focuses on building intuition and practical skills.
• vs. "Doing Bayesian Data Analysis" by Kruschke: Kruschke's book (the "puppy book") provides a gentle introduction to Bayesian methods but focuses more on basic concepts and less on causal modeling. Statistical Rethinking covers more advanced topics and integrates causal reasoning throughout.
• vs. "Causal Inference in Statistics" by Pearl et al: Pearl's book focuses exclusively on causal inference with heavy emphasis on the mathematical foundations. Statistical Rethinking integrates causal reasoning into a broader statistical modeling context with more practical examples and computational guidance.
• vs. "Introduction to Statistical Learning" by James et al: ISL provides an excellent introduction to machine learning from a frequentist perspective. Statistical Rethinking offers a Bayesian alternative with more emphasis on causal modeling and scientific inference rather than prediction.
• vs. "Statistical Rethinking" 1st Edition: The 2nd edition expands coverage of Gaussian processes, measurement error, and missing data while refining explanations and examples throughout. The core approach remains the same but with additional topics and improved pedagogy.

QUOTES

• "Make no mistake: you will wreck Prague eventually." - This enigmatic opening quote sets the tone for the book's philosophical approach, suggesting that statistical models, like visitors to Prague, will inevitably cause problems if used without understanding.
• "A common notion about Bayesian data analysis is that it is distinguished by the use of Bayes' theorem. This is a mistake." - This quote challenges the misconception that Bayesian statistics is simply about applying Bayes' theorem, emphasizing instead the broader philosophical approach to modeling and inference.
• "People commonly ask what the correct prior is for a given analysis [which] implies that for any given set of data there is a uniquely correct prior that must be used, or else the analysis will be invalid. This is a mistake." - This quote addresses a common misconception about Bayesian priors, emphasizing that they represent structural assumptions rather than subjective beliefs.
• "We don't use the command line because we are hardcore or elitist (although we might be). We use the command line because it is better." - This quote reflects the book's practical philosophy, advocating for tools and approaches that may have a steeper learning curve but ultimately provide better understanding and control.
• "Making choices tends to make novices nervous. There's an illusion sometimes that default procedures are more objective than procedures that require user choice, such as choosing priors. If that's true, then all 'objective' means is that everyone does the same thing. It carries no guarantees of realism or accuracy." - This quote challenges the notion of objectivity in statistical analysis, arguing that thoughtful choice is preferable to rote application of default procedures.

HABITS

• Prior Predictive Simulation: Regularly simulating data from the prior predictive distribution to understand model implications before seeing actual data
• Posterior Predictive Checking: Consistently checking model fit by comparing observed data to data simulated from the fitted model
• DAG Drawing: Drawing causal diagrams before specifying statistical models to clarify assumptions and guide model building
• Model Comparison: Using information criteria and cross-validation to compare multiple models rather than selecting a single "best" model
• Visualization: Creating plots to understand model behavior, check assumptions, and communicate results effectively
• Iterative Model Building: Approaching statistical modeling as an iterative process of building, checking, and revising rather than a linear procedure
• Computational Thinking: Using simulation and computational methods to build intuition about statistical concepts
• Sensitivity Analysis: Testing how conclusions depend on assumptions about model structure, priors, and data quality
• Scientific Question Focus: Starting with clear scientific questions rather than statistical procedures when approaching data analysis
• Uncertainty Communication: Honestly communicating uncertainty and limitations in statistical conclusions

KEY ACTIONABLE INSIGHTS

• Draw Causal DAGs Before Modeling: Always start by drawing causal diagrams that represent your assumptions about the relationships between variables, then use these diagrams to guide model specification
• Use Prior Predictive Simulation: Before fitting models to data, simulate data from your prior distribution to ensure your priors are reasonable and to understand model implications
• Check Models with Posterior Predictive Checks: After fitting models, simulate data from the posterior distribution and compare it to your observed data to check model fit and identify potential problems
• Avoid Automatic Model Selection: Never automatically select the model with the best information criterion value; instead, compare multiple models and consider model averaging
• Embrace Multilevel Modeling: Learn to use multilevel models with partial pooling, as they represent one of the most powerful tools for handling grouped data and complex structures
• Visualize Everything: Create plots to understand your data, check model assumptions, evaluate model fit, and communicate results. Visualization reveals insights that numerical summaries miss
• Think Computationally: Use simulation and computational methods to build intuition about statistical concepts rather than relying solely on mathematical derivations
• Focus on Scientific Questions: Start with clear scientific questions rather than statistical procedures, and build models that help answer those questions
• Communicate Uncertainty Honestly: Always communicate the uncertainty in your estimates and predictions, and acknowledge the limitations of your models
• Iterate and Refine: Approach statistical modeling as an iterative process of building, checking, and revising models rather than a one-time procedure

REFERENCES

• Bayesian Statistical Theory: The book draws on foundational work in Bayesian statistics by authors like Jaynes, de Finetti, Savage, and others, presenting these ideas in an accessible form
• Causal Inference Research: References developments in causal inference by Pearl, Greenland, Robins, and others, integrating these ideas into practical statistical modeling
• Computational Statistics: Incorporates advances in computational methods for Bayesian inference, particularly Markov Chain Monte Carlo and Hamiltonian Monte Carlo
• Scientific Modeling Literature: Draws on literature about scientific modeling and inference across multiple disciplines, including ecology, anthropology, psychology, and epidemiology
• Statistical Philosophy: Engages with philosophical literature about the nature of statistical inference and the role of models in science
• Information Theory: Uses concepts from information theory, particularly KL divergence and entropy, for model comparison and understanding
• Multilevel Modeling Literature: References extensive work on multilevel and hierarchical modeling, particularly by Gelman and others
• Model Checking Methods: Incorporates literature on posterior predictive checking and other model diagnostic techniques
• Scientific Examples: Uses examples from real scientific research across multiple disciplines to illustrate concepts and methods
• Computational Tools: References documentation and literature on computational tools like R, Stan, and various probabilistic programming languages

Crepi il lupo! 🐺