Bayesian Statistics

In research, we use statistics to help us reach conclusions about our observations. Traditionally in psychology, we use something called a p-value to determine whether some effect we are interested in (such as whether children in the Parents as Teachers program are better prepared for kindergarten compared to similar children not in the program.) Much research you read about in the popular press (whether psychological, biomedical, etc.) was tested relying on p-values.

What is a p-value? It’s actually super confusing and most people (including well-educated people who use p-values) don’t understand it! Even those who understand the mathematical and conceptual basis for a p-value have difficulty explaining it in a way that others can understand.

If you want a definition here it is: The p-value tells you the probability of getting a result as extreme as the one you observed, assuming that there is no effect in the real world and if you ran your study a gazillion times. By convention, if that probability is less than 5% (or 1 in 20), then you reject the idea that there is no effect and decide that there must, in fact, be an effect. A p-value does not tell you the probability that you are right or wrong (or the probability that the results occurred by chance). Is that clear? No? You’re not alone.

Furthermore, over-reliance on p-values contributes to all sorts of questionable research practices, some intentional, some unintentional, many somewhere in the middle.

There has been an increasingly loud call by statisticians and statistically-minded psychologists to move away from reliance on p-values. Some have called for something called confidence intervals, which I won’t get into here. Others have called for the use of Bayesian statistics, which is what I’m learning this week!

I’m taking a class at the ICPSR Summer Institute, which offers week-long workshops. I’m taking Introductory Bayesian Statistics with John Kruschke. There are several benefits of Bayesian statistics over more traditional analyses. And after Day 2, I’ve become convinced that it is more intuitive than p-values. However, you have to talk about probabilities a lot, which makes most people’s heads hurt. (There are a lot of Greek letters, making “It’s all Greek to me!” jokes both nearly irresistible and very cringe-worthy.)

All that Greek and expression of probabilities make it seem much more complicated than traditional statistics. (It’s also possible I’ve forgotten how confusing traditional stats were the first time I learned them.) However, the results are more in line with what most people think p-values and confidence intervals are telling them. And it gives more info out of a single analysis.

That said, I’m only 2 days into Bayesian statistics, so I’m faaarrrr from an expert. But I’m really excited about what I’m learning. I’m excited to help my students interpret Bayesian results, if not the math or underlying structure. I’m also excited to try some analyses on my own data.

An important caveat from a non-expert: There’s a lot of responsibility on the part of the researcher be as honest as possible about their data and results. Some of the problems of the p-value are about its interpretation rather than something inherent in p-values. I’m sure there are ways in which uninformed, unscrupulous, and/or highly pressured researchers can mis-specify, misinterpret, or distort Bayesian analyses. I doubt its a cure-all, but I remain excited! (Here is a discussion of how Bayes won’t save us all.)