I am starting this blog to share some ideas on Bayesian statistics, computation, philosophy etc.
The title is stolen from de Finetti’s “Probability Theory”:
The full quote is:
“The abandonment of superstitious beliefs about the existence of Phlogiston, the Cosmic Ether, Absolute Space and Time, … , or Fairies and Witches, was an essential step along the road to scientific thinking. Probability, too, if regarded as something endowed with some kind of objective existence, is no less a misleading misconception, an illusory attempt to exteriorize or materialize our true probabilistic beliefs.”
Its a mysterious quote, and although I think I understand it – de Finetti is subtle and it is easy to find parts of his writing contradict your understanding of his position.
The plan with this blog is to give a few key posts that I hope fill a gap in the common understanding about what Bayesian statistics is about. I am dissatisfied with a number of the common sources and philosophical arguments (both for and against Bayesian statistics). I take what I feel is the simple and conservative position of the operational subjective theory by de Finetti, but leads me to the uncomfortable position of disagreeing with famous and well respected Bayesians and of course non-Bayesians.
I will also respond to a number of ideas in blogs / papers etc.
A nice videolecture to start the discussion is the talk by Michael Jordan of Berkeley. As promised, I will, (as uncomfortable as it makes me) start off by disagreeing with this highly admired scientist (admired for very good reason I might add). At one point Jordan says that computing integrals over both the sample space and the parameter space is “a little bit Bayesian and a little bit frequentist”. The type of Bayesian statistics that I wish to discuss the operational subjective theory could be defined as the approach where both these integrals are carried out. This might seem like splitting hairs, but it is important that the type of Bayesian statistics being described is the most philosophically well developed (and sadly this is rarely the case).
Subjective Bayes has two types the “classical subjective” and the “operational subjective”, there are also objective Bayesian and non-Bayesian theories. The operational subjective is one of the least well described on the internet, in classrooms and in books – this blog will attempt to help alleviate this at least a little.
In order to introduce the blog, I will point to some useful references that I will position myself with respect to in future posts.
- This scholarpedia article outlines the classical subjective point of view.
- An unusually good series of blog posts by Greg Gandenberger
- I very much admire Frank Lad’s book which provides the operational subjective theory up front without diversions into arguing against other philosophical positions.
- A more influential book is Bernado and Smith, which bridges between the classical subjective and operational subjective theories (Smith actually translated de Finetti’s probability theory).
- Joseph Kadane has an outstanding free online book.
- Andrew Gelman: very high volume on a number of statistics (and non-statistics) topics. Philosophically very pragmatic (or not very pure). Gelman is one of the driving forces for the amazing stan package (also some very good books).
- Xian Robert: Christian Robert’s blog on statistics and MCMC.
- Deborah Mayo: Mayo is a die hard frequentist. She makes some interesting points about what she calls “howlers” i.e. stories (or jokes) which illustrate counter-intuitive behaviour of frequentist statistics. I will address these at some point.
- Larry Wasserman: Amazing blog, Wasserman is a natural contrarian, this makes for a very lively, technical and interesting blog (sadly now defunct).