The jury is still out on moralizing gods, but the initial response by Whitehouse and Francois is not encouraging

The latest salvo in the saga of the moralizing gods comes from Whitehouse and Francois. Following two critiques of their recent Nature paper on the role of moralizing gods in the evolution of large, complex societies, one by Beheim et al. and one by Slingerland et al., they have posted a preliminary response on co-author Peter Turchin’s blog. There are four main elements in their response, only one of which actually addresses their methods. Let’s start with the other three, and then move on to methods.

Whitehouse and Francois reiterate the importance of a scientific, data-driven approach to history, to test hypotheses about cultural evolutionary processes. I could not agree more. As I acknowledged in a Twitter reply to Turchin earlier this week, I am very interested in this paper and its critiques precisely because it has great potential to shed light on some fundamental questions about cultural evolutionary processes.

They lament that the debate around their paper is happening outside the bounds of traditional journals, and therefore without the benefit of peer review. They miss the important point that the activity of the past week is the very essence of peer review.

The posted critiques of their paper are post-publication peer reviews. Every tweet and blog post that addresses substantive points of their original article or of the critical papers is an instance of post-publication peer-review. This whole exercise strongly reinforces my belief that broad-based, post-publication peer review is much more effective than its narrower pre-publication cousin.

A broader range of practitioners with a greater diversity of expertise gets to participate in the evaluation of papers much more quickly than under the narrow pre-publication review system. The reaction to Whitehouse et al. is a shining example of the power both of post-publication peer review, and of rapid dissemination of papers through online platforms.

Unfortunately, instead of focusing on the very substantive critiques leveled at their paper, Whitehouse and Francois spend some time questioning the motivations of their critics. “Those leading the criticisms against us are closely associated with a rival database which is at a much earlier stage of development but which may hope to catch up if only we can be slowed down”.

It is certainly true that many (most?) of the authors involved in the critique so far are associated with The Database of Religious History, a database like Whitehouse et al.’s Seshat. It may even be true that it is a rival database. If I remember correctly, the two projects have a common origin.

It turns out I was a discussant at one of their seminars in 2015. I remember mostly reminding participants in The Database of Religious History (the rival database) of the serious limitations of their questionnaire-based, short-term field observation data on beliefs in moralizing gods. But that’s for another post.

In any case, none of these parish rivalries should have any bearing on whether the critiques articulated are serious and valid. The fact is, critiques have been articulated, and they seem serious on their face. I would like to see some serious responses.

Whitehouse and Francois dismiss in one short paragraph what I see as the most substantive and serious critique so far, from Beheim et al. Pointing to an explanation from Savage, they address Beheim et al’s subsidiary critique that correcting forward bias in the data allows the main result to be reversed.

But they don’t mention at all Beheim et al’s much more serious critique that Whitehouse et al code “not available” as “absent” in their database, thereby creating a situation in which societies in which there is no fairly extensive written record, show up as having no moralizing gods. Bret Beheim posts a very clear and succinct reminder of this main critique in the comments section below the post.

Savage does mention the “not available” vs “absent” issue, but his response to it makes it seem that this is a conscious assumption deeply embedded in the analysis, rather than an error. I am not sure I understand his defense on that point. I am left with the feeling that Savage doesn’t read that critique the same way I do. I need to think about that some more.

As for the data quality critique from Slingerland et al., whose most important point in my view is not the quibbling over who’s experts are more wrong, but rather the extent of interpolation in time and space in the original analysis, they don’t address it at all, beyond questioning the motives of the critics, and saying that they are preparing a rebuttal for publication.

So of course, I very much look forward to substantive responses from Whitehouse et al., but their initial reaction to the critiques is not encouraging.