There is still no evidence that Gunung Padang is an ice age pyramid, and I am still not a fan of retraction

The peer reviewed paper published last year in Archaeological Prospection that claimed Gunung Padang in Indonesia is a 25k year old pyramid has now been retracted by the journal. As I noted at the time, the paper does not support its main claim. It doesn’t provide evidence that Gunung Padang is anything but a volcanic hill with a recent (couple of thousand year old) archaeological site and landscaping near the top. An impressive and significant archaeological site, but without evidence that it was built by the survivors of Graham Hancock’s Ancient Apocalypse.

Danny Hillman Natawidjaja, the lead author of the retracted paper, has now released the correspondence between the research team and the publisher that led to the retraction. It makes for very instructive reading, and is full of lessons, both for the public and for researchers.

A few things to clarify first. When researchers say that the Natawidjaja et al. paper does not support its claims, they are not necessarily saying that Gunung Padang is not a pyramid, that it isn’t older than 14 000 years, or that it is impossible that it is an ancient pyramid. They are saying that the paper does not prove that it is. The difference may seem subtle to some, but it is important.

Some of those academics who do say that Gunung Padang is not a pyramid made by humans 14k years ago are using an unfortunate shorthand to say that the paper simply doesn’t show that the claim is true. As academics, we use those shorthands often with each other, but they don’t serve us well when we interact with the public (or with our students), and we should always keep this in mind. 

Certainly at least some academics mean it literally when they say that Gunung Padang isn’t an ancient pyramid. Strictly speaking, they are overstating their case just as much as Natawidjaja and his team. It could be, although it isn’t likely, but no one, including Natawidjaja et al. has shown that it is.

The exchange: A matter of burden of proof

In the email exchanges between the Natawidjaja team and the publisher, the publisher relays concerns from researchers that the paper does not support its central conclusion. The publisher focuses on what they call “a major scientific error” in the paper: “The soil samples have no associated cultural materials (no archaeological artifacts of anything connecting them to humans), and so they only date the natural materials that make up the soils. In fact, these dates may not even accurately date the soils, which are notoriously difficult to date.”

In other words, you can’t just radiocarbon date some soil around the base of an andesite outcrop and claim that you have dated an ancient pyramid. You actually need some indication that humans were involved at some point. Of course, that is only one of the problems with the paper, and in my view not the most serious one, but it is probably enough to earn a retraction, and it is the easiest to substantiate.

The main problem with the paper for me is that it consists of claims and doesn’t provide evidence to support those claims. The main claim is that 1) Gunung Padang is not a natural volcanic hill but is in fact “sculpted and then architecturally enveloped during the last glacial period between 25 000 and 14 000 BCE” (Natawidjaja et al. 2023).

The authors respond to this by trying to reverse the burden of proof and put it on the reviewers. They simply repeat their claim that “our geophysical data extensively supports the assertion that Gunung Padang is a substantial pyramid-like structure, not merely natural hills with surficial stone terraces (Unit 1).” 

Later, they reinforce this with “Experts contesting the artificial nature of Units 1, 2, and 3 must substantiate their claims with tangible evidence and comprehensive descriptions to refute our findings.”

Actually, no, as the kids would say. That’s not how it works. It is up to Natawidjaja et al. to show that their observations suggest or establish that Units 1,2, and 3 are artificial. The reviewers are simply saying that they don’t see how the evidence presented supports the claim. The reviewers are not making the claim. They are simply looking for evidence in the paper that the claim is supported.

Both in the article and in the exchange with the publisher, Natawidjaja et al. rely on what they repeatedly refer to as their “observations” of Gunung Padang to support their claim that it is not a natural hill. The missing ingredient is comparison with surrounding and similar volcanic hills.

It isn’t enough simply to claim that “In nature, columnar rocks are typically perpendicular to their flow directions, mostly aligned with the geological layer.” It may be true, but I wouldn’t know that from reading the paper itself, or the rebuttal to the reviewers. 

One has to provide evidence that this is so at surrounding hills, and that Gunung Padang sticks out like a sore thumb in that comparison. Some quantification (i.e. measurement) of how Gunung Padang differs from its neighbours would be more than welcome.

Even with essentially nothing more than wikipedia knowledge of columnar basalt (which, I don’t mind saying, is what I have), I can evaluate that the paper and its defense by the authors don’t come close to adequately supporting their claims, simply from a basic research design perspective. Nevermind the geology.

If, for instance, I want to claim that object A, whatever it may be, is fundamentally different from all similar surrounding objects, I can’t just describe object A and be done with it. I have to show how objects B, C, D, etc, are all pretty similar to each other, and that A is completely unlike them. Then I can start looking for a reason for which A is somehow different.

To apply this to the specific case, if Gunung Padang is part of a group of volcanic hills in a region, the null hypothesis (the starting point of the investigation, essentially) should be that it is a natural hill like the others. Then the researchers test the hypothesis that it is the same as the others. They do this by comparing it to others. 

The paper could have started by showing that Gunung Padang hill is structurally different from, say, 10 other hills in the region. To stay on the theme of columnar basalt, this could have involved graphs showing the angle of the columns relative to the geological layer in which they are found, to use the article’s terms. 

If all other 10 hills showed a perpendicular angle and Gunung Padang was the only one to significantly deviate from that, then there would have been a reason to start asking whether it was modified by humans, and that would still have been only one of many possibilities to investigate.

The additional material presented by the authors in their rebuttals, contrary to their claims, doesn’t do this either. It merely restates their position, using the same data, still about Gunung Padang.

What does it mean that the paper is retracted?

I have seen some reactions online that equate the article’s retraction with “censorship”, or “suppression.” The retraction is neither. The article is still available online, and now the correspondence that led to the retraction is also available. Together, they make for some very interesting reading.

Formally, a retraction means that the editors and publishers of the original paper no longer think that its results are reliable, or its claims adequately supported. That’s really all it means.

Unfortunately, retractions also have professional consequences for those involved in them. They are widely perceived as some kind of academic punishment, and if the cause of the retraction is outright scientific fraud, like making up fake data (which is not the case here), they are sometimes followed by actual sanctions.

I agree with Natawidjaja et al. that in this case, a more productive approach would have been “that the critiques from the third parties and our responses be published together as a Letter to the Editor or Commentary.”

The kind of retraction we are dealing with here is the result of a discussion about what the evidence shows and what it doesn’t, and how it relates to the claims made in a paper. That discussion is helpful in all kinds of ways, from actually evaluating the claims, to public education, etc.

As we saw from some of the online reactions, there is also the fact that the retraction itself becomes a “thing.” There was an opportunity here to focus the discussion on the reasons for not accepting the claims made in the paper, and instead we are discussing the retraction. That’s not ideal. We also happen to apparently conform to the picture that people like Hancock want their audience to have of us. We shouldn’t overly worry about this last, but we shouldn’t completely ignore it either.

Why wasn’t this caught in peer review?

According to the publisher, “”The peer review process for this article focused on the geophysical prospection aspect of the paper, resulting in comments and revisions that improved the article’s presentation of geophysical methods and results. After publication, experts in archaeological interpretation and radiocarbon dating contacted the journal, sharing concerns that the archaeological evidence did not support the conclusion that the site is an ancient pyramid.”

That’s a pretty clear indictment of the narrow pre-publication review system that we currently have, and a vindication for those who argue for a broad post-publication model. Especially for such an interdisciplinary paper, I wouldn’t expect any two or three individuals, which is the number involved in a typical peer review, to be specialists in everything required. Nor would I expect the editor necessarily to know which experts to mobilize, or for those experts to be available on short notice.

The problem in this case, as in many others, is not that the review “failed.” The problem is that the public, the media, and even many academic colleagues have been trained to expect peer review to be an arbiter of truth, when it can’t be expected to be more than a first approximate filter. That’s a completely academically self-inflicted injury.

I would say the consequences are much worse when that first filter gets it wrong in the other direction, that is when it rejects a paper that should have gotten through. Yet in many ways, we have constructed a system in which a false positive becomes a much bigger catastrophe than a false negative.

There is little harm in letting an unsupported or even a false claim through, if active, broad, post-publication review is a well established tradition, is expected by the academic community, by the public, the media, and policy-makers, and if it isn’t seen as an attack but as a further contribution.

I would rather see ten papers like this one get media coverage than miss a truly important one because the wrong mix of experts were on the initial review. Even if in the end, it is more work for me.