When I posted part 1 of my discussion of Untermassfeld, @Iza_Romanowska pointed out that “false positives are infinitely more damaging than false negatives and this should be our guiding principle”. I agree. Archaeologists learn mainly by finding what we don’t expect to find, and even more by finding what we expect not to find. That makes some classes of false positive extremely dangerous.

False negatives

False negatives can still be problematic. Because we understand so little of taphonomy, not finding what we expect to find in archaeology is not much more useful than finding what we expect and know already. Unfortunately, absence of something expected is often treated as a significant signal. A false negative can be damaging to archaeology in the long-term, especially that in many circumstances, it is almost impossible to develop criteria to distinguish between true and false negatives.

We all know that the archaeological record is a fragmentary and distorted signal of past behaviour, and that it is fragmentary and distorted in ways that we don’t understand. Yet we often insist on “believing” negative results, for example, that there are no Australopithecenes in Central Asia 3 million years ago, or that there are no fluted points in North America 18 000 years ago, simply because none have been found despite being looked for.

Given the almost complete obscurity of taphonomic processes, combined with the scarcity and patchiness of archaeological effort on the ground, the only rational position in the face of absence of archaeological evidence must be agnosticism. Accepting negative results as real encourages us to stop research in particular areas, whereas the history of archaeology repeatedly shows that perseverance pays off. This is not surprising, as archaeological fieldwork is essentially a low-odds game of chance. False negatives can slow the pace of research in some areas and can keep us from bumping into interesting stuff for longer than would otherwise be the case. While false negatives can cause significant damage, false positives can be devastating, especially when they take the form of an unexpected find.

Expected false positives of something previously documented

Finding what we expect to find is obviously not terribly useful. Yes, it helps us fail to reject our understandings of the past, but it adds no new significant knowledge. Once we know that there are Australopithecenes 3 million years ago in East Africa, or fluted points in North America 12 000 years ago, finding more does not add much to our knowledge.

False positives that fall into this category are likely not too damaging. Identifying naturally broken quartz as the result of tool-making on an already known site on which quartz tools are found is not good, but the damage will probably be limited. It won’t force us to significantly revise our existing understanding of the past.

Expected false positives of something previously undocumented

This is perhaps the most dangerous of the false positives. The Piltdown hoax falls squarely into this category. Piltdown perfectly fit the expectations of those who were looking for it. It confirmed the big brain first model that dominated human evolutionary theory at the time, and reinforced the idea that Europeans were most culturally advanced because they had had the most time to move up the scale of progress. Piltdown made perfect sense in its context, and it was a very costly false positive. These are the ones we have to guard against most assiduously, but they are also the ones to which we are most vulnerable, and the ones that are most damaging.

Unexpected false positives

Finding something unexpected is always helpful. It forces us to question our understanding of the past and often sends us in new and interesting directions. I personally think we don’t encourage enough blind exploration in archaeology, especially in surveying. Being surprised is productive.

A false positive of something unexpected will lead us to ask new questions that may not be productive in the long-term, and could divert significant effort down an unproductive dead-end until it is identified as a false positive.

Contrary false positives

We learn most in archaeology when we find something we are expecting not to find. If our understanding of the past is accurate, this find should not be there. This is about the only clear, unambiguous signal we can get in archaeology. It tells us very clearly that our understanding of the past is wrong and that we need to revise it. Because it is such a clear signal, and because it forces us to change the way we think about the past, a contrary false positive can be as damaging as an expected false positive of something not previously documented.

However, we are somewhat better protected against contrary false positives than against something like Piltdown, because we are naturally driven to question contrary findings. But should the false positive make it through our filters, it risks doing serious damage. Another problem is that because we are so well protected against contrary false positives, we risk ignoring true contrary finds. The balance of critical thinking between rejecting contrary false positives and accepting true contrary finds is a very high-stakes game for archaeologists, with potentially catastrophic consequences if we get it wrong one way or the other.

Which brings me back to the usefulness of the idea of a purgatory of possibles in archaeology. We have to be reluctant to reject possibilities unless a contrary find (rather than the absence of finds) forces us to. At the same time, we have to be very critical of proposed positives. We have to maintain a large store of possible understandings of the past and be comfortable exploring them.

One thought on “Untermassfeld reflections part 3 : The damage that false positives do

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s