Why statements about cultural change are probably wrong

Contention: that asking for evidence before believing claims of any particular socio-cultural change is reasonable, because there are more false claims about cultural changes than true ones.

This is aside from the hopefully more obvious position that asking for evidence before believing claims is reasonable.

Standard of Proof

At first glance this might seem like a painful thing to gather evidence for, because socio-cultural trends are:

  1. Spread across a wide variety of topics, each requiring their own special expertise for the interpretation of evidence;
  2. Often not well captured by data, or recorded by many different particular bodies;
  3. Often political topics, with motivated readings of various data points.

This would be a horrible topic to try and gather evidence on, but thankfully there is a bit of a shortcut.

It doesn't matter for the point of debate what the actual truth of any individual claim is, just that there are more false claims than true ones. We can establish this by merely counting unique contradictory claims about socio-cultural changes. For example, claims that:

  1. There is more crime than a year ago
  2. There is less crime than a year ago
  3. There is the same amount of crime as a year ago

are clearly all contradictory, so at least two of them must be false, which is most of them.

My proposed process, then, is to find or create a list of claims about cultural change, and see for how many of them I can find contradictory claims in this manner. Obviously I'll be trying to match the context as closely as possible, so that statements truly are contradictory and not just entanglements of truth (for example, there are lots of claims about the crime rates in particular towns or cities, which are not contradictions of claims about national rates, and of course claims made years ago don't contradict ones made today about short-term trends).

As I see it, there are four possible outcomes from such a survey.

  1. Most claims about cultural change are false.
  2. Claims about cultural change are equally likely to be true or false.
  3. Claims about cultural change are mostly in agreement.

As mentioned, I'm not interested in adjudicating the actual truth behind statements, but I would consider result 2 or 3 a failure for my contention.

Data Collection

Quick searches for ready-made corpora didn't turn anything up. I'll dig somewhat deeper for a bit, and then come up with a sampling process if I have no luck.