Expert credibility in climate change?

I recently stumbled upon one of the most meaningless papers I have ever seen, it is called "Expert credibility in climate change" by Anderegg, Prall, Harold and Schneider. The paper "proves" that the scientists advocating an anthropogenic greenhouse warming (AGW) are statistically more credible than the "unconvinced". Their main goal is to convince people that they should join the AGW bandwagon simply because it is allegedly more credible.

In essence, the authors show that the AGW protagonists have more published papers in climate journals and more citations. The authors then carry on with an elaborate statistical analysis showing how statistically significant the results are. The first thing that popped into my mind is the story about a statistician who proved that 87.54% of all statistical research is meaningless...

Now more seriously. With or without the fancy statistical analysis, and in fact, with or without the data, I could have told you that the scientists in the believer camp should have more papers and many more citations. But this has nothing to do with credibility. It has everything to do with the size of the groups and the way their members behave.

Since the AGW protagonists have the tendency to block the publication of papers that don't follow their party line (and if you think otherwise, read the climategate emails), it is way easier for the AGW protagonists to have any paper get published. Just as an example, the above meaningless paper passed the peer review process of PNAS. And of course it did! It did so because it was much more likely to reach peers in the AGW protagonist group. If I would have tried myself to get a similar paper published, it would have been thrown down the stairs (and rightly so because of its meaniglessness [yes, its a new word]). But any paper my colleagues and I try to publish gets such a hostile confrontation that it is simply very hard to publish at all. The bottom line is more papers for the AGW protagonists and less papers for those who are more critical.

In fact, I have no idea how the "average" "climate expert" could have published 408 climate publications. Over say 30-40 years of activity it means a paper once every month or so. Of course, it could be that the average expert simply contributes just a little to each paper, whereas a denialist expert usually publishes with less co-authors. Here's another possibility the authors didn't consider.

Since there are more protagonist papers around, they cite each other more and violà, you get that the more numerous group has more papers per person and more citations per paper. You don't need to be Einstein to figure this out.

Let me end with a comparison.

In 1912, Wegener came out (like a few others before, I should add) with the idea that continents drift. At worst, he was mostly ignored. At best, he did get attention - he was proven to be wrong. The tide turned only in 1960's when paleomagnetic data showed quite unequivocally that indeed the continents move (today, this can also be measured with GPS). So, for more than 50 years, if one would have carried a similar analysis to W. R. L. Anderegg et al., or to N. Oreskes (Science 306:1686, 2004), one would have reached the conclusion that the truth is with the more credible majority thinking that continents are stuck in their place.

As you can see, science is not a democracy, just by counting people, or more sophisticatedly, counting papers per expert and citations per expert, doesn't imply that the majority or apparently more credible group is correct, irrespective of how fancy the analysis might look. Just do the science and the truth will emerge from it.

Oh, and one last (unrelated) anecdote. Talking about the number of co-authors on a paper, Prof. Shri Kulkarni from Caltech has said something along the following line:

"If you sum up the self-claimed contribution of each author of a multi-authored paper, you'll get roughly the square root of the number of authors."


Comments (7)

  • anon
    Andrew (not verified)

    The numbers aren't even correct! They have Richard Lindzen having 140 papers over his career, but using criteria which were as conservative as I could be with them, from his CV and some very recent papers I am aware of, I get 166. From comments I've seen by others elsewhere, the paper count for several authors appear to be off by very large amounts.

    Jun 27, 2010
  • anon
    Avfuktare Krypg... (not verified)

    This *socialogical* study trying to show that sceptics don't really have the expertise to have a say on AGW was performed by - not a single social scientist (the authors count engineer, blogger and climate scientist).

    If relevant, narrowly defined, expertise is so important when commenting on climate science, why is the total absence of relevant expertise among the authors not important to the quality of their work?

    Jul 01, 2010
  • anon

    Indeed, following their line of argumentation... we shouldn't believe their "non-credible" research...

    Jul 01, 2010
  • anon

    viola > voilà!
    Weneger > Wegener
    Caltech was > Caltech has

    Jan 11, 2011
  • anon


    Jan 04, 2012
  • anon

    The IAC documented "biased treatment of genuinely contentious issues" in its review of IPCC procedures but stopped short of explicitly citing it as "selection bias".
    Publication bias is well known but (in medicine at least) it is generally interpreted to refer to the tendency for journals to prefer papers with positive (as opposed to negative) results. However, the AGW proponents have invented a new form of publication bias - exclusion of any contrary opinion/evidence.

    The body of the IAC report on its review of IPCC's procedures was substantially scathing.

    Aug 29, 2011
  • anon

    Actually, I don't think that it is the first time in history that a group of scientists with some scientific agenda hushes up opponents. And unfortunately, I don't think it will be the last. The main difference is that now there is a lot of political agenda mixing with the scientific one.

    Jan 04, 2012