Quality Control Sorely Needed In Climate Science: Half Of Peer-Reviewed Results Non-Replicable, Flawed.

“A number of biases internal and external to the scientific community contribute to perpetuating the perception of ocean calamities in the absence of robust evidence.” Duarte et al., 2015

Image Source: Larcombe and Ridd, 2018

Within a matter of days after the press release for a newly published Nature paper spewed the usual it’s-worse-than-we-thought headlines throughout the alarmosphere (Washington Post, BBC, New York Times), the paper’s results were assessed to have “major problems” by an author of multiple CO2 climate sensitivity papers (Lewis and Curry, 2015, 2018).

A glaring miscalculation was quickly spotted that changed not only the results, but consequently undermined the conclusion that estimates of climate sensitivity to doubled CO2 may be too low.

And yet the paper was able to pass through peer review anyway.

Dr. Michael Mann’s error-riddled 2016 paper

A few years ago Dr. Michael Mann was the lead author of an embarrassingly non-scientific paper fraught with glaring methodological and statistical errors.

A post-publication reviewer (statistician Dr. William Briggs) wrote in his point-by-point critique of the paper that “Mann’s errors are in no way unique or rare; indeed, they are banal and ubiquitous.”

Despite the glaring errors, the paper made it through peer-review and was published in Nature‘s Scientific Reports journal anyway.

“Hoax” papers can get published in 70% of peer-reviewed journals

Analyses indicate that “fake peer review” often goes undetected, and as many as 7 of 10 peer-reviewed journals are apt to publish a deliberately-written “hoax” paper.

Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s shortcomings immediately. Its experiments are so hopelessly flawed that the results are meaningless. … The hoax paper was accepted by a whopping 157 of the journals and rejected by only 98. Of the 106 journals that did conduct peer review, 70% accepted the paper”  (Murphy, 2017  The Failure of Peer Review)

A new paper cites analyses that find half of peer-reviewed science results are flawed, not replicable

Earlier this year, a review paper (Larcombe and Ridd, 2018) published in the journal Marine Pollution Bulletin delivered a stinging rebuke to the modern version of science’s disturbing lack of replicability and verifiability.

Image Source: Larcombe and Ridd, 2018

The authors go on to detail a large volume of examples when peer-review failed to detect errors in Great Barrier Reef (GBR) coral research.

Confirmation bias appears to permeate the peer-reviewed literature, slanted in the direction of finding evidence for catastrophic decline in coral health.  This isn’t the first time that marine research has been called out for overselling calamity (see Cressey, 2015, “Ocean ‘calamities’ oversold, say researchers – Team calls for more scepticism in marine research.”) and falling “into a mode of groupthink that can damage the credibility of the ocean sciences”.

As just a single example among the many provided, Larcombe and Ridd reviewed the De’ath et al. (2009) study in which an “unprecedented” decline in GBR corals was alleged to have occurred between 1990-2005.

After a reanalysis of the measurements and methods used, Larcombe and Ridd used corrected data to show there has actually been “a small increase in the growth rate” of corals since the early 1900s (see below image) instead of the dramatic decline after the 1990s documented in the peer-reviewed paper.

These errors slipped past the reviewers’ notice too.   The publication of flawed results has seemingly become so common that it’s no longer even surprising.

“This paper [De’ath et al. (2009): Declining coral calcification on the Great Barrier Reef.] studied 328 corals on the GBR, and indicated a 14% reduction in growth rates between 1990 and 2005. It stated that the corals of the GBR are declining “at a rate unprecedented in coral records reaching back 400 years”. Subsequent reanalysis of the data indicated that the apparent recent reduction in growth rate was caused by a) problems with the physical measurements of calcification, which systematically biased recent growth bands to give lower growth rates (D’Olivio et al., 2013; Ridd et al., 2013), and b) an unjustified assumption that coral growth rate does not change with the age of the coral (Ridd et al., 2013). With these taken into account, the dramatic fall in growth rate after 1990 is no longer evident, and a small increase in growth rates since the early 1900’s appears (Fig. 6). Further, D’Olivio et al. (2013), working on a different set of GBR corals, showed an increase in coral calcification rates on middle and outer shelf reefs, which together represent 99% of GBR corals, of 10% for the period ~1950 to ~2005, but a decrease of 5% per decade between 1930 and 2008 on inner-shelf reefs, which represent only 1% of GBR corals. Therefore, it would be hard to glean from these datasets that there is a documented decline in coral ‘growth’ parameters, and even harder to attribute change to a particular cause.”

Image Source: Larcombe and Ridd, 2018

24 responses to “Quality Control Sorely Needed In Climate Science: Half Of Peer-Reviewed Results Non-Replicable, Flawed.”

  1. Georg Thomas

    What people do not understand is that science IS NOT AN AUTHORITY but A PROCESS OF CHALLENGING AUTHORITY.

    A very primitive, immature and unscientific conception of science has captured the public mind — science as the repository of ultimate truth. The dignity and authority of good science — which is always fallible and hospitable to questioners — is being abused to make people believe in ultimate truths when the purview of genuine science is not final but incessantly evolving knowledge.

    By debasing science to the status of ultimate source of apodictic truth, the charlatanry is invited of those who are eager to impress the seal of unquestionable authority on their agendas, no matter what.

    Some will act this way because they are dishonest, even quite consciously so; others succumb to dazzlement and self-conceit. Both have lost touch with the scientific method.

    As science is moved away from its critical-corroborative capacity toward a role of validator of what is socially acceptable, the type and attitude of those associated with it shift away from the profile of the soberly objective researcher toward the zealot — high priest or humble believer — of socially fashionable articles of faith.

    At some point the latter type is so predominant in terms of power and numbers that “scientists” no longer understand the point of science, having lost the capacity for rational self-control. At this point, a modern society tilts back into an age of mythical thinking.

    1. Robert Folkerts

      Georg

      Great summary. Public need to be educated about this !!

    2. SebastianH

      is being abused to make people believe in ultimate truths when the purview of genuine science is not final but incessantly evolving knowledge.

      Keep that in mind next time you guys go on and on about 30 year old predictions … climate science is evolving as well. That’s why there are adjustments, better models and better understanding of the mechanisms today than in previous times. Skeptics however seem to be somehow drawn to the past.

      1. Adrian

        Better models? Are you for real? Better for what? For fooling people? That’s why they can have in them purely imbecile bugs like this one: https://github.com/ddbkoll/PyRADS/issues/2 Exactly because they cannot be tested. They don’t predict, they only emit bullshit. They might get better at that, but for prediction, nope. A real scientific model can be tested against reality and such idiotic bug could not survive. Without that check against reality, such a bug could survive in a computer model forever. In this case, they were lucky I looked into it, annoyed being by ignorant individuals that blindly believed stories of Earth going Venus as the propaganda said that this computer model said (no, it did not, but the propaganda said it nevertheless). Next time you put your ignorant blind faith in climate computer models, be certain you checked them all line by line. They might have thousands of errors. Even without those errors, the Lyapunov exponents still take care of their exponentially amplified bullshit, but at least you could claim they are ‘better’.

  2. Don from OZ

    WOW Georg WORDS of WISDOM and TRUTH

  3. Penelope

    Oh dear, I’m afraid that the juxtaposition of “quality” with “control” makes me uneasy. I should almost say that science publishing is too controlled now. If each climate journal were independent I’m sure quality would rise. I tried to find out how many publishers control the journals, but was unsuccessful.
    Does anyone know?

    L.A. Times is determined to increase the quality of its climate science coverage:

    “Newspapers should be truthful. That goes for every single page.
    The L.A. Times recently won national attention and praise for spelling out its policy of refusing to publish the claims of climate deniers.”

    “Forecast the Facts, a project that aims to improve the quality of coverage of climate change in the press, launched a petition calling on the New York Times, Washington Post, USA Today, and Wall Street Journal to refuse to print letters that deny basic science.”
    https://grist.org/climate-energy/readers-to-papers-stop-publishing-letters-that-deny-climate-reality/

    Sorry, I don’t think “control” is going to give us quality.
    I don’t know how to get quality, but we can all agree that decentralization of control is necessary, no?

    Regarding the MSM’s part in supporting the AGW hoax, I note that more than 100 companies controlled US major media in the 80s, and today 95% of it is controlled by 6 companies.

  4. Skeptik

    The percentage of rubbish papers that pass the peer review process appears to differ widely between one scientific discipline and another. Medicine seems to generate a much higher percentage than,say, solid state physics.

    This is probably as it should be. In the case of rare diseases it is unlikely that any one institution will see enough examples for any apparent pattern to pass the normal tests of statisitical significance. It is only ny stimulating other researchers in other institutions to examine their own data that we can, eventually, end up with a sample size large enough to determine whether the apparent pattern is really meaningful.

    In examining the performance of peer review we need to look at rejection percentageas as well as acceptance ones. How effective is peer review at rejecting dross and how often does is reject gold dust?

  5. SebastianH

    Quality Control Sorely Needed In Climate Science: Half Of Peer-Reviewed Results Non-Replicable, Flawed.

    As opposed to skeptic’s papers which are totally replicable? 😉

    https://link.springer.com/article/10.1007/s00704-015-1597-5

    Look, peer review isn’t perfect, but it’s what we’ve got. It’s funny that you attack peer review here while at the same time you are amongst the skeptics that emphasize “peer reviewed” all the time when talking about new skeptic’s papers …

  6. Nigel Franks

    Rather a misleading headline: it implies thatthe 50% applies to climate science yet you don’t offer any evidence in the article.

  7. Dr Tim Ball - Historical Climatologist

    On Feb 13, 2018: The judge dismissed all charges in the lawsuit brought against Dr Tim Ball by BC Green Party leader Andrew Weaver. It is a great victory for free speech.
    ‘The Deliberate Corruption of Climate Science’.
    https://www.youtube.com/watch?v=tPzpPXuASY8
    “Human Caused Global Warming”, ‘The Biggest Deception in History’.
    https://www.youtube.com/watch?v=tPzpPXuASY8
    https://www.youtube.com/watch?v=sO08Hhjes_0
    https://www.technocracy.news/dr-tim-ball-on-climate-lies-wrapped-in-deception-smothered-with-delusion/
    http://www.drtimball.com

  8. Yonason

    QUALITY ASSURANCE vs QUALITY CONTROL

    “Quality Assurance is process oriented and focuses on defect prevention, while quality control is product oriented and focuses on defect identification.”

    With any standardized “product” you will always have defects. ALWAYS! The goal is to minimize the number and impact of those defects. That is the job of Quality Control.

    What then is the “product” being produced, and how are “defects” in it identified and minimized, both during development and after it is completed?

    What is the standardized completed “product” we are talking about? Isn’t it just the finished scientific paper? But isn’t each and every one by definition unique? So, how does one exercise Quality Control over something to which it does not, and cannot apply?

    The closest we can come to that is to have the product examined by other scientists with expertise in the same or closely related fields, and hope that they are knowledgeable and honest enough to catch real flaws that can be fixed, and not reject the product for reasons other than real defects that can’t be fixed. We already have that. It’s called Peer Review.

    But hasn’t this question arisen because peer review itself isn’t working properly? So, how do we control for a situation in which the controllers in place are themselves defective? I think it’s obvious that more of the same is probably not the best solution to that problem.

    Well, what about Quality Assurance? After all, it’s purpose is to prevent defects during product development. Perhaps this is the most appropriate tool to use? And, even though the “product” is unique, one still wants it to be as free as humanly possible of defects.

    So, who are most qualified to identify and eliminate defects during “development” of this “product?” Aren’t they the scientists doing the research? Of course they are, and that’s what they do if they are any good at it. Anyone who has ever been involved in scientific research knows that competent and honest scientists already do this. They review the results every experiment in excruciating detail before planning and proceeding to the next. They have to. It’s integral to the process.

    But if we already have mechanisms in place to address the issues, and they aren’t working, what more can we do? After all, they should and often do work. So why don’t they always work?

    As long as one has to rely on the competence and integrity of QA/QC teams, who are themselves not free of defects, we can’t expect a different outcome. And no amount of added bureaucracy will change that.

    The appropriate processes are in place. It’s not the processes, it’s the people involved who need to change.

    One of the main problems is that Universities and government agencies tolerate substandard work because it brings in more money than honest work does. Climategate showed us what happens when scientists and research are sold to the highest bidders. It’s not pretty. The levels of oversight that already exist were easily hijacked by them. I don’t see how giving them another level to pervert could possibly be a solution.

    1. SebastianH

      Climategate? Why is this even a thing? You skeptics are making up a hoax story like that go with it for years until it becomes folklore or a legend amongst you guys where nobody really remembers what actually has happened, but the version that you remember is somehow what you base your believes about climate science on. Correct?

      Who quality controls all the skeptic prophets who write nonsense day in day out? Since it doesn’t really matter, it often doesn’t get refuted by anyone and some poor souls begin to think those „findings“ are real. That there are contradictions everywhere. They believe what they are being told by blogs like this one. Without much quality control … I have confidence that scientists can figure out what is going on with the climate, but I have zero confidence that blogexperts and conspiracy fans that flock to this skeptic meme can ever change their opinions towards non-crazy ones …

      1. Yonason

        Climategate? Why is this even a thing?”SebH

        Why is perversion of peer review even “a thing?” Because it’s broken, and the warmunista science charlatans are among those who broke it.
        https://www.masterresource.org/climate-science/climategate-is-peer-review-in-need-of-change/

        Shame on us for caring about scientific integrity?

        NO!

        Shame on SebH for not only not caring, but trying to con others into not caring as well, …and lying to them about why they shouldn’t.

        Climategate exposed the dry rot that is rampant in climate science. It is a BIG deal. If it isn’t “a thing,” then nothing is.
        https://climatechangedispatch.com/climate-change-the-most-massive-scientific-fraud-in-human-history/

        See also here.
        https://www.masterresource.org/climategate/revisting-climategate-climatism-falters/
        “These e-mails provide an insight into practices by researchers that are poor science at best and fraudulent at worst. Bias, manipulation of data, avoidance of freedom of information requests, and efforts to subvert the peer-review process are apparent, all to further the “cause” of man-made global warming.

        Some more Climategate links…

        https://www.c3headlines.com/climategate-climate-liars/page/5/

        https://www.realclearpolitics.com/articles/2009/11/24/the_fix_is_in_99280.html

        http://scienceandpublicpolicy.org/images/stories/papers/originals/Monckton-Caught%20Green-Handed%20Climategate%20Scandal.pdf

        It’s pretty obvious from the facts that SebH hasn’t a clue what he’s writing about.

        1. SebastianH

          Not allowed to reply to Yonason? Why are you protecting this guy?

  9. scott allen

    Seb, “peer review isn’t perfect, but it’s what we’ve got.” is a very poor defense of the modern scientific method. Modern science is broken, when a researcher sends a paper to a publisher most publishers ask for suggestion on who should peer review the paper, that’s not peer review that’s “pal review”. The government bases policy/laws/spending on this science which later turns out to be false/fraudulent/wrong, public opinion is also shaped by this “science” which can and does lead to deaths (Wakefield study) which people are now refusing to get vaccines for their children. Most of these studies are no better then a “WAG”.
    An excellent article was written in the New Yorker in 2010 (hardly a climate denier magazine)about how much we believe is just not true

    https://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off

    Most people don’t believe modern science because of people like you, who stand behind the modern scientific research as a shield and a weapon, to bludgeon opponents only to be told years later “oh we were wrong and now you should believe this”.

    Even the prestigious Nature magazine show most science is not reproducible which should be the hallmark of all research.

    https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970

    Science is broken, go to the website Retraction Watch to see how broken science is…….

  10. M E

    I have zero confidence that blogexperts and conspiracy fans that flock to this skeptic meme can ever change their opinions towards non-crazy ones …”

    That is a very interesting point of view! Does SebH think that insulting people makes them change their minds? I hope he doesn’t use this attitude in daily life. It’s asking for trouble in my experience.

  11. Weekly Climate and Energy News Roundup #335 | Watts Up With That?
  12. Half Of Climate Science Results Non-Replicable, Flawed | PSI Intl

    […] Read more at No Tricks Zone […]

  13. Half Of Climate Science Results Non-Replicable, Flawed – Newsfeed – Hasslefree allsorts

    […] Read more at No Tricks Zone […]

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this. More information at our Data Privacy Policy

Close