Do science findings feel more novel, robust? They are — at least, in language

Do you think the write-up of scientific results has gotten more rosy over time? If so, you’re right — the use of positive language in science abstracts has increased by 880% since 1974, according to new findings reported in the British Medical Journal.

Researchers led by Christiaan H Vinkers at the University Medical Center Utrecht in The Netherlands found that, among PubMed abstracts:

The absolute frequency of positive words increased from 2.0% (1974-80) to 17.5% (2014), a relative increase of 880% over four decades. All 25 individual positive words contributed to the increase, particularly the words “robust,” “novel,” “innovative,” and “unprecedented,” which increased in relative frequency up to 15000%.

It sounds a lot like a scientific version of the Lake Wobegon Effect, and it’s consistent with previous results using news stories. The findings prompted the authors to draw a pithy conclusion:

Apparently scientists look on the bright side of research results. But whether this perception fits reality should be questioned.

This trend is problematic, Vinkers told us:

If everything is ‘robust’ and ‘novel’, then there is no distinction. In that case, words used to describe scientific results are no longer driven by the content but rather by marketability. We think this changing attitude/culture is problematic for the credibility of science. We did not measure real scientific progress (it is even debatable whether this is possible), but it seems unlikely that the 900% increase in positive words is realistic and reflects true progress in science. Experiments fail and have failed regularly and results are not always clear (but you still have to write them down). If the language increasingly diverges from the actual results, quid est veritas?

Most likely, authors are marketing their findings as a result of the intense pressure scientists feel to publish papers, and the value journals place on positive findings, he added. All of which contributes to the ongoing issue of reproducibility in science:

…your manuscript must stand out amongst thousands of others, and one way to go about it is to lose the nuance and present a more black-and-white picture which may not be necessarily untrue but may be somewhat exaggerated without extensively addressing the limitations. This has direct consequences for science reproducibility: there is limited room for doubt and nuance, and these terms should be at the core of science. In turn, replication efforts are regarded to be less attractive and it is also harder to get these replication studies published. And there is the thing George Orwell described in 1984: If thought can corrupt language, then the language can corrupt thought. This makes the described trend even more problematic.

The trend was somewhat less notable in high-impact biomedical journals, and more notable among authors based in non-English speaking countries, the authors note in “Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis.”

But scientists aren’t just being more positive — they’re also including more negative terms, such as “disappointing,” “unacceptable,” and “weak.” Use of these and 22 other negative terms rose from 1.3% in the 1970s to 3.2% in 2014, a more than 250% jump.

During the study, the authors scanned all abstracts in PubMed published between 1974 and 2014, looking for 25 positive and 25 negative words, along with some neutral terms. The positive terms they searched for were:

Amazing, assuring, astonishing, bright, creative, encouraging, enormous, excellent, favourable, groundbreaking, hopeful, innovative, inspiring, inventive, novel, phenomenal, prominent, promising, reassuring, remarkable, robust, spectacular, supportive, unique, unprecedented

And here were the negative terms:

Detrimental, disappointing, disconcerting, discouraging, disheartening, disturbing, frustrating, futile, hopeless, impossible, inadequate, ineffective, insignificant, insufficient, irrelevant, mediocre, pessimistic, substandard, unacceptable, unpromising, unsatisfactory, unsatisfying, useless, weak, worrisome

It’s time for a change in the culture of academia, the authors conclude:

It is time for a new academic culture that rewards quality over quantity and stimulates researchers to revere nuance and objectivity.

But how to do that? Vinkers told us he believes scientists have to try to fight against the pressures that cause them to over-market their findings:

Rather than giving in to this pressure to deliver, we need to be realistic in what science can offer and the speed in which developments take place. This needs not only be communicated to the general public but also to news media which sometimes unthinkingly write on science results without proper context and without the nuances and limitations that are essential for a correct understanding and interpretation of scientific results. For scientists, some of the more direct possibilities to improve the current situation relate to the current publication culture and the way science quality is (bibliometrically) measured. Progress in science is essential but need to be based on realism. There should be room for ‘slow science’. In this context, our results confirm and support the Dutch Science in Transition initiative (see http://www.scienceintransition.nl/english).

Our co-founders agree. Of course, with a wink, the authors made sure that next time someone searches for positive words, their paper will show up. Here’s their conclusion:

Despite the steady increase of superlatives in science, this finding should not detract us from the fact we need bright, unique, innovative, creative, and excellent scientists.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.