Category: studies about peer review

“Evidence-based medicine has been hijacked:” A confession from John Ioannidis

ioannidis
John Ioannidis

John Ioannidis is perhaps best known for a 2005 paper “Why Most Published Research Findings Are False.” One of the most highly cited researchers in the world, Ioannidis, a professor at Stanford, has built a career in the field of meta-research. Earlier this month, he published a heartfelt and provocative essay in the the Journal of Clinical Epidemiology titled “Evidence-Based Medicine Has Been Hijacked: A Report to David Sackett.” In it, he carries on a conversation begun in 2004 with Sackett, who died last May and was widely considered the father of evidence-based medicine. We asked Ioannidis to expand on his comments in the essay, including why he believes he is a “failure.”

Retraction Watch: You write that as evidence-based medicine “became more influential, it was also hijacked to serve agendas different from what it originally aimed for.” Can you elaborate? Continue reading “Evidence-based medicine has been hijacked:” A confession from John Ioannidis

Papers with simpler abstracts are cited more, study suggests

J informetricsResearch papers containing abstracts that are shorter and consist of more commonly used words accumulate citations more successfully, according to a recent study published in the Journal of Informetrics.

After analyzing more than 200,000 academic papers published between 1999 and 2008, the authors found that abstracts were slightly less likely to be cited than those that were half as long. Keeping it simple also mattered— abstracts that were heavy on familiar words such as “higher,” “increased” and “time” earned a bit more citations than others. Even adding a five-letter word to an abstract reduced citation counts by 0.02%.

According to Mike Thelwall, an information scientist at the University of Wolverhampton, UK, who was not a co-author on the paper: Continue reading Papers with simpler abstracts are cited more, study suggests

Researchers’ productivity hasn’t increased in a century, study suggests

Screen Shot 2016-01-19 at 10.50.25 AMAre individual scientists now more productive early in their careers than 100 years ago? No, according to a large analysis of publication records released by PLOS ONE today.

Despite concerns of rising “salami slicing” in research papers in line with the “publish or perish” philosophy of academic publishing, the study found that individual early career researchers’ productivity has not increased in the last century. The authors analyzed more than 760,000 papers of all disciplines published by 41,427 authors between 1900 and 2013, cataloged by Thomson Reuters Web of Science.

The authors summarize their conclusions in “Researchers’ individual publication rate has not increased in a century:”

Continue reading Researchers’ productivity hasn’t increased in a century, study suggests

Fast-tracked PNAS papers are cited less often — but gap is shrinking

PNASAn analysis of more than 50,000 papers submitted to Proceedings of the National Academy of Sciences (PNAS) shows that those published using its “contributed track” — in which academy members can fast-track their own papers by coordinating the peer-review process themselves — have been cited less often than regular submissions, but that gap is shrinking.

Although the overall average difference in citations between contributed and regular submissions was 9%, the yearly difference has declined from 13.6% in 2005 to 2.2% in 2014, according to the new study, posted before peer review on the preprint server bioRxiv by Phil Davis, an independent researcher and publishing consultant based in New York.

The contributed track is a long-standing editorial practice of PNAS, which has triggered concerns from some academics that say Continue reading Fast-tracked PNAS papers are cited less often — but gap is shrinking

Why retraction shouldn’t always be the end of the story

rsc-logoWhen researchers raised concerns about a 2009 Science paper regarding a new way to screen for enzymatic activity, the lead author’s institution launched an investigation. The paper was ultimately retracted in 2010, citing “errors and omissions.”

It would seem from this example that the publishing process worked, and science’s ability to self-correct cleaned up the record. But not so to researchers Ferric Fang and Arturo Casadevall.

Fang, of the University of Washington, Seattle, and Casadevall, of Johns Hopkins — who have made names for themselves by studying retractions — note today in an article for Chemistry World that

Continue reading Why retraction shouldn’t always be the end of the story

Do science findings feel more novel, robust? They are — at least, in language

BMJ-Avatar-160x160

Do you think the write-up of scientific results has gotten more rosy over time? If so, you’re right — the use of positive language in science abstracts has increased by 880% since 1974, according to new findings reported in the British Medical Journal.

Researchers led by Christiaan H Vinkers at the University Medical Center Utrecht in The Netherlands found that, among PubMed abstracts: Continue reading Do science findings feel more novel, robust? They are — at least, in language

How does gender influence publishing? A window into one journal

fec_29_10 100Male and female reviewers may rate papers the same way, regardless of whether the authors are male or female — but women are more likely to get the chance to review papers (and get their own papers reviewed) if other women are involved, according to studies of the review process at Functional Ecology.

In their comprehensive study of manuscripts put through the peer review process at the journal from January 2004 through June 2014, the authors found that the average review scores of manuscripts was roughly the same regardless of whether the reviewer — or editor — was male or female.

The authors also scanned papers submitted to the journal between 2010 and 2014 to look at the impact of gender among authors, and also found papers with female authors receive equivalent scores to papers by men: Continue reading How does gender influence publishing? A window into one journal

Predatory journals published 400,000 papers in 2014: Report

BMC medicineThe number of so-called “predatory” open-access journals that allegedly sidestep publishing standards in order to make money off of article processing charges has dramatically expanded in recent years, and three-quarters of authors are based in either Asia or Africa, according to a new analysis from BMC Medicine.*

The number of articles published by predatory journals spiked from 53,000 in 2010 to around 420,000 in 2014, appearing in 8,000 active journals. By comparison, some 1.4-2 million papers are indexed in PubMed and similar vetted databases every year.

These types of papers have become a major problem, according to Jeffrey Beall, a librarian at the University of Colorado Denver who studies the phenomenon: Continue reading Predatory journals published 400,000 papers in 2014: Report

Should peer review be open, and rely less on author-picked reviewers? Study says…

BMJ openAfter reviewing hundreds of peer review reports from three journals, authors representing publishers BioMed Central and Springer suggest there may be some benefits to using “open” peer review — where both authors and reviewers reveal their identity — and not relying on reviewers hand-picked by the authors themselves.

But the conclusions are nuanced — they found that reviewers recommended by authors do just as good a job as other reviewers, but are more likely to tell the journal to publish the paper. In a journal that always uses open reviews — BMC Infectious Diseases — reviews are “of higher quality” than at a journal where authors are blinded to reviewers, but when one journal made a switch from a blinded to an open system, the quality didn’t improve.

Here’s what the authors conclude in the abstract of the paper, published today in BMJ Open: Continue reading Should peer review be open, and rely less on author-picked reviewers? Study says…

Can you spot the signs of retraction? Just count the errors, says a new study

downloadClinical studies that eventually get retracted are originally published with significantly more errors than non-retracted trials from the same journal, according to a new study in BMJ.

The authors actually called the errors “discrepancies” — for example, mathematical mistakes such as incorrect percentages of patients in a subgroup, contradictory results, or statistical errors.

The study doesn’t predict which papers will eventually be retracted, since such discrepancies occur frequently (including one in the paper itself), but the authors suggest a preponderance could serve as an “early and accessible signal of unreliability.”

According to the authors, all based at Imperial College London, you see a lot more of these in papers that are eventually retracted: Continue reading Can you spot the signs of retraction? Just count the errors, says a new study