Fast-tracked PNAS papers are cited less often — but gap is shrinking

An analysis of more than 50,000 papers submitted to Proceedings of the National Academy of Sciences (PNAS) shows that those published using its “contributed track” — in which academy members can fast-track their own papers by coordinating the peer-review process themselves — have been cited less often than regular submissions, but that gap is shrinking.

Although the overall average difference in citations between contributed and regular submissions was 9%, the yearly difference has declined from 13.6% in 2005 to 2.2% in 2014, according to the new study, posted before peer review on the preprint server bioRxiv by Phil Davis, an independent researcher and publishing consultant based in New York.

The contributed track is a long-standing editorial practice of PNAS, which has triggered concerns from some academics that say it puts the journal at risk of becoming a “dumping ground” for subpar papers. A 2014 analysis by Nature revealed that 98% of papers submitted via the journal’s contributed route were published, compared to only 18% of regular submissions.

Preferential treatment of its allegedly stellar academics is supposed to speed up publication processes at the journal compared to its other established competitors — Nature, Science and Cell — the academy’s advocates say.

PNAS hired Davis last year to investigate the performance of the different types of submissions. He investigated 55,889 research papers published between 1997 and 2014, a period in which PNAS grew by 44%. He found that papers in the social sciences had the greatest disparity, with a 12% citation gap between contributed submissions and regular submissions, also known as “direct submissions.”

Davis described his findings from “Comparing the Citation Performance of PNAS Papers by Submission Track” in his blog:

The top 10% of contributed papers outperformed the top 10% of direct submission[s], but at the bottom end, contributed papers were more likely to remain uncited after their first two years of publication. Consistent with our main findings, both of these effects have also been attenuating over the past decade and were not detectable in the last five years of publication.

Members are limited to submitting four papers per year through the contributed track. The Nature report showed that out of the 3100+ academy members who could submit contributed papers, fewer than 1400 did so between 2004 and 2013. Thirteen scientists used the contributed route most to their advantage, each publishing 30 or more papers between 2004 and 2013.

Here’s how the contributed paper review process works, according to the PNAS website:

When submitting using the Contributed process, members must secure the comments of at least two qualified reviewers, each from a different institution and not from the authors’ institutions. Reviewers should be asked to evaluate revised manuscripts to ensure that their concerns have been adequately addressed. Members’ submissions must be accompanied by the names and contact information, including emails, of knowledgeable experts who reviewed the paper, along with all of the reviews received and the authors’ response for each round of review, and a brief statement endorsing publication in PNAS. Reviews must be on the PNAS review form. Members must select reviewers who have not collaborated with the authors in the past 48 months.

PNAS also had a third publication route  — called the “communication track” — where manuscripts submitted by non-members were passed onto colleagues who are members, who would then help push them through peer review. This track was abolished in 2010, due to decreased popularity.

Psychologist David Rand and evolutionary biologist Thomas Pfeiffer, both formerly based at Harvard University, compared the three tracks in a 2009 PLOS ONE study. Their data — on PNAS papers published from mid-2004 to mid-2005 — showed that contributed papers receive an average of 10% fewer citations than direct submissions. But the most cited contributed papers were cited more often than the most cited direct submissions.

We asked Rand, now based at Yale University, to review the latest analysis. He told us:

The implication is that weakening peer review…allows lower quality papers through, but also…facilitates the publication of really important, ground-breaking work that perhaps would have difficulty getting published via more traditional routes.

In the last decade, PNAS has moved towards tightening its editorial policies around the contributed track to stamp down on misuse. Rand added:

As PNAS has put more restrictions…on the contributed track, the average performance of contributed papers has increased…

But Rand noted that there is also an argument for having a less constrained contributed track – namely, the ongoing concern that the peer-review process can limit innovative thinking. This would coincide with Davis’s finding that the top contributed papers now get roughly the same citations as the top direct submissions, he said:

One could argue that it is worth allowing some lower quality papers through in order to also let really innovative [papers] get published.

Of course, the Davis and Rand studies are limited by their reliance on citation counts as a measure of quality, which does not capture all dimensions of the paper’s impact. Another constraint is that although all contributed papers had at least one academy member as an author, it’s not clear if the originators of direct submissions were members or non-members. The paper also didn’t note how much overall citations to PNAS papers as a whole have changed over the study period.

We reached out to Professor Sir Alan Fersht, a biophysicist at the University of Cambridge, UK, and an associate editor at PNAS. Fersht often uses the contributed track for his own papers, publishing 32 papers via the route between 2004 and 2013. He told us:

All papers published in PNAS, whether they be contributed or by the regular route, have always been subjected to rigorous peer review by two or more referees and then subjected to further oversight by an Associate Editor or Board Member. The few abuses of the system have gradually been squeezed out.

We also contacted PNAS, and will update if they respond.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.