High-profile indexing service punishes 20 journals, issues unusual warning about five others

If scientific publishing were the World Cup, twenty scientific journals are being effectively taken out of competition today. And five others are being given a stern first-time warning.

Every year, Clarivate Analytics, a company that indexes more than 11,000 journals — and which, in turn, designates their powerful, but controversial, Impact Factors and rankings, based on citation rates — issues an annual report, noting how journals’ metrics changed, and which ones showed worrisome behavior that might be an attempt to game the system, such as citing themselves too frequently. And this year is no different: In this year’s Journal Citation Reports (JCR), Clarivate is stripping 20 journals of their Impact Factors by suppressing them from the rankings.

That decision can have far-reaching effects, both for those authors who’ve published in these journals, and for the journals themselves, since researchers will often go elsewhere in search of journals whose rank is recognized by their tenure and promotion committees. (Of course, many, including us, have argued that Impact Factor is not the best way to judge research.)

But this year’s JCR includes an entirely new element. For the first time, Clarivate has issued an expression of concern, a tool normally used by journals to flag papers that readers should treat with caution. This year’s JCR includes an expression of concern for five journals that displayed a “problematic pattern of citations:”

We have initiated a process of investigation of the journals named above, along with affiliated publications. When we have completed a thorough review, we will take any appropriate action; this may result in one or more journals being removed from the JCR, with a possible further action of removal from indexing in Web of Science.

The five flagged journals are:

• Nanoscience and Nanotechnology Letters
• Journal of Biobased Materials and Bioenergy
• Journal of Biomedical Nanotechnology
• Journal of Nanoscience and Nanotechnology
• Bone Research

The first four publications — published by American Scientific Publishers — displayed a “problematic pattern of citations,” while Bone Research, published by Springer Nature in partnership with the West China School of Stomatology, Sichuan University, and the International Chinese Musculoskeletal Research Society, received “a particular concentration of citations,” according to Clarivate.

Crunching the numbers

The journal is just five years old, and publishes on average 30 papers per year. That small number could explain some of the anomaly, as any citations to it would have an outsized effect on its Impact Factor. But it wasn’t just the Impact Factor that showed a dramatic increase, Clarivate tells Retraction Watch. It was also the journal’s Eigenfactor Score, which takes the size of the journal into account. Here’s the trend, provided by Clarivate:

When normalized for 2014 values, the Impact Factor shows a 9-fold increase, while the Eigenfactor Score shows a 17-fold increase. Still, it was the fact that so many of the citations came from particular sources that raised red flags, according to the company:

The 4 named [journals] have many articles with citations to Bone Res that seem to have little scholarly relevance, are unusual in their pattern of appearance in the citing articles, and are disproportionate in their concentration on the [journal impact factor] years of Bone Res.

For example, one article, “Fe3O4@SiO2 Nanobeads Based Mixed Agglutination Reaction Assay for Analyzing Male/Female Fertility,” includes 99 references, of 14 which are to Bone Research in a period that would count toward the latter’s Impact Factor. In addition,

the citations are presented as part of a large patch of non-specific references to supporting materials.  As scholarly reference practice, this is of questionable value, but the content of those clustered references is of further concern.

A precedent?

A spokesperson for Clarivate confirmed that this was a first for the JCR:

We didn’t want to act to suppress the journals without investigation, but we needed time to do the right kind of investigation.

The spokesperson added:

We wanted to inform users and we will follow up with the results of the review, as well as appropriate action.  This may be to possibly to remove some journals from the reload, possibly just to state the outcomes and “withdraw” the expression of concern.

Springer Nature tells Retraction Watch:

We can confirm that we are fully cooperating with Clarivate Analytics to investigate these anomalies in citations in the journal.

Like previous JCRs, this year’s also includes a list of “suppressed” titles — meaning journals whose impact factor will no longer be calculated by Clarivate, due to “the distortion of the Journal Impact Factor by an excessive concentration of citations,” either because they cited themselves significantly more often than average, or seemed to be involved in “citation stacking” — a sort of “you scratch my back, I’ll scratch yours” scheme that has landed journals in hot water in the past.

The 20 suppressed journals include titles in a range of subjects, from Aquaculture Economics & Management and the Archives of Budo to the Chinese Journal of Mechanical Engineering, LIVER CANCER, and Pediatric Dentistry. The journals’ publishers include Elsevier and Karger.

Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].