High-profile indexing service punishes 20 journals, issues unusual warning about five others

If scientific publishing were the World Cup, twenty scientific journals are being effectively taken out of competition today. And five others are being given a stern first-time warning.

Every year, Clarivate Analytics, a company that indexes more than 11,000 journals — and which, in turn, designates their powerful, but controversial, Impact Factors and rankings, based on citation rates — issues an annual report, noting how journals’ metrics changed, and which ones showed worrisome behavior that might be an attempt to game the system, such as citing themselves too frequently. And this year is no different: In this year’s Journal Citation Reports (JCR), Clarivate is stripping 20 journals of their Impact Factors by suppressing them from the rankings.

That decision can have far-reaching effects, both for those authors who’ve published in these journals, and for the journals themselves, since researchers will often go elsewhere in search of journals whose rank is recognized by their tenure and promotion committees. (Of course, many, including us, have argued that Impact Factor is not the best way to judge research.)

But this year’s JCR includes an entirely new element. For the first time, Clarivate has issued an expression of concern, a tool normally used by journals to flag papers that readers should treat with caution. This year’s JCR includes an expression of concern for five journals that displayed a “problematic pattern of citations:”

We have initiated a process of investigation of the journals named above, along with affiliated publications. When we have completed a thorough review, we will take any appropriate action; this may result in one or more journals being removed from the JCR, with a possible further action of removal from indexing in Web of Science.

The five flagged journals are:

The first four publications — published by American Scientific Publishers — displayed a “problematic pattern of citations,” while Bone Research, published by Springer Nature in partnership with the West China School of Stomatology, Sichuan University, and the International Chinese Musculoskeletal Research Society, received “a particular concentration of citations,” according to Clarivate.

Crunching the numbers

The journal is just five years old, and publishes on average 30 papers per year. That small number could explain some of the anomaly, as any citations to it would have an outsized effect on its Impact Factor. But it wasn’t just the Impact Factor that showed a dramatic increase, Clarivate tells Retraction Watch. It was also the journal’s Eigenfactor Score, which takes the size of the journal into account. Here’s the trend, provided by Clarivate:

Impact Factor Eigenfactor Score
2014 1.31 0.00014
2015 3.549 0.00053
2016 9.326 0.00153
2017 12.354 0.0024

When normalized for 2014 values, the Impact Factor shows a 9-fold increase, while the Eigenfactor Score shows a 17-fold increase. Still, it was the fact that so many of the citations came from particular sources that raised red flags, according to the company:

The 4 named [journals] have many articles with citations to Bone Res that seem to have little scholarly relevance, are unusual in their pattern of appearance in the citing articles, and are disproportionate in their concentration on the [journal impact factor] years of Bone Res.

For example, one article, “Fe3O4@SiO2 Nanobeads Based Mixed Agglutination Reaction Assay for Analyzing Male/Female Fertility,” includes 99 references, of 14 which are to Bone Research in a period that would count toward the latter’s Impact Factor. In addition,

the citations are presented as part of a large patch of non-specific references to supporting materials.  As scholarly reference practice, this is of questionable value, but the content of those clustered references is of further concern.

A precedent?

A spokesperson for Clarivate confirmed that this was a first for the JCR:

We didn’t want to act to suppress the journals without investigation, but we needed time to do the right kind of investigation.

The spokesperson added:

We wanted to inform users and we will follow up with the results of the review, as well as appropriate action.  This may be to possibly to remove some journals from the reload, possibly just to state the outcomes and “withdraw” the expression of concern.

Springer Nature tells Retraction Watch:

We can confirm that we are fully cooperating with Clarivate Analytics to investigate these anomalies in citations in the journal.

Like previous JCRs, this year’s also includes a list of “suppressed” titles — meaning journals whose impact factor will no longer be calculated by Clarivate, due to “the distortion of the Journal Impact Factor by an excessive concentration of citations,” either because they cited themselves significantly more often than average, or seemed to be involved in “citation stacking” — a sort of “you scratch my back, I’ll scratch yours” scheme that has landed journals in hot water in the past.

The 20 suppressed journals include titles in a range of subjects, from Aquaculture Economics & Management and the Archives of Budo to the Chinese Journal of Mechanical Engineering, LIVER CANCER, and Pediatric Dentistry. The journals’ publishers include Elsevier and Karger.

Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at team@retractionwatch.com.

6 thoughts on “High-profile indexing service punishes 20 journals, issues unusual warning about five others”

  1. This is a good start in terms of addressing the problem of citation fraud at the journal level to boost the journal impact factor and other measures. However, Clarivate needs to address this problem at the individual researcher level; my research has shown that many of the highly cited researchers have very high proportions of self- and affinity group citations, many of which are questionable in terms of direct relevance. I have also found evidence of citation cliques among some of the researcher citation networks.

  2. I found a more extreme case, this paper below (Nanoscience and Nanotechnology Letters), with a title: “Towards Highly Linear High Resolution Successive Approximation Register ADCs for the Internet of Things”
    show no contents related to bone research, yet cited 11 papers from the journal “bone research”. There is indeed very serious problem of citation manipulation.

    1. I think one can already start with the question why this paper was published in a journal on Nanoscience and Nanotechnology at all? I know this field is sometimes defined rather broad, but *this* broad?!

      Note that the many citations to Bone Research are included in the one single section that contains the word “nano”:
      “Finally, it was found that nanomaterials have been paid much attention for their preparation,10–31 modification, 32–55 and applications 56–75 in various fields, 76–95 we believe our research will be helpful for nano-sensing technology.”

      The paper has only 9 citations referring to the actual work done, and a whopping 86 that supposedly put the work in a broader perspective in this last sentence. It is also not only Bone Research that features there, but several other journals from the same publisher.

  3. Is there an overlap in terms of editors between the “Bone Research” collaboration, and the four “American Scientific Productions” outlets?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.