More than 70% of the citations in one journal were to other papers in that journal. Another published a single paper that cited nearly 200 other articles in the journal.
Now, Clarivate, the company behind the Impact Factor, is taking steps to fight such behavior, suppressing 33 journals from their indexing service and subjecting 15 more to expressions of concern — all for apparent self-citation that boosted the journals’ rankings.
The list includes some of publishing’s biggest players: Nine journals published by Elsevier, seven by Springer Nature, six by Taylor & Francis, and five by Wiley.
Given many universities’ reliance on journal rankings to judge researchers’ work as part of tenure and promotion decisions, Clarivate’s suppression of a journal — meaning denying it an Impact Factor — can have far-reaching effects. Impact Factors are based on average citations to articles in a journal over a particular period of time. Many, including us, have argued that Impact Factor is not the best way to judge research — for reasons including relative ease of gaming such metrics.
Clarivate’s Web of Science indexes 21,000 journals, but only 12,000 are included in the annual Journal Citation Report (JCR) and receive Impact Factors. There are currently two reasons for suppression from the JCR: journal self-citation and “citation-stacking,” behavior which is sometimes referred to as taking part in “citation cartels” or “citation rings.” None of the suppressions or expressions of concern this year were for citation stacking. When the company suppresses a journal from the JCR, it continues to count its citations, but does not assign it an Impact Factor.
The company has changed the way it considers self-citation and revised its methodology, Web of Science editor in chief Nandita Quaderi said, in response to patterns it’s observed in the past few years.
This year, Clarivate also subjected 15 journals to editorial expressions of concern. Such journals had small numbers of articles with very high levels of citation to articles in the same journal that can have a significant effect on Impact Factor and rankings. Looking only at overall journal levels of self-citation would miss such patterns, Quaderi said, because it would not take directly into account how these citations are “concentrated” in particular articles or time periods.
Nothing changes for JCR journals subject to such flags, but they are monitored closely for re-evaluation and may lose their Impact Factors in succeeding years, or be removed from indexing altogether.
This move marks only the second time that Clarivate has issued expressions of concern. In 2018, they did so for five journals in the bone field, four of which went on to be suppressed the following year. All five are now reinstated in the JCR, Quaderi told Retraction Watch.
At least 25% of the references in all of the suppressed journals were self-citations, with 71% of the citations in one journal, the Journal of Environmental and Engineering Geophysics, to other papers in that journal. Leading the pack of those journals subjected to an expression of concern was the Journal of Cleaner Production — with more than 11,000 self-citations, out of some 47,000 references, for just under 25%. One article in the Journal of Allergy and Clinical Immunology-In Practice cited other papers in the same journal 191 times.
Five of the suppressed or flagged journals are in cardiology, including JACC: Cardiovascular Imaging, which has an Impact Factor above 10, ranking it in the top 10 journals in the field. Three other cardiology journals given an expression of concern by Clarivate have Impact Factors above 5.
Three of the other journals subjected to an expression of concern were in allergy and immunology.
Publishers of the journals affected by today’s announcement said they were reviewing the new listings. An Elsevier spokesperson said:
We support Clarivate in their efforts to maintain the integrity of the scientific record. Unusual self-citation patterns can arise for various reasons, including a journal covering a very niche discipline or publishing articles with unusually high concentrations of self-citations.
We are working closely with the Editors (and our society partners in the case of society owned journals) to analyse the specific self-citation pattern of each journal and, if necessary, to improve practices for the checking of references for quality and integrity.
Alison Mitchell, Springer Nature’s Chief Journals Officer, told Retraction Watch:
Obviously we take all concerns raised about our journals seriously and impact factors are an important (but not the only) metric by which our journals are judged. At this point we’re still looking into the concerns raised, so we can’t comment further.
A Wiley spokesperson told Retraction Watch:
We have received the JCR confidentially and are under strict embargo until it is released by Clarivate, so we have not yet been able to share or review the report with our partners. We can follow up with more insight next week, after further analyzing the results and having a chance to communicate with our partners.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
It would be better to suppress this medieval concept of impact factor once for all.
Yes, exactly.
Suppressing the IF of Zootaxa is clearly showing up the absurdity of IF. I hope that this Clarivate’s foul will open eyes of scientists and they will refuse this nonsensical concept.
Interesting list. I used to read The Journal of the Experimental Analysis of Behavior (JEAB – one of the expressions of concern journals) quite often, and I doubt there is anything fishy going on with the high rate of JEAB citations in JEAB. There really aren’t any other journals for a lot of the work that is published in JEAB, so it isn’t surprising that JEAB papers tend to cite a lot of earlier JEAB papers.
The same is probably true of the International Journal of Medicinal Mushrooms, which is on the list of suppressed journals.
The ban on International Journal od Systematic and Evolutionary Microbiology (IJSEM) seems to me to be a thoughtless application of a general algorithm. As the ultimate authority for microbial taxonomy, it is not weird to find that it cites itself often, just like (for example) a Supreme Court opinion is likely to cite previous jurisprudence and past landmark cases from itself.
As the publisher for IJSEM we have made the same representation to Clarivate. IJSEM is the journal of record for bacterial taxonomy, and as the only venue for such work it is unsurprising that the journal has a higher than usual rate of self-citation. Furthermore, we are unable to verify Clarivate’s claim to a self-citation rate of over 55%: data from Scopus indicates a self-citation rate of 33% in 2019, which continues a trend of decreasing self-citations following a peak of 44% in 2016. This data is supported by Leiden University’s CWTS, by Dimensions, and by Crossref.
We have asked Clarivate to share their data, and to talk with us about the situation, but to date they have refused to do so.
Impact factor is a poor proxy for quality.
Journal self-citation rate is a poor quality for lack-of- quality.
Well done Clarivate! You scored a double!
Outstanding efforts by you folks you will bring ethics in Publishing.
Journals should have papers’ rating system like Amazon products. Researchers from worldwide should be asked to rank the papers and be allowed to make comments on that paper which should be visible to everyone.
Regular citations is most flawed technique. Many editors and associate editors of mostly Elsevier journals made publication a business and they publish each others papers
This is a good step towards bringing justice and integrity back to the research world.
Dr. Ismail Badran
According to Clarivate, Zootaxa is the most indispensable tool for taxonomic and biodiversity researchers.
http://www.organismnames.com/metrics.htm?page=tsj
As a biodiversity researcher, why should I care about a company that does not care about biodiversity?
IJSEM not getting listed in the IF is totally an unfair case, it is the sole journal for bacterial taxonomy studies and naturally self citation becomes an essential part of the paper drafted to the journal, its inevitable, without understanding this basis and without discussing with appropriate authorities, Clarivate removing this journal from the IF list is totally ridiculous approach
Paraphrasing Goodhart’s law: Once a metric is used as an incentive, it stops being a useful metric.
Clarivate is now suddenly shocked, shocked to find that self-citation is going on in here! Why are they suddenly interested in this? This self-citation racket is old news.
I once had a paper come back from peer review with almost no negative comments, but with a note from the editor asking me to cite more papers from their journal. I dutifully complied, of course. Why hold up publication standing up for principle when I could spend the time jumping through the hoops of another journal’s formatting and CYA COI exercises.
What CYA COI exercises are? (not found in urban dictionaries)
CYA = “cover your ass”, i.e., reduce your vulnerability (to attack or criticism); COI = “conflict of interest”.
One of these suppressions, Zootaxa, is based on a misinterpretation, and damaging to a field (not just a journal). Biological taxonomy is foundational to all of biodiversity study, but its citation patterns are peculiar because of the nature of the field. We acquire very few citations per year, but for many decades, and the citations are mostly from other taxonomists. There are few experts in each group of organisms, and few journals that publish this research. Because the papers are often long and the experts unable to pay heavy page charges, they flocked to Zootaxa’s quick and inexpensive model. Zootaxa is not predatory or unscrupulous; it has simply been successful at enabling a field. Its very success at attracting a large fraction of the good publications in a field is now a reason to diminish it???? To strip it of impact factor will mean that experts in some countries and institutions who require impact factor will no longer be able to publish there — which may mean they will no longer be able to publish anywhere feasibly. This will especially hit taxonomists in developing countries who are unable to afford page charges.
I would like to echo Dr. Maddison’s sentiment here. Zootaxa is one of the few avenues for taxonomic research and is a critical hub for basic foundational biodiversity science, such as species descriptions, see http://www.organismnames.com/metrics.htm?page=tsj based on Clarivate’s own data.
I also echo Wayne Maddison’s comments. According to Clarivate itself, Zootaxa is responsible for over 25% of new taxon descriptions and nomenclatural acts in Zoology. It is therefore hardly surprising that there should be a similar percentage of self-citations. It has nothing to do with the journal’s quality.
This decision by Clarivate will, at a stroke, have massively impacted the career prospects and employability of swaths of early career researchers working in countries where funders and employers use the IF as a criterion for funding and promotion.
The bigger question is why we are allowing a private for-profit company like Clarivate to set itself up as self-appointed and unaccountable judge, jury and executioner of journals and careers alike.