Retraction Watch

Tracking retractions as a window into the scientific process

How to spot a “citation cartel”

with 7 comments

Iztok Fister Jr

Do you know the difference between a group of researchers in the same field who cite each other’s related work, and a group of authors who purposefully cite each other in order to boost their own profiles? It’s not easy to do, say researchers in a new article about so-called “Citation cartels.” In Frontiers in Physics, Matjaz Perc and two Iztok Fisters (Senior and Junior) from the University of Maribor in Slovenia present an algorithm to help identify groups of researchers citing each other for overly collegial reasons. (For more on the phenomenon, see a recent column in STAT by our co-founders.) We spoke with first author Iztok Fister Jr.

Retraction Watch: What exactly are “citation cartels”? How do they differ from groups of researchers in the same field who tend to cite each other because their research is related in some way, without any nefarious intent?

Iztok Fister Jr: There is often a very thin line between groups of researchers in the same field who cite each other because their research is related and a citation cartel. An “exact” definition of a citation cartel is quite impossible, and some form of in-depth human review will always be needed to determine the existence of each particular cartel conclusively. Our algorithm works well in that it is able to identify candidates, or likely members of a citation cartel, but on its own, the output is definitively not enough to point a finger and say “this is a citation cartel”. One then needs to take time, ask experts to read a couple (or many, depending on the size of the potential cartel) of papers, possibly collect info from the authors if needed, and so on. As with any shady activity, those who indulge in it do their best to cover their tracks, and this makes the job difficult. In our experience, a citation cartel differs from the ordinary in that it usually involves one or more or all of the following: i) a small number, often just two or three, journals are involved; ii) similarly, the diversity of authors involved is small, i.e., smaller as one would expect for a healthy research community; iii) often there is a large overlap of editors in the journals that sustain a particular cartel. You do not see these things when groups of researchers in the same field cite each because their research is related. We were alerted to the work of Phil Davis on the same subject, which regrettably we were not familiar with at the time of writing our paper, and his specific examples confirm what can be defined as citation cartels (1, 2).

RW: How big of a problem do citation cartels pose to the literature? Are they present more frequently in some fields?

IF: Statistically speaking the problem is negligible. Only a minute fraction of the research community indulges into these practices. However, with the rise of bibliometric and altmetric indicators and their inclusion into habilitation and funding criteria, the pressure on individual researchers, as well as the pressure on institutions and universities as a whole, to perform well in these numbers is rising. It is therefore important to develop tools and to raise awareness of the pitfalls that come with putting bibliometric indicators over traditional peer review and expert recommendations. There are certainly differences between fields in that in some there is more pressure and a greater importance put to technical indicators, and this also leads to these fields being more exposed to the problem of citation cartels.

RW: You present a fairly sophisticated algorithm for identifying a citation cartel – can you explain your methodology?

IF: Essentially, our algorithm is very much akin to a community detection algorithm of a network. A community in a network is defined as a group of people that have a greater density of links amongst themselves as with the rest of the network. In analogy, our algorithm looks for authors or groups of authors that cite each other more than they do other authors of groups of authors that belong to the same field. There are a couple of other details, but this is the gist of it.

RW: For readers who aren’t able to apply the algorithm themselves, what are some obvious signs of a citation cartel – ie, to the “naked eye?” 

IF: It is difficult to spot something with the naked eye, as a cartel typically involves quite a large number of papers. One needs to have access to an extensive database, and it definitively helps to have some data mining tools at one’s fingertips. Of course, if you see a paper authored by X that cites an author or a group of authors 10 times, and then you look at those 10 papers and see that they are citing previous papers of author X equally ferociously, it is worth a deeper inspection. But such obvious-to-the-eye cases are rare. We again refer to work by Phil Davis for some examples.

RW: What can readers and journals do to combat the problem of citation cartels?

IF: Readers probably not a lot, at least not directly. If, as a reader, I am suspecting a cartel, I could stop reading from the introduction onward. And if many do that, this probably will reflect badly on the recognition and the impact of such research. Journals can of course do much more, especially by carefully selecting their editors, who should, preferably, be experienced researchers with ample refereeing and publishing experience before assuming their roles. Many journals and publishers also run “education” campaigns, and they keep their editors and referees informed and reminded about proper practices. Such actions surely help.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Written by Alison McCook

January 18th, 2017 at 9:30 am

Comments
  • bigcitylib January 18, 2017 at 10:02 am

    It would be interesting to know if Dr. Fister sees these cartels as being more prevalent in certain fields.

  • TL January 18, 2017 at 10:29 am

    “Statistically speaking the problem is negligible. Only a minute fraction of the research community indulges into these practices […] It is therefore important to develop tools and to raise awareness of the pitfalls that come with putting bibliometric indicators over traditional peer review and expert recommendations.”

    So the lesson is: stop caring about “citation cartels” and start using measures other than the number of citations to measure scientific quality.

    • DF January 18, 2017 at 12:59 pm

      What are these “other” measures and how can they not also be similarly manipulated?

      • Anonymous January 19, 2017 at 4:17 am

        Splot on. Reminds me of the Goodhart’s law (https://en.wikipedia.org/wiki/Goodhart's_law):
        “When a measure becomes a target, it ceases to be a good measure.”

      • TL January 19, 2017 at 5:40 am

        Having peers read and evaluate papers is always a good start.

  • El Naschie et Ji-Huan He January 18, 2017 at 5:19 pm

    I don’t know whether people remember El Naschie and Ji-Huan He. There’s a write up here:

    https://simplificationadministrative.org/2010/11/28/la-bibliometrie-deja-cassee/#english

    Unfortunately the links to Thomson Reuters and probably many others no longer work.

    Their story suggests that another way to obtain a sample highly enriched in citation scammers is simply is to look at the stars of bibliometrics! If they are in your field and you’ve never heard of them, you are probably good to go.

    • ICC January 24, 2017 at 6:05 pm

      This story was truly shocking but many people in the field saw it coming years ahead. It was unfortunate to observe how slowly Elsevier acted. Even worse, however, is that Ji-Huan He, currently best known for being a citation-cartel mastermind (while engaging in pseudoscience with El Naschie), is still on editorial boards at Elsevier (e.g., https://www.journals.elsevier.com/applied-mathematical-modelling/editorial-board)! Which, of course, is the precisely what the last of the q&a answers above discourages…

  • Post a comment

    Threaded commenting powered by interconnect/it code.