Over the past year, amid announcements of thousands of retractions, journal closures and a major index delisting several titles, executives at the troubled publisher Hindawi have at various times mentioned a “new retraction process” for investigating and pulling papers “at scale.” The publisher has declined to provide details – until now.
So far in 2023, Hindawi has retracted over 8,000 articles – more than we’ve ever seen in a single year from all publishers combined. And Hindawi is not done cleaning up from paper mills’ infiltration of its special issues, according to a new report from its parent company, Wiley.
Reckoning with Hindawi’s paper mill problem has cost Wiley, which bought the open-access publisher in 2021, an estimated $35-40 million in lost revenue in the current fiscal year, Matthew Kissner, Wiley’s interim president and CEO, said on the company’s most recent earnings call. Wiley will stop using the “Hindawi” name next year, Kissner told investors.
The publisher has issued a whitepaper, “Tackling publication manipulation at scale: Hindawi’s journey and lessons for academic publishing,” which explains “what happened at Hindawi” and the process the company developed to investigate and retract thousands of articles from special issues.
According to the whitepaper, Hindawi’s research integrity staff identified “suspicious patterns” in multiple special issues around the same time as “independent researchers with an interest in research integrity, also began noticing indicators of large-scale systematic manipulation.”
In September 2022, the publisher announced an initial batch of 500 retractions. Then, the whitepaper states:
These initial investigations coupled with the findings of independent researchers pointed to the infiltration of Hindawi [special issues] SIs at an even greater scale than first anticipated, making it clear that thousands of manuscripts would need to be investigated.
The publisher halted publication of special issues for a few months beginning in October 2022, and reassessed all special issue manuscripts with “a comprehensive checklist of specific hallmarks of papermill papers.” Hindawi also “carried out a thorough review and strengthening of our existing checks on editors, authors and reviewers,” according to the whitepaper. To investigate thousands of already-published papers, it developed “a new protocol aimed at detecting manipulation patterns and retracting papers rapidly and at scale.”
Hindawi essentially created a checklist of criteria for scoring articles, and all that met a certain threshold were retracted. The whitepaper lists several “indicators of manipulation” Hindawi has been using as evidence to retract articles:
- Discrepancies in scope.
- Discrepancies in the description of the research reported.
- Discrepancies between the availability of data and the research described.
- Inappropriate citations.
- Incoherent, meaningless, and/or irrelevant content included in the article.
- Compromised or manipulated peer-review.
The publisher explained:
The decision to retract, given this evidence, is based on the rationale that the publication process has been undermined and we can no longer vouch for the integrity of the article. We have intentionally limited the specific details of what is under investigation in the retraction notice, in part because it is essential that the intelligence shared with bad actors is restricted. Moreover, to proceed expeditiously and communicate with thousands of authors, standardized wording of retraction notices was essential.
Hindawi outsourced the work of assessing individual papers to outside vendors. At least two people, trained by the publisher’s staff, evaluated each paper according to Hindawi’s criteria and filled out a questionnaire in a “bespoke software application.” In addition:
computational tools were used to provide additional supporting evidence about fabricated content, plagiarized peer-review reports, and suspect peer review turn-around times. We also built on the valuable work done by independent research integrity sleuths, by collecting, evaluating, and categorizing comments provided on PubPeer as part of Smut Clyde’s list.
Besides retracting thousands of papers, Hindawi has used the results of its investigation to ban “several hundred” guest editors of special issues from future editorial roles and publishing articles. The publisher has also instituted “much more stringent checks” on proposals for special issues and guest editors, as well as “much greater scrutiny of peer review.”
As well as continuing retractions, Hindawi plans to issue expressions of concern for entire special issues “to alert readers that they should take additional care interpreting all papers” when the publisher suspects more articles in a particular issue “are likely to be problematic” but has not yet investigated or found proof of issues.
Multiple sleuths Retraction Watch asked to comment on the whitepaper complimented Hindawi for taking action and sharing information, but also questioned whether the publisher had really learned its lesson, and pointed out further work to do.
“I think it’s a good thing they’re documenting what they’re doing and putting out information about it,” Adam Day, developer of the Papermill Alarm, said. “I hope it’s something other publishers take a lead from and feel encouraged to deal with their own problems openly as well, because this is affecting a huge number of publishers.”
However, Day noted that he had still seen papermill articles in Hindawi journals as of this spring, after the publisher resumed special issues. “I think they’ve definitely made a big impact on the problem, but I don’t know that they’ve completely solved it yet,” he said.
Jennifer Byrne, leader of the Publication and Research Integrity in Medical Research group at the University of Sydney in Australia, said that Hindawi’s practice of intentionally limiting the information in retraction notices should be balanced with the need for transparency. “It would be helpful for future retraction research scholars if at least some specific information about the reasons for retraction were disclosed,” she said.
Byrne also said verifying the identities of guest editors for special issues is not enough to ensure quality, as “many guest editors could have superficially convincing credentials, ‘supported’ by publications from paper mills.” The difficulty will grow “as publications from paper mills become increasingly sophisticated and difficult to detect,” she predicted.
Rather, the topics submitted with proposals for special issues “deserve more stringent scrutiny,” she said:
Catch-all, duplicative special issue topics invite paper mill submissions, particularly in fields where genuine research remains very difficult, expensive and/or slow.
Cyril Labbé, who with Guillaume Cabanac and Alexander Magazinov developed the Problematic Paper Screener, praised the whitepaper’s acknowledgement of sleuths who “are working daily, most often pro-bono, to fix the work that has not been done properly by publisher.”
Cabanac added to the whitepaper’s list of recommendations for other publishers: “fund sleuths and credit them.”
Dorothy Bishop, who has examined paper mill activity in Hindawi special issues in depth, said she was “very pleased to see the publisher has at last grappled with the need for retractions ‘at scale.’” Banning editors and authors associated with paper mills is “a great step in the right direction, especially if they can share information about banned individuals with other publishers,” she said.
But Bishop critiqued the framing of the document:
The report presents Hindawi as a victim of an “academic culture of ‘publish or perish’, which has incentivized unethical behaviour”. What it omits is the influence of the commercial publisher culture of greed, which aims for massive growth in the number of published papers, with associated growth in profits.
By her calculations, Hindawi would have brought in millions of dollars from article processing charges for now-retracted papers: “The authors of these articles don’t get their money back.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, subscribe to our free daily digest or paid weekly update, follow us on Twitter, like us on Facebook, or add us to your RSS reader. If you find a retraction that’s not in The Retraction Watch Database, you can let us know here. For comments or feedback, email us at [email protected].
What will happen to the 8000 APCs? will it go to the charities? any plan?
Isn’t “Hindawi” owned by Wiley, isn’t Wiley retracting 8,000 articles?
I think that Wiley should take ownership of the 8,000 retracted, or to be retracted articles.
I get it, “Hindawi” is Wiley’s paper mill division and we are supposed to believe that the rest of Wiley stable of journals is reputable.
Everything is a shame. Money for fees everywhere. Ethically Wiley-Hindawi should come back millions of dollars at least to Institutions if wanna punish fraudulent authors. Science today is a shame.
I have carefully read the white papers published by Wiley and Hindawi, and unfortunately, they did not reflect the responsibilities they should bear. On this matter, Wiley and Hindawi should compromise by preventing peer review in advance. Moreover, what I am curious about is that the article has been retracted, why not return and exchange the APC? This seems to be a Ponzi scheme. I hope Wiley and Hindawi can return the APC to the author who retracted it, which seems more reasonable.
Hindawi made a great business for the past few years, while being careless with handling the peer-reviews. The publisher is fully in charge and should take responsibility. Carelessness of the journals and their editorials not acceptable. The publisher and the editors did not do their job correctly. The editors also must be taken accountable. Can you publish the names of the editorials public?
This would be a great step. Listing authors of retracted hindawi papers is another good step. With regular retractions I am highly sympathetic, but paper mill papers are intentionally fraudulent. The authors are not guilt free.
“what I am curious about is that the article has been retracted, why not return and exchange the APC? ” The problem is that the APCs have already been used to pay the employees for the processing. To return the APCs, they would first have to deduct it from their employees’ salary 😦
They can’t (and shouldn’t) refund APCs for fraudulent submissions, because that will make the problem even worse: Submit, pay, and, hey, if you get bust, no big deal, you get your money back, and you can try again somewhere else.
Hindawi has been completely shut down, and they can retracted all papers. What I am more concerned about is how to handle millions of APCs, which is something the community should be concerned about. I think Wiley should use all of these APCs for donations, which seems to be the most reasonable way. This task should be for future detectives. Only in this way can Wiley act like the manifesto in the white paper, otherwise they are lying.
The research community must ensures non of the editorials involved should be given the role of the editorial ever again.
Can RW make interviews with the editors involved for an insight into why they did not do their job with dignity and responsibility that this important job deserves.
The publisher must ensure that none of the involved editors are reinstated in editorial roles. It’s crucial to understand why these editors didn’t uphold the dignity and responsibility expected of this vital position. This understanding is key to safeguarding the integrity of future editorial endeavors.
Additionally, I think that a list of those editors should be submitted or available for other publishers to be aware of the unethical behavior of those editors. In my opinion, the open science movement that all publishers took action a couple of years ago just opened up a huge abyss between developed and developing or underdeveloped nations, and in the end, science seems to be more unethical than ever.
Can someone reveal the editors’s identities and their institutes? who are these careless individuates? Those who really made the issue and now disappeared and soon they will infect other journals.
I agree: we need to be told the names and institutional accreditations of the persons – and, possibly, institutions – who have corrupted/perverted academic publishing. Without that, what’s to prevent them simply moving on and doing the same all over again?
A number of the Chinese-owned Elsevier journals have also been publishing garbage: https://www.sciencedirect.com/science/article/pii/S241464472300091X
I have no idea how this slipped through the editor and supposedly, peer reviewers.
Hindawi has not done it properly. For example, look at this obviously questionable special issue, which is not retracted: https://www.hindawi.com/journals/ecam/si/567910/
All its 4 articles are absolutely out of the scope of the special issue, and worse: even out of the much broader scope of the journal itself!
Special Issue: “Traditional and Herbal Medicine in Oral Tissue Engineering”.
Journal: “Evidence-Based Complementary and Alternative Medicine”.
The published articles have nothing to do with ’tissue engineering’, ‘traditional or herbal medicine’ (or even herbal dentistry), or ‘complimentary medicine’ (or even complimentary dentistry):
https://www.hindawi.com/journals/ecam/2021/4743411/
https://www.hindawi.com/journals/ecam/2021/3349433/
https://www.hindawi.com/journals/ecam/2021/1248531/
https://www.hindawi.com/journals/ecam/2021/3304543/
ps. Perhaps the last one above is very weakly about complimentary dentistry (the journal’s scope). Still not at all relevant to tissue engineering (the special issue’s scope). The rest are 100% outside the scope of the journal itself (and of course, the special issue).
pps. All these 4 scope-irrelevant articles are exclusively from Iran. Its lead guest editor and 4 out of its 5 editors are from Iran too.
pps 2. The scientific quality of all these 4 articles are very very, extremely low.
If the articles were of poor quality, it is the duty of the editor not to send them to the referee! It is the referee’s duty to reject. It’s like you forgot the main task of the journal!
After recent events, Hindawi has become extremely slow. Very very slow. Their initial screening has become very slow. Their review has become very slow. Overall, Hindawi has become extremely inefficient after those 10,000 retractions.
We had submitted this paper to a Hindawi journal (Veterinary Medicine International); it took them about 8 months to process it. They rejected it saying “After a thorough review, I regret to inform you that your manuscript can not be further processed in VMI.”
Then they listed ZERO lines of review or ZERO reasons as their THOROUGH review! Neither the reviewer nor the editor listed even one WORD, let alone one reason for rejecting the paper, yet they called it a THOROUGH review!
I would appeal but I know that they might take a couple of years just to process our appeal. So I give up!
We appealed, asking the editor to either tell us the reasons for rejection or repeat the peer review with some reviewers who do count some reasons for rejection or acceptance. Guess what? The editor didn’t even care to read our appeal. He rejected our appeal saying that the authors did not change their article to remedy the [non-existent] rejection reasons!! This means that not only the editor had not read our paper or the non-existent review report in the first place, but also he did not read our appeal too!
The merger is now officially complete. Hindawi is officially down. None of its journals’ websites work any longer; all of them redirect to Wiley. The only Hindawi website that still remains is this blank page: https://www.hindawi.com/