Here at Retraction Watch, we’ve covered somewhere shy of 2,000 retractions in our nearly five years of existence. With this post, we may be more than doubling that total count.
That’s because it looks like IEEE may have retracted thousands of meeting abstracts. Yes, thousands.
We don’t know the exact number, but a search for “retraction” in the abstracts of the 2011 International Conference on E-Business and E-Government (ICEE), held May 6-8 2011, brings up 1,281 results.
The same search on the site of the 5th International Conference on Bioinformatics and Biomedical Engineering (iCBBE), May 10-12, 2011, brings up 1,085 results.
We only opened the first few retraction notices, but they seem to all be the same. Headed “Notice of Retraction,” they read:
After careful and considered review of the content of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE’s Publication Principles.
We hereby retract the content of this paper. Reasonable effort should be made to remove all past references to this paper.
The presenting author of this paper has the option to appeal this decision by contacting [email protected].
Not all the abstracts from the meetings appear to have been pulled, as the main abstract page for the ICEE meeting shows more than 2,300 results.
We also saw this note, which appears to affect more than 50 additional conferences (including the 2012 ICEE and iCBBE meetings):
IEEE has a long-standing commitment to ensuring the high quality of its conferences and of the conference proceedings published in IEEE Xplore®.
Through its regular conference quality-monitoring process, IEEE became aware of inconsistencies in some conferences with regard to the quality of the peer review and technical program development. In 2010, IEEE launched an in-depth investigation through its Technical Program Integrity Committee, a volunteer-based committee responsible for ensuring the quality of IEEE publications.
The committee concluded that the procedures followed by the technical program organizers of certain conferences were insufficient to assure compliance with IEEE’s high standards for quality publications. Therefore, IEEE has decided not to publish the proceedings from these conferences in IEEE Xplore. A list of conferences impacted since 1 January 2012 can be found below.
Where copyrights were transferred to IEEE, those transfers of rights are now null and void and revert back to the authors. Authors wishing confirmation of their copyright ownership should complete the form below. Upon receipt, IEEE will send a confirmation e-mail indicating that the transfer of copyright for the author’s paper is now “null and void” as provided on the IEEE Copyright Form, and that the author is the copyright owner of the paper.
If Copyright Forms have not yet been signed and transmitted, then copyright will remain with the author. In either case, the authors are free to seek publication of their papers in other journals or venues.
From the language, we can’t tell if these are official “retractions” or not — “decided not to publish” could mean the proceedings never appeared in the first place. But the meetings in the note above range in years from 2012 to 2014, so depending on when the note was published, some of the conference proceedings may have already appeared.
We emailed and called press contacts at IEEE, and got this statement (which unfortunately still leaves us with a lot of unanswered questions):
On an annual basis, IEEE works with more than 400,000 authors worldwide to deliver quality content in the academic publishing industry. To ensure that IEEE’s global quality control process is followed IEEE maintains strict governance guidelines for evaluating IEEE conferences and publications. Unfortunately sometimes we are faced with issues. In these instances, IEEE takes steps to mitigate their effect and preserve the value of our content.
The episode raises flashbacks from another major retraction from IEEE, when the publisher and Springer pulled some 120 papers created with random paper generator SCIgen. Springer’s final report on that episode can be found here.
Update 5:59 p.m. eastern 6/25/16: In response to a follow-up question about how many retractions have occurred and when, an IEEE spokesperson told us:
IEEE is committed to providing superior quality in our published materials. For IEEE, the need to inspect our digital library and to retract, if necessary, is not a one-time occurrence but an ongoing process. As a result, it is not feasible to specify the exact number of articles involved at any single point in time. In addition, to limit the number of retractions that are necessary we have developed a process to screen materials prior to submission to IEEE Xplore®. When we discover content that does not meet our standards, we address the matter quickly.
We also asked for more information about this note, and whether it describes retractions or papers that never appeared, and got this response:
The notice you referenced is informational and for guidance to authors if their paper has been retracted or not in the IEEE fields of interest.
Hat tip: Matt Hodgkinson
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.
Note that these are not just abstracts, but full conference papers. I estimate the total at approximately 9,700 papers from 173 conferences:
http://ieeexplore.ieee.org/search/searchresult.jsp?queryText=.QT.notice%20of%20retraction.QT.&newsearch=true
As you note, the reasons for retraction are not given.
The “Technical Program Integrity Committee Actions” page you link to was published by September 2011:
https://web.archive.org/web/*/http://www.ieee.org/conferences_events/conferences/publishing/author_form.html
I believe these represent entire conferences where IEEE agreed to publish the proceedings, but then cancelled the agreement before doing so. I count 167 conference proceedings cancelled, dating back to 2010. These are separate to the retractions.
Good point about “IEEE agreed to publish”, but I think IEEE has a responsibility to maintain a high standard of quality. And they should retract publications (of course, with sufficient reason that should be made public) if the need arises. The interesting question is if the organizers of the conference were…well, not so honest in the review process…does IEEE have an obligation to honor their agreement to publish? I think “no”, but then what can authors do? Should they complain to IEEE or the organizers of the conference? Especially if tax payer money has been spent through conference registration and travel.
It would be nice if IEEE blacklisted such conference organizers. (Assuming that was the cause…we don’t really know.) But the authors are still stuck. For an entire conference, maybe some papers should never have been let through, but certainly not all.
Wow!, what is actually the criteria for the retraction?
Anyway, authors still have the option to appeal to the decision, that’s good. But, somehow it looks unprofessional.
For some background about the IEEE’s publishing principles, see page 84 onwards at:
http://www.ieee.org/documents/opsmanual.pdf
Thanks for following up on this. As noted by Thomas, these are conference papers and not abstracts. The retractions may be connected to follow-up by the committee that cancelled the IEEE running many conferences in 2011.
A search of IEEE Xplore shows 9,695 retractions ever by IEEE: http://ieeexplore.ieee.org/search/searchresult.jsp?queryText=.QT.notice%20of%20retraction.QT.&newsearch=true
The main conference publications affected by these bulk retractions are:
2011 International Conference on E-Business and E-Government (ICEE): 1,281
2011 5th International Conference on Bioinformatics and Biomedical Engineering (iCBBE): 1,084
2010 International Conference on Computer Application and System Modeling (ICCASM): 442
2011 2nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC): 442
2010 IEEE International Conference on Advanced Management Science (ICAMS): 431
2010 3rd IEEE International Conference on Computer Science and Information Technology (ICCSIT): 401
The top 10 conference locations for IEEE retractions are all in mainland China (Wuhan has 2,755) and the top 25 author institutions are all from mainland China.
These notices give no date of retraction, but the retractions for the biggest two probably happened between March and July 2014, according to the Wayback Machine captures:
http://web.archive.org/web/*/http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=5779756
http://web.archive.org/web/*/http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=5779818
The most affected conference, ICEE, seems to have gone under after being pulled by IEEE (http://web.archive.org/web/20120623002734/http://www.icee-meeting.org/2012/). The second-most affected conference, iCBBE, is still being run by the same researcher, Kuo-Chen Chou (http://www.icbbe.org/2015/), with no mention of the bulk retractions, and is being published this year by CRC Press/Taylor & Francis: http://www.tandf.net/books/details/9781138027848/
Interesting that they mention the copyright transfer agreements void.
So as a question not directly linked to this specific case:
If authors sign a copyright transfer agreement, do they get the copyright back if the paper eventually gets retracted? Has anyone at RW looked at these questions? If authors ever want to republish the data, they would have to first get copyright back, wouldn’t they?
Great question, genetics. The print version of the retracted paper will remain in the journal and, in many cases, so does the PDF when the journal is available online, though the PDF is marked as retracted. So, based on these considerations, I would assume that the original journal maintains the copyright. But, you raise an interesting question. In cases were the retractions are based on legitimate errors and the the authors republish their paper with corrections in a different publisher that normally retains authors’ copyright, what then?
Here’s a poll on this question from 2013: http://retractionwatch.com/2013/09/17/ask-retraction-watch-can-data-from-retracted-papers-be-republished/
Oh dear … I had totally forgotten about that RW post.
Slightly different circumstances, but my parents have published several textbooks. One of them went out of print, and the original publisher didn’t want to republish it. So my parents asked the publisher to give them the copyright back; that happened, and the book was republished with another publisher.
I suspect that (as you mention) the same process can be followed if a paper is retracted and the authors wanted to (re)publish the data — they could ask the journal to give them the copyright back.
In computer science the primary publishing venues are peer-reviewed conferences. These conferences are typically affiliated with a non-profit professional organization, like the ACM or IEEE, which publishes the conference proceedings, but the conference itself is run by volunteers. Having chaired an IEEE conference this year, I can affirm that the publishing organization has no involvement in paper reviewing. In fact nowadays the IEEE doesn’t even do any editing for some conferences – the PDFs are uploaded directly to the IEEE digital library by the conference organizers.
This works well for top-tier conferences (e.g. ones like IEEE Infocom or ACM SOSP, both with <20% acceptance rates) but not so well for bottom-tier ones. Much like journal publishers trust the editorial boards of their journals, the IEEE has to trust the organizers of its conferences, and in both cases it's quite messy when that trust is found to be misplaced.
Hi so What’re the recent developments?
The list of retracted conferences can be found here:
https://www.ieee.org/conferences_events/conferences/publishing/author_form.html