Have you seen our “unhelpful retraction notices” category, a motley collection of vague, misleading, and even information-free entries? We’d like to make it obsolete, and we need our readers’ help.
Here’s what we mean: Next month, Ivan will be traveling to Rio to take part in the World Conference on Research Integrity. One of his presentations is a set of proposed guidelines for retraction notices and their dissemination that we hope will inform publishing practices and severely limit the number of entries in our “unhelpful retraction notices” category. In September, for example, we announced that our guidelines would be linked from PRE-val, which “verifies for the end user that content has gone through the peer review process and provides information that is vital to assessing the quality of that process.”
Here’s a draft of our proposed guidelines, which include many of the items recommended by the Committee on Publication Ethics and the International Committee of Medical Journal Editors:
Retraction notices meeting bare minimum requirements will:
- include the reason for retraction, in clear, unambiguous language that differentiates misconduct from honest error
- indicate which aspects of the paper are affected (i.e. which specific data or conclusions are invalid)
- indicate who initiated the retraction and which authors agreed to the retraction
- be linked prominently from all versions of the abstract
- be freely available (not paywalled)
- be communicated swiftly to indexes (eg PubMed, Thomson Scientific’s Web of Knowledge)
- be marked clearly as a retraction, rather than erratum or corrigendum
- indicate when the retraction notice was published (to differentiate this date clearly from when the original paper was published).
Optimal notices will:
- indicate when the journal was first alerted to potential problems
- indicate whether there was an institutional investigation, and if so, the result
- indicate whether other papers by the same group will be affected
- only include statements about more recent replications if these have been validated by a third party
- avoid euphemisms (eg for plagiarism)
- indicate whether the authors will be sanctioned by the journal
- indicate whether any lawsuits have been filed regarding the case
We look forward to your suggestions, which will inform our presentation and the final version of these guidelines, which we hope to publish. Feel free to comment below, or email us.
Instead of calling everything a “retraction” maybe journals should indicate with the name what sort of retraction it is. So we might use “Technical Retraction” for cases of genuine error, and “Fraudulent Retraction” for deliberate cheating.
That way the reason for the retraction would be clear without reading the small print. It would also remove, or reduce, the stigma for those who retract papers for reasons other than fraud.
I think a few things:
1) Simon, let’s not complicate an already complicated publishing landscape with categories of retractions. If everything that is listed above is listed in a retraction notice, then there is no need for a categorization. It will simply be a complete retraction notice. Having all that information would make it transparent enough.
2) I would not make the “optional” items optional. Make the whole list of 16 points obligatory. By making it optional, you are giving the authors, editors and publishers the opportunity of evading “the truth”.
3) Penalties for those who do not comply. If a member journal or publisher is a COPE member, and if these are COPE guidelines, for example, and these guidelines are not followed, then there must be a penalty, for example, they lose membership for 1-2 years, something like Thomson Reuters does to those journals that cheat the system (e.g., excessive self-citation) and then lose their IF scores. The problem is not creating lists of rules, everyone can do that easily because there are sufficient resources around. The problem is applying them and assuring that the journals play ball using the same rules that the authors have to play ball with.
c) How will RW deal with silent/stealth retractions for their data-base? How will they track them, and quantify this pool of “silent” disappearances from the literature? These might surely be the worst offenders of the rules of retraction since they silently remove retracted papers, without notices, from the public’s knowledge.
d) Assuming that you get this list of 16 points included as compulsory items, and you get the main STM publishers to hook onto this, either through COPE, or through the ICMJE, then will the rules be applied retroactively? In other words, will new rules be applied to old retraction notices? There are so many opaque, incomplete and unhelpful retraction notices which are not the authors’ faults. It is the publishers that have issued such statements to the public, so the publishers must be held responsible now, and any time in the future, for what they have published and released, in exactly the same frame of justice that authors must be held accountable for what they have published. The system cannot work when you apply two separate strata of rules.
e) I wish the RW team well in Rio and hope that the list of 16 points gets adopted quickly as a future and retroactive post-pub set of rules for retraction notices. It’s time to hold all parties accountable, not only the authors and scientists.
Many of these recommendations may be equally applicable to editorial expressions of concern.
Jaime, you have read “optional” where Ivan wrote “optimal”. That is, those items contribute the greatest weight to the total goodness of the notice; they are precisely what editors should not opt out of!
Perhaps it will be necessary to invent “Post-Retraction Peer Review”, in which peers “review” “opaque, incomplete and unhelpful retraction notices”. I’m not sure how that would work, in detail…
Lee, I wish to clarify. In my view, the terms “optional” and “optimal” are the same, because the “optimal” points were listed separately to the “bare minimum” points. If you simply remove that dividing line, and make them all “bare minimum”, then the issue of terminology evaporates.
As to your second comment, please see my latest opinion paper* on traditional PPPR in which I clearly indicate that PPPR also encompasses a scrutiny of the publishers, their operating models and any other aspect that pertains directly to the published literature, including retraction notices. The issue is not to create new terms for every new concept, but rather to educate all parties that close scrutiny, critical evaluation and – most importantly, change – are required. I am doubtful that the mainstream publishers would embrace these 16 points because, quite simply, I would see this as their final downfall (simply because too much would be revealed about the errors, weaknesses and problems of their systems and peer review structure) (IMHO).
* Teixeira da Silva, J.A. (2015) Debunking post-publication peer review. International Journal of Education and Information Technology (Public Science Framework) 1(2): 34-37. http://files.aiscience.org/journal/article/html/70390007.html
You need a list of what the retraction notices MUST NOT contain:
– claims about more recent replications by the authors themselves
– claims like “our results remain valid” and similar
So in your view, the example below – which retraction watch called, “a nice example of doing the right thing” – should be forbidden?
http://retractionwatch.com/2015/03/03/catalyst-for-change-grad-student-catches-error-in-chem-experiments-prompts-retraction/
Your suggestion inhibits authors from doing the right thing when their own efforts result in a failure to reproduce or a reinterpretation that undermines the original conclusion. Bad idea.
These look like good guidelines, which surely would have helped the WIley / Ed Wegman / Yasmin Said case, a few years ago, which has been (briefly) reignited in last month or so.
See Ed Wegman, Yasmin Said, Milt Johns Sue John Mashey For $2 Million.
About the same time as Climate science critic Wegman reprimanded by one university committee while another finds no misconduct here:
a) They wrote 2 obviously-plagiarized articles for the journal they edited/
b) Such were reported to Wiley, in the graphic side-by-side highlighted fashion.
‘c) Wiley stonewalled, then let them quietly replace the articles. No retractions.
d) After complaints, the just disappeared silently from the masthead in June 2012.
e) Then, in March 2014, they initiated lawsuit, but didn’t tell me for a year, and escalated with a second, so they wanted $2MJ for losing their jobs.
Fortunately, once the case was actually known, it was so defective it didn’t get far.
John, to simplify the situation, in your opinion, how much fault did Wiley have, and what could have been done differently?
Well, there were at least 3 issues reported in March-May 2011:
a) Yasmin Said was listed on masthead as Professor, Oklahoma State University, whereas she had never worked there, and was actually Research Assistant Professor, George Mason U. They finally, mostly-fixed that by September.
b) Wegman and Said(2011) was documented in March by Deep Climate.
c) After early notes by me, Said and Wegman(2009) was documented by Ted Kirkpatrick in early May. Just flip through his 10-pager, as it’s the simplest. DC did a more extensive one in October and found more.
Our various emails then urged retractions and scrutiny of the peer review process.
===
What could they have done: well, in May 2011, they could have fixed Said’s false rank and affiliation, and retracted the papers with the sort of notice that RW doesn’t like, i.e., “We apologize for carelessness in sources, so retract these papers”.
The big PDF attached to my blogpost has the texts of what we telling them, and the answers.
Let’s be brutally honest, guidelines don’t count for s***!
COPE is all about “guidelines” and one only has to look at the blatant flouting of those guidelines that occurs every day by their “members” to see that a fallacy the whole thing is.
If you’re going to do this properly, how about a set of “standards”, then get then ratified by NCBI, then NIH – if you don’t publish your work in a journal that agrees to play by the rules, you don’t get funded.
Nothing makes scientists actually do stuff faster than the thought of losing funding. Look how quickly everyone started complying with the open access “guidelines” once NIH threatened to pull the $$plug.
Nothing will make journals sit up and do things differently, faster than a threat from NIH that says “do not publish your work here if you want to be funded”.
It’s an ecosystem of Journals, Investigators, and Funders, and the solution is RULES. Guidelines simply do not work.
A quick look at SJR suggests that the USA produces some 23% of world research publications. A further quick web search suggests that the NIH supports about half of all USA research. Your suggestion with NIH as a watchdog/enforcer works for only some 10% of publications. Any viable strategy needs to be less America centric. I am not wedded to any of the numbers, but they are probably not far off reality.
Paul, your remark is spot on. I agree, all documents flaunted by the ICMJE and COPE (and any other “ethics” body) needs to be referred to as a rule. No more soft talk. I have two papers right now in review: one that challenges the COPE “guidelines”; the other that looks at how several COPE members are flaunting these “guidelines”, literally making a joke of them. When COPE members are not able to respect COPE “guidelines”, then this brings double disrepute; to the COPE members, and to COPE, who receive money (and not a small amount either) for membership. The bottom line: ethics cannot be bought. It is earned.
A retraction notice including these 16 items would be really fine and in most cases no longer than one page. The single issue I would like to comment is about the third bullet:
“indicate […] which authors agreed to the retraction”.
The issue is that sometimes, some authors can’t be reached, despite the best efforts of the editors. Some authors may have retired, or passed away, or just went underground. I thus suggest something like:
“indicate […] which authors, among those who were reachable, agreed to the retraction”.
Previous comment should read: “15 items”.
I think a good follow-up post to this would be to describe what sanctions would be appropriate for a particular type of retraction. If the sanctions for a particular type of misconduct were predictable then authors might be more willing to correct a paper or retract a minor error before it snowballs into a major error that could do serious damage to their reputation.
Retraction Watch (or another site) could consider a game of “Be the Editor” inspired by some of the case studies on this site. A fictional situation is described where an article has been retracted, or the Editor has to make a decision to retract or correct. The reasons for the retraction are known (or unknown, or possibly derived through the grapevine) to the “Editor”. The field of study, type of journal and it’s prestige level, how well established the authors are, their type of institution are and the country of origin of the paper could be given as variables, among others. The readers of Retraction Watch could then vote on a range of possible sanctions that the “Editor” could take against the authors in order to gauge depth of feeling towards types of situations and how the community would respond if placed in the Editor’s position (including anonymous, covert responses a.k.a “limited public sanction by journal but brutal peer review next time I see their Nature paper or even grant application” responses) across a wide range of possibilities. Would such a study be potentially informative?
This is a fine start, but the first point need more details and emphasis than just “include the reason for retraction, in clear, unambiguous language that differentiates misconduct from honest error,” I have said many times on RW and at ORI that retractions and corrections for research misconduct should NAME the person who was found in an investigation or who admitted (or who directed staff) to have falsified, fabricated, or plagiarized the results. It is a matter of fairness to try to clear the other authors who were not aware or were mislead by the perpetrator.
The second (related) optimal point also needs more explanation than just, “indicate whether there was an institutional investigation, and if so, the result.” If there was an institutional misconduct investigation (or ORI oversight) that found misconduct, that should state the NAME of the person who committed the misconduct. However, if the retraction is for an honest error, then in fairness a confidential investigation should not be mentioned — unless the issue was made public and the authors want to make public notice of the no-misconduct finding in the investigation.
As to the last optimal point, “indicate whether any lawsuits have been filed regarding the case,” this would be inappropriate in the journal’s view and the authors’ view as being irrelevant to the purpose of the retraction and probably misleading or harmful (anyone can file a trivial suit, as we have often seen on RW). If the journal or institution or author was being sued, it would rarely be advantageous for them to so state – and irrelevant to the purpose for the retraction notice.
Of course, Paul is right in noting that guidelines are not enforceable on journals, and some journals adopt the COPE guidelines but the editors and publishers ignore them in specific problematic cases.
I think it is better, when there is a clear distinction between the essence of the retraction and the conduct around that action. The first two bulletpoints (why and where) are the essence, The other points are important, but of a different order. So my suggestion would be to replace the 8+7 bullet points to 2 + 13.
NotClareFrancis, I agree with your proposal to a certain extent, but a sanction across international boundaries would likely never work. So, the idea is nice, in theory, but in practice, it will have absolutely no effect. The “shame factor” is still the best bet we have. If we can get 20-30% of the world’s scientists to wake up and browse RW and PubPeer each morning before they start working, there will be this intrinsic element of “fear” that places a check (to some level) on misconduct, because no-one wants to be the next headline. This is not the state of science envisioned 5 or 10 years ago, but this is a necessary reality now.
The question we should be asking is why most scientists do not browse RW or PubPeer every morning before they start their experiments? I claim that there is a sector of scientists who continue to exist in a safe and comfortable cocoon, shielding them from the need to wake up to this ugly truth every morning. These individuals need to know that their niche is no longer safe. Scientists must therefore shame editors, publishers, and others, who use double standards.
What the points indicated by RW indicate is that there is, unbelievably, still a gross lack of accountability by the publishers. So, to hold them more accountable, we have to push for these points to get accepted, and applied, including retrospectively.
Unfortunately, there are far too few vocally critical scientists out there.
I think this publisher nailed it: The ideal general purpose once and for all retraction form: check one or more of the boxes below
http://www.scirp.org/journal/PaperDownload.aspx?paperID=56623