A team of Australian medical writers who analyzed four decades worth of retractions has reached the conclusion — we trust you’re sitting — that people in their profession are more honest than, well, the rest of us.
According to the authors, articles in the medical literature are substantially less likely than other papers to be retracted for any reason, including mistakes or misconduct, if they have a medical writer as a declared co-author. The same applies for articles produced with the help of drug and device makers, either financial support or authorship assistance, the study found. And when both occur, retractions are vanishingly rare. Indeed, they found no instance of a retraction resulting from misconduct.
The study, which appears in Current Medical Research & Opinion, has flaws, which we’ll lay out in a minute. But the main conclusion strikes us as reasonable and not terribly surprising. After all, industry may have a bad habit of trying to hide its fingerprints in publishing, but the stakes of perpetrating fraud in the process are orders of magnitude greater than the PR bruises of poor disclosure.
What’s more, medical writers are, in the end, largely transcribers of the ideas, often communal, of others. Going off the reservation isn’t encouraged. And they’re not involved in the generation or analysis of data, so whatever mischief they can effect is limited. Plagiarism, perhaps, but fabrication and other fraud, less likely.
The researchers, led by Karen Woolley, CEO of ProScribe Medical Communications (which “specialises in writing scientifically credible, high quality documents” for clients in pharma, biotech and academia), found 520 retractions in Medline of papers published between 1966 and 2008 that involved human subjects, or about one a month over the 42-year-period. Of the 520 papers, only 463 made the analysis — studies that had “both the full original retracted publication and the associated retraction notice available online.”
Of those, 213 (46%) were retracted because of misconduct, according to the researchers. That figure is nearly twice as high as the 28% found by Liz Wager and Peter Williams in their new paperon how journals deal with retractions — raising the uncomfortable possibility that fraud is more common in the literature involving human subjects.
In the Woolley study, of the papers with acknowledged medical writers, 23 were retracted, but only two involved misconduct. Of the 36 retracted articles (8%) with industry support, 8 (4%) involved misconduct. And when medical writers and industry support collided, the researchers found only two retractions, neither of which involved misconduct.
Retractions for fraud or other reasons occurred much more often with papers having one author, articles from countries considered to be low or middle income — Iran, China, India, etc. — and papers with an author who’d had at least one other retraction. All of which matches our brief experience covering this subject and, as far as the first part, makes perfect sense: after all, it’s easier to get away with fraud when there’s no one to rat you out.
The authors present some rather unconvincing odds ratios to stratify the incidence of retractions — papers that did not receive industry support were a whopping 3.7 times more likely to be retracted for misconduct than for an error. But it’s probably best not to dwell on those since they’re really just statistical tinsel.
Which brings us to those flaws. The biggest caveat that we can see is that, although the study claims to be looking at retractions linked to medical writing, what it’s really looking at is retractions linked to fully disclosed medical writing. Valuable information, sure, but a much different breed of marsupial.
We spoke with retraction scholar and medical writer Grant Steen about the paper, and his reaction was cool:
I think [Woolley is] trying to give a free pass to medical writers based on very flimsy evidence.
The authors declare the two major limitations of their study to be its exclusion of papers written in a language other than English and of articles that did not address non-clinical issues.
Not so, says Steen, who has published two recent studies on retractions. “The major limitation is depending on people to disclose” the use of medical writers, he says.
We also can’t help but chuckle at the last couple of lines of the paper, an off-putting blend of self-serving defensiveness, faux piety and meaningless expression.
The results from this study justify increased attention on factors that were significantly associated with misconduct retractions, such as the absence of declared medical writers, the absence of declared pharmaceutical industry support, single authorship, and the first author having at least one other retraction or an affiliation with a low/middle income country. These results suggest that the risk to the integrity of the literature from non-commercial factors must be managed with as much vigor and rigor as the risk from commercial factors.
So, medical writers and pharma industry good, academics, disadvantaged foreigners and people with “non-commercial” interests — if anyone knows what that means, please let us know — bad.
Now, if only we could find a few good medical writers…
Oh the irony.
The biggest complaint against medical research is that it largely is correlation = causation. So isn’t is neat that this report is no different?
I hope Retraction Watch readers can access the full article so they can make their own judgement on our paper and the analyses (which were, as we state in our paper, conducted by an independent academic statistician). We also highlighted in our paper that we were interested in assessing the involvement of declared medical writers in misconduct retractions. If any reader has a robust method of assessing ghostwriting assistance, please let us know. Why not measure what people are doing right (declaring medical writing assistance) rather than trying to estimate what people are doing wrong (ghostwriting)? We also stated in our paper that we encourage continued scrutiny of industry-sponsored research. Nevertheless, our evidence indicates that non-industry sponsored research deserves equal, if not greater, scrutiny. We also stated in our paper that we applaud efforts from lower-income countries to promote research integrity. We help many authors in lower-income countries to become aware of and adhere to ethical publication practices. Again, I hope readers of Retraction Watch can read the full paper before they form an opinion about our research. We knew our results would challenge popular opinion and this would make some people uncomfortable. We stand by our results and have to rely on people taking the time to read through the full paper before expressing an opinion. We certainly encourage others to repeat our study…there is enough opinion out there, we need more evidence.
Did the authors declare an interest? If not, I demand tehy retract their paper, in the interests of irony.
Bob, as you would see in the paper, full and complete disclosures of our interests were made.
Hi Bob,
As per my comment above, I encourage readers of Retraction Watch to read the full paper before they offer an opinion on the paper. As professional medical writers, who are aware of and adhere to ethical publication practices, we certainly ensured we declared any financial and non-financial competing interests. A certified publication professional would do no less. Just in case you are not aware, there is an international certification scheme available for medical writers (a rigorous test on 150 topics relevant to ethical medical publication practices). I also recommend you read our paper in JAMA where we investigated the prevalence of declared medical writing assistance – we have been declaring our presence since we first started assisting authors with their papers.
OK, first of all, conflict of interest declaration: I run a medical writing company.
To me, the results seem entirely plausible. Fraud is generally something that’s easier to get away with if you have a small number of people who are “in on it”, and make sure that everyone outside that small circle doesn’t have enough information to spot the fraud.
Medical writers are frequently not part of the core team that ran a clinical study, but are brought in at the end to help with publishing the results. It seems likely to me that it would be unlikely for a medical writer to be a sufficiently “insider” member of the team to be convinced to go along with any fraud.
And given that medical writers typically scrutinise the data very carefully when writing papers, it’s even more unlikely that you could pull off a fraud without the medical writer figuring it out.
So to me, these results completely ring true.
@amarcus41: it sounds like you don’t believe the results. Could you point to another piece of evidence that you think backs up your point of view? Or is your point of view opinion-based, rather than evidence-based?
Adam — Thanks for your comments. As we wrote, we find the results unsurprising, too, in part for the reasons you mention. Is fraud easier to perpetrate when no one else has access to your data or manuscript? Absolutely. Are fraudsters more likely to be recidivists? Indeed. And someone with a prior retraction based on a simple error might well be more likely to have a subsequent retraction stemming from that same problem. So no shocker there, either.
Our skepticism stems largely from the significant limitation of the study — the inability to analyze papers on which the work of medical writers has not been disclosed — and the implicit claim that medical writers add a veneer of ethical purity. In our view, transparency and the willingness to disclose (authorship, pharma involvement, etc.) probably are the best insurance against misconduct.
Yes, I agree that the results are only applicable to disclosed medical writers. There are medical writers out there (we don’t know how many: my own research suggests slightly under half of them, but that figure is probably not very reliable) whose involvement is not disclosed, and since ethical standards dictate that medical writers’ involvement be disclosed, the undisclosed medical writers are probably less ethically minded than the disclosed ones. So it’s quite possible that fraud and other forms of misconduct are significantly more common among the undisclosed medical writers than among the disclosed ones.
Still, it does seem to me that Woolley et al’s study gives us good reason to think that those medical writers who are following standards of disclosure are probably helping to raise standards generally.
Thank you Adam and Adam for a more balanced view of what our paper did focus on (declaration of medical writing assistance), rather than what it did not focus on (ghostwriting). I do believe – and we now have additional evidence to support this opinion – that professional medical writers (ie, those who are aware of and adhere to ethical medical writing principles) can help ensure ethical reporting. We can be valuable allies in the fight against ghostwriting and we can be valuable contributors to the TIMELY publication of high quality manuscripts.
Perhaps the most dishonest researchers do not retract their own work if it is found to be faulty, or even if they discover fraud within their team.
That would make retraction an act of honesty, which was significantly associated with the lack of a medical writer!
I love it when you can interpret the results both ways!
Having said that, bringing in someone from outside to write up incorrect research seems unlikely to happen (unless the mistakes are accidental). I suggest that medical writers are not employed by researchers that knowlingly mislead.
Do the commenting medical writers ever uncover fraud or misleading research?
Interesting post DMCW! As one of the commenting medical writers, I can respond to your query as to whether medical writers ever uncover fraud or misleading research. In our experience (10 years of working with authors from every continent except Africa), we have not uncovered fraud or fabrication, but we have prevented plagiarism (ie, misconduct that could result in a retraction) and we have allowed important papers to be prepared far more quickly than they would have otherwise (we help authors meet their obligation to share results in a timely manner). Our PhD-qualified and highly trained writers often educate and alert authors – particularly inexperienced or very busy authors – to ethical publication practices and to best practice guidelines for reporting. The authors are not trying to do the wrong thing, but they may not be aware of the guidelines that ensure they do the right thing. Ignorance is not as bad as ill intent, but can still result in a poorly reported study. We are passionate about helping to ensure research results are shared through publications and believe that professional medical writers (not ghostwriters!) can be an incredibly important ally in the efforts to preserve the integrity of the literature.
@dmcw: my answer to your question is very similar to Karen’s. I don’t think I’ve ever uncovered deliberate fraud, but we have often had to educate authors about the correct way to prepare manuscripts, for example ensuring that data are presented fully, based on what was prespecified. And what has happened a great many times is that my writers have uncovered cock-ups in the way data have been analysed, which have then been fixed before publication rather than having to be retracted afterwards.
Readers of Retraction Watch can now access our peer-reviewed publication in full for free…thanks to CMRO. Please feel free to ask me any questions about the veracity of the data after you have read the paper in full. Better yet, please repeat our study. Some people, with biased agendas, are finding it hard to accept the results (no matter that the results come from MEDLINE and that an indepedent academic statistician conducted all the analyses and that peer-reviewers scrutinised the paper…).
Just out of curiosity Adam and Ivan…how many of your retraction watch postings lead back to papers with declared medical writing and pharmaceutical industry support?