Do investigations into research misconduct allegations need better standards? The Association for the Promotion of Research Integrity (APRIN) in Japan, a group of volunteers who “commit themselves to the promotion of research of high integrity” and provide “e-learning material for research ethics education,” thinks so. Today, we present a guest post by Iekuni Ichikawa, who chaired an APRIN committee that recently came up with a new checklist for such investigations, about the effort.
The procedures currently employed by various institutions in Japan are highly variable; hence there is a risk that complainants or respondents might be treated unfairly and that the public might not be informed of the facts of the matters. We organized the Research Misconduct Investigation Standardization Committee of APRIN in July 2017 to propose standardized procedures for handling investigations of alleged research misconduct. Here, we present the “Checklist for Investigating Allegations of Research Misconduct,” the fruits of our discussions.
Background
Confronted by the recent rise in reports of research misconduct in Japan, government ministries and public funding agencies have issued guidelines on how to respond to allegations of research misconduct, mandating research organizations to deal more stringently with inappropriate conduct of research. Their guidelines outline in general terms what should be done in investigations of allegations of research misconduct. However, they do not provide specifics, such as criteria for finding or not finding misconduct. As a result, organizations are forced to invest unnecessarily large amounts of time and effort, while using their own standards in adjudicating research misconduct allegations. Furthermore, organizations might have to reopen their investigations if flaws are found in their procedures and/or reports. This situation strongly calls for standardization of the procedures for investigating allegations of research misconduct and reporting the results.
The List
Our “Checklist for Investigating Allegations of Research Misconduct” (hereafter, The List) describes critical points for each stage of investigations in chronological order starting from receiving allegations through preparing final reports. The List further calls attention to “Actions,” reminding, for instance, about recommending the suspension of the use of outside funds at an appropriate time, and to “Conformity with rules,” which emphasizes that investigations should conform to rules and regulations drawn up by research organization, government ministries, and funding agencies. As such, The List also serves as a tool for self-checking when preparing reports. The applicability of The List is not limited to medical and life sciences; it is intended for use in all areas of research, including science, technology, humanities, and social sciences.
In preparing The List, we referred to “the Guidelines for Responding to Misconduct in Research” from the Japanese Ministry of Education, Culture, Sports, Science and Technology (effective August 26, 2014) and the article, “Institutional research misconduct reports need more credibility” (Gunsalus et al. JAMA 319: 1315-1316, 2018, published as a statement of the Expert Meeting held in Chicago, Illinois, U.S. in December 2017). This article presents a “Peer Review Form for Research Integrity Investigation Reports” which lists key aspects of investigation reports for quality evaluation. In contrast, The List itemizes critical issues that must be considered when initiating, conducting and reporting the results of investigations.
We wish to add that The List will require revision in the future as the concept of research misconduct changes along with methods of research.
About APRIN
APRIN was founded in April 2016 by volunteers who commit themselves to the promotion of research of high integrity. APRIN’s activities include, among others, providing e-learning material for research ethics education.
Acknowledgments
This article is a concise version of the original article published in Japanese language in December 5, 2018 in Gakujyutu-no-doko.
We are deeply indebted to the inputs and suggestions from Dr. Susan Garfinkel at the U.S. Office of Research Integrity (ORI) (former Director of the Division of Investigation Oversight, ORI), Ms. Zoë Hammatt (former Director of the Division of Education and Integrity, ORI), Dr. Ivan Oransky (Co-founder of Retraction Watch), and officials of the Ministry of Education, Culture, Sports, Science and Technology, Japan Society for the Promotion of Science, Japan Science and Technology Agency, and the Japan Agency for Medical Research and Development.
Conflict of Interest
We received no financial support from any individuals or organizations in preparing The List.
Participants in the discussions of the Misconduct Investigation Standardization Committee
Hiroaki Aihara (The University of Tokyo), Fumie Arie (Sophia University), Makoto Asashima (Professor Emeritus, The University of Tokyo), Hiroyuki Daida (Juntendo University), Jun Fudano (Tokyo Institute of Technology), Yasuhiro Fujiwara (National Cancer Center Japan), Shinji Fushiki (Professor Emeritus, Kyoto Prefectural University of Medicine), Robert J. Geller (Professor Emeritus, The University of Tokyo), Kazuo Hatano (Astellas Pharma Inc.), Toshio Homma (Shinshu University), Iekuni Ichikawa (Shinshu University; Chair), Minoru Kimura (Tokai University), Naoki Kondo (The University of Tokyo), Toshio Kuroki (Professor Emeritus, The University of Tokyo), Kohichi Miki (Keio University), Ikuo Morita (Ochanomizu University), Kazuhisa Nakayama (Kyoto University), Kosaku Nitta (Tokyo Women’s Medical University), Rei Nouchi (Shinshu University), Akira Shinohara (Osaka University), Mikiko Shiomi (The University of Tokyo), and Masayuki Yoshida (Tokyo Medical and Dental University).
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Good effort but I see a few problems.
The guidelines are only that – guidelines. As such even then they are, at best, weak. What is needed to route out cheating in science, be it science-fraud, using fake data to obtain government or charity money or whatever is INDEPENDENT reviews. A friend of a friend should never investigate each other.
Nobody from any insitution should be investigating anybody from the same insitution, or even other insitutions, it simply doesnt work – they all collaborate or know each other. There should be entirely independent bodies, with government backing, that a whistleblower can contact and send information to look into.
I would suggest the PRIMARY POWER of any such body would be to prohibit ANY employee from an institution applying for ANY grants for any reason until all misconductees are removed from their positions by the employer. So, lets say University X has been shown to have employed Professor Y who committed misconduct, perhaps Professor X is a good friend of the Dean who then, as the University senior authority protects his friend and no miscodnuct is found.
An independent body may find there has been miscoduct, not only by the Professor, but also the Dean. The independent body could then ban University X from ALL funding applications from ALL employees until the misconductees are removed.
University investigative panels, often made up from friends of any accused, would very soon stop protecting science-frauds.
I will point out that the existing ORI guidelines (governing the large majority of biomedical research in the US) specifically direct institutions to select committee members who do not have “unresolved personal, professional, or financial conflicts of interest with those involved with the inquiry or investigation.”
I have administered dozens of misconduct proceedings personally, and am familiar with numerous other cases and never once can I recall there being a “friend of the accused” on the faculty panel examining the allegations. Surely this has occurred in certain instances, but I really don’t think it’s fair to say it happens “often.” At smaller institutions, where the leadership structure is more compact and sometimes senior investigators are one level beneath the high-level institutional administrators, this can definitely be a little bit of a struggle, but in my experience it’s always been manageable. I can think of a few cases where an alternate “deciding official” (who, under many institutional policies ultimately signs off for the institution on investigations and reviews appeals) was selected because the “default” deciding official was a supervisor of the respondent in the official chain-of-command.
You make some statements that, from my first-hand experience and from my reading of RetractionWatch and pubpeer, etc., are inaccurate. For example,
“Nobody from any institution should be investigating anybody from the same institution, or even other institutions, it simply doesn’t work – they all collaborate or know each other.” [spelling corrected]
First, not everyone from an institution knows everyone else there, much less collaborates with them. Second, as I have chaired multiple inquiry committees and investigation committees, I will state that there was absolutely no influence on our fact-finding, interviews, deliberations, or final findings and recommendations that were related to committee members’ affiliations. Third, if your contention is true, how do you explain what you have seen at this website? How did Wayne State University make findings against respondents in their two fairly famous cases? How about the Chicago cases? What about the recent case at Washington University in St. Louis? Etc. If your assertion was true, none of these would have seen the light of day.
I am strongly in favor of three changes to research misconduct processes, for different reasons than you stated:
1. Include investigation committee members who have no affiliation with the institution. The reason for this is that it would remove one issue commonly exploited by respondents: that the affiliated member(s) were conflicted AGAINST them.
2. Enlist legal guidance for investigation committees an their inception. As respondents increasingly use the courts to re-decide their cases (e.g., the Kripke saga), misconduct investigations are likely to face more legal scrutiny. In fact, revamp institutional guidelines for investigating misconduct so as to harmonize with more stringent legal requirements. This will likely strengthen the investigation if/when it arrives in front of a judge. For example, if your institution does not swear-in witnesses for their interviews, start doing that immediately!
3. Possibly most important, put some teeth into misconduct sanctions! It’s currently laughable. Millions are wasted on bogus work, money that was procured through fraudulent means. Nothing will come of it, other scientists will be misled, and HONEST researchers are disadvantaged relative to the cheaters. As an example of the magnitude of the research misconduct problem, a colleague recently made a gross estimation. He looked at curcumin studies by a handful of researchers who have lost multiple publications to retraction and/or have been found to be guilty of misconduct. He estimates that these studies alone squandered more than ten million US dollars. Just imagine the lost opportunities this represents.
So…MAKE RESEARCH FRAUD A CRIMINAL OFFENSE!
If 25% or more of medical researchers admit to being aware of science-cheaters and only a handful of misconduct investigations find misconduct from the hundreds of thousands of researchers out there I would suggest there is something very very wrong with the investigative processes/investigators at institutions.
The actual numbers are probably far higher.
http://retractionwatch.com/2018/01/30/reports-misconduct-scientists-industry-academia/
I disagree with Frank Dodd, research misconduct can never be a criminal offence per se. However, if such fraudulent data have been used in ANY grant applications there are already laws in place which are not, by and large at this time, being utilised for criminal prosecutions. For example, here in the UK there are Theft and fraud offences which could be used, entirely separately, from any University, non-legally binding investigation. Such investigations must not use University panel decisions but should come under an independent enquiry. We should remember employees of Universities can come under all sorts of pressures to ensure no misconduct is brought into the light.
Thanks to RW for making accessible the APRIN list! Many of these points, and many more, are covered in a series of training presentations that ORI created in its “RIO Bootcamp” Program, and are available on a website designed of university officials. I wish ORI would make that source of wisdom accessible for the whole community, and not just university administrators. They could do it at a key stroke.
1. The APRIN list starts with the onset of the decision to initiate an investigation. It would be useful to include a short outline section on how to decide as to whether an investigation is warranted, one that involves a progression, or “step-wise” approach. For example, the PHS Regulations governing ORI stipulate preliminary phases in the process, specifically 1) an assessment phase, and 2) an inquiry phase. The “investigation” phase that the APRIN list describes appears to refer only to what ORI would treat as a third phases (or stage).
2. One of the key factors affecting the success of an investigation involves the logic underlying a professionally, sensitively, and discretely executed sequestration of evidence that enters into phase 1 and 2. For example, an institutional official can visit a scientist in the “assessment stage” and have a private chit-chat about an allegation. But if the scientist responds, “Gee -whiz, funny thing, but I can’t seem to show you the notebooks (or computers) showing that data, or basis for a questioned image,” then an immediate sequestration of evidence is warranted.
3. Supporting Dr. Jondro’s comment, in my 20 years at ORI I did not see evidence that a committee made up of faculty members at the same institution limits fact-finding and the reporting the results that ORI needed to make its decision, per se.
4. I did see a basis for forming an opinion that a blanket criminalizing of what is defined as “research misconduct” would i) limit reporting of allegations, ii) discourage initiation of investigations, and iii) severely limit the needed corrections of the research literature that follows fact-finding process of those allegations. Just consider the HMS Partners case, for example.
John Krueger, ORI (1993-2013)
Thanks for your thoughts, Dr. Krueger — it’s really nice to have well-established, known voices of experience in the Research Integrity field chiming in on these topics. The ORI boot camp is excellent, and I’d also echo your suggestion that it be made more widely available.
Under every institutional policy I’ve worked under, the sequestration is meant to occur immediately after the RIO determines the allegations fulfill the statutory definition of “research misconduct,” were not obviously brought in bad faith, and are indeed within his/her institutional purview to investigate. In my experience, most respondent attorneys are very eager to criticize the timing, method, personnel qualifications, and any other aspect of the sequestration that’s maybe imperfect — and rarely do institutions truly have expert digital forensics on hand and often institutional officials do apply some discretion during this process in order to minimize laboratory disruption or to prevent uninvolved parties from becoming aware of the inquiry. Often these things seem to lead to attorney questions over the validity of evidence, and although I’ve never had a case fall completely apart over it, it can be a really difficult thing for an institution to pull off in a legally unassailable way, even with everyone trying their best. At my present institution right now I’m trying to work with our IT folks to ensure the storage platform we’re moving to will enable better silent, back-end sequestration and provide nice audit trails for everything. I’d strongly recommend RIOs work closely with their technology departments to proactively explain their needs and prepare for a misconduct sequestration before it happens.
That said, I think it all very much supports your point about criminalizing RM. I’ve said it to many people, that respondent attorneys make our job difficult enough as it is. Justifiably so, one could easily argue — professional livelihoods and personal reputations are at risk in these proceedings, and the accused certainly has every right to make a vigorous defense of him/herself. Should one’s own freedom be thrown into the mix, I would expect that this would only make respondents dig in harder, cooperate less, and involve attorneys even more often than they already do. And indeed, I think that would ultimately lead to less of the tainted literature being corrected/retracted, which is what I feel we need to prioritize most of all. The legal aspect is so often the rate-limiting step so to speak when the scientific aspect is very frequently self-evident, as many of the frequent commenters here and on PubPeer would attest.
@John Kruger
Look at Prof Richard Eastell (https://mellanbycentre.org/richard-eastell/) still winning many research excellence prizes (https://en.wikipedia.org/wiki/Richard_Eastell) who was in a case involving Aubrey Blumsohn, and others. It is clear that research misconduct does not only go unpunished with the current system, but may well often be rewarded. This may well involve millions of pounds and decades of research. The public may well be put in danger due to research misconductees being allowed to continue their wares.
I suspect this is the tip of the iceberg. There do need to be independent groups investigating all suspected research fraud. Evidence placed on Pubpeer could be a good starting point.
For example, researcher A puts professor Ys data on pubpeer, questioning the validity of Professors Ys research (in a publication in Nature, say). An INDEPENDENT body could then request all raw data from Professor Y and make an independent, non-bias investigation. Universities could be made to agree to these findings as a mandatory requirement to apply for all government and charity funding. Indeed all HE funding too.
Human Resources, whose main role is to protect the reputation of their respective University, would be forced to have a change of tactics whereby they could remove science-fraudsters if an independent body found they were guilty of research misconduct.
What is transparent for all to see is that scientists and investigative committees at present may be liable, for whatever reason, to not find misconduct where it does exist and this may pose dangers to the public. Therefore it is in the interest of all to have a robust, fail-proof system in place to protect the public from science-fraudsters. Safety for all is certainly within the remit of public interest.
Another issue is the role of RIOs. At some institutions, these can be people of varying levels and experience, and often times many hats. To lay all the responsibility on this person at any institution, which occurs, is also a liability to an institution unless that person is well-experienced and has strong institutional legal support/involvement. The role of RIO is a risk to an institution and along with suggestions above, should also be reevaluated as a way of “enforcement”.