Every year, the U.S. Office of Research Integrity (ORI) issues a series of findings against researchers it has determined committed fraud of some kind. In contrast to many agencies such as the National Science Foundation and around the world, the ORI names the offender, describes the offense, and states the penalty – often a temporary ban on federal funding, for instance.
2016 brought a lot of changes to the agency – it was director Kathy Partin‘s first year, which brought reports of staff unrest (and threats to resign). But there was another notable feature of last year, as we and others have noted – the agency issued only seven findings of misconduct, compared to 14 in 2015 and 11 the year before. (Once a sanction has lifted the finding disappears from the ORI site; here is a count of the agency’s activity from previous years.) We spoke to Partin about this relative lack of action from the agency last year.
Retraction Watch: Why are there so much fewer notices this year than last?
Kathy Partin: First, I want to clarify that while publishing notices in the Federal Register of findings of research misconduct is a very important part of ORI’s work, it is not our only endeavor. ORI engages in a robust array of activities related to oversight of research misconduct investigations and education about the responsible conduct of research. For example, on the education side, we recently released new video case studies and infographics addressing common research integrity issues for research integrity instructors and research teams to use.
On the investigation side, there are several ways our review of a case could end, including by making findings of research misconduct (e.g., through a settlement or issuing a charge letter), or closing a case without a finding of research misconduct. Our output is affected by a number of factors, including how many allegations we receive, whether they involve research misconduct, whether they are simple cases or complex, etc. We also have several cases that we’ve been working on that are very near completion, so we hope to be ready to publish findings on them soon. But there are probably other factors that also account for at least part of the decline. Specifically:
- We currently have three cases for which our findings have been appealed (this step occurs before ORI publishes an official finding). (See our FAQs for more about our process.)
- We are also working on several extremely large and complicated cases, with more than 100 figures that are alleged to have been manipulated. These cases span a period of many years and involve tens of millions of grant dollars. These are critically important to review, but definitely take much more time for us to make appropriate findings.
- I knew when I was hired to direct ORI that our office was understaffed – and had been understaffed for years. Our Division of Investigative Oversight, which reviews cases for research misconduct, had experienced several years of staffing shortfalls. The good news is, since I have been here, we have hired three new investigators, who are now being trained or will soon be trained.They are terrifically talented, but it will take them some time to get up to speed on our specific regulations and processes. Being an investigator is a demanding job, and it is important to maintain the high quality of our scientist investigators. I am happy to report that we are still hiring and working very hard to recruit the best people. After they are fully trained, our new investigators will be ready to assume a full case load.
You also have to remember that ORI is not just about publishing research misconduct findings. Our staff delivered 3 [Research Integrity Officer, or RIO] boot camps in 2016, attended and presented at many national meetings, planned meetings for 2017, produced 4 issues of the ORI Newsletter, and have produced new infographics and video cases for RCR instructors. In many ways it has been an extremely productive year for ORI.
RW: We know that there have been stories about some internal unrest within the department – has that had any influence on the agency’s productivity, and its ability to issue notices?
KP: This has been a year of transition for all of us. Both Scott Moore, ORI Deputy Director, and I, represent new leadership for ORI. We offer a fresh perspective. We are both very curious people (which is usually the case with scientists), and since joining ORI, we have been asking a lot of questions about why and how things get done. We have also undertaken a number of internal assessments and analyses to measure workflow, and although this work is critical to building up an office, those discussions can take all of us away from the task of analyzing case evidence.
We are looking closely at our underlying infrastructure and identifying areas that have worked well for ORI and also, we are trying to see if there are areas that we should grow or change over the next 5-10 years. In the short term, this may temporarily slow us down in some areas, but in the long run, we will really be a lot better equipped to move ORI forward to meet the needs of today’s fast-paced research environment. Our investigators are outstanding, and Scott and I hope to leverage their excellence to position ORI for whatever changes the future brings us.
RW: We spoke with one ORI investigator who is unhappy with the direction the agency is going in, and she said her main issue is your lack of emphasis on intent – i.e., universities can make findings of misconduct without proving it was intentional and knowing (ie, not the result of carelessness), while the ORI has to gather evidence of intent that must pass muster by a legal team before issuing a finding. Would you agree with this characterization?
KP: No. Under 42 CFR 93.104, both the institutions and ORI are required to use the same elements for a finding of research misconduct, which means (and I will cite the regulation here):
93.104 Requirements for findings of research misconduct.
A finding of research misconduct made under this part requires that—
(a) There be a significant departure from accepted practices of the relevant research community; and
(b) The misconduct be committed intentionally, knowingly, or recklessly; and
(c) The allegation be proven by a preponderance of the evidence.
So there are times when we cannot make a finding, despite being convinced that misconduct was committed, because the evidence does not meet these regulatory requirements. I completely understand and appreciate how frustrating this is for both ORI and for the institution. One way to address this is for all of us to be as well-trained as possible in investigating research misconduct in accordance with our regulation. For this reason, ORI offers RIO boot camps to help train institutional RIOs, and we conduct workshops on misconduct processes, and will be co-sponsoring a sequestration workshop, in collaboration with the HHS OIG and the NSF OIG. As challenging as this process is, I believe we can work together to minimize the number of cases for which someone who has committed research misconduct is not held accountable for the damage they have done to the integrity of science.
RW: While there are people who are pleased with your work, we know there are some staff who are unhappy. How would you describe their main source of unhappiness?
KP: It would be presumptuous for me to speak for them. Any time there is new leadership, there is an adjustment period where all members of the team have to get to know each other and each other’s styles. We have a great team, and I am excited about ORI’s future.
RW: Zoe Hammatt recently resigned as director of the Division of Education and Integrity – have other people left, and do others plan to? Is the agency facing manpower issues that have hampered its ability to make findings?
KP: As you know, I can’t speak about specific individuals, but as I said before, when I joined ORI, I knew parts of the office were extremely short-staffed. And, I think a new leadership team should always be prepared for some turnover. Fortunately, however, after years of staff shortages, ORI has moved forward with hiring three new investigators (two are already at ORI, one will soon join). They are now being trained or will soon be trained in our policies and procedures. And we are actively hiring for other long-vacant positions (we have made eight hires since I joined ORI). I am very hopeful that we are moving in the right direction.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
By and by post-publication review is taking things out of the hands of the ORI.
Education has not worked so far.
“education about the responsible conduct of research. For example, on the education side, we recently released new video case studies and infographics” is quaint.
“we have hired three new investigators, who are now being trained or will soon be trained.”
Does the direct think that 3 people will be a match for a multi-billion dollar industry?
I’m always interested about what people mean when they claim that “education has not worked.” What does that mean to you? I assume that you know that in the U.S., misconduct is defined as fabrication, falsification, or plagiarism. Which scientists fail to teach their students to be honest?
And what do you mean by “Does the direct think that 3 people will be a match for a multi-billion dollar industry?” Do you think research misconduct brings in billions of dollars?
“Does the direct[or] think that 3 people will be a match for a multi-billion dollar industry?”
3 people likely costs 300K. The organization is overwhelmed according to the present director.
“Do you think research misconduct brings in billions of dollars?”
About 10% of the total.
“We have also undertaken a number of internal assessments and analyses to measure workflow, and although this work is critical to building up an office, those discussions can take all of us away from the task of analyzing case evidence.” Brain-to-ears: “Whoa, does not compute.” ORI’s investigations branch had been functioning remarkably well despite the lack of a director in the two-year interims before and after David Wright; It remains unclear what was broken?
From the staff I knew (when I left in 2013), the four investigators that left ORI after Dr Partin arrived (not counting the former Deputy Director had been pulled back to help) were the four most experienced. So, who is left to do the training? Won’t that take time from “measuring workflow”? In resetting the course, the last thing a new captain should do is capsize the ship.
In truth, ORI’s problem sits not in its director’s office, but in the lack of support and understanding ‘upstairs’ in OASH. Where is NIH in this (after all, isn’t it their laundry)?
The fundamental problem for timely case resolution has always been with ORI’s meager legal support, which for the last decade was offsite and part time, if not largely absent compared to the early days when ORI’s lawyers were in-office. (Afterwards, OGC became known as the ‘black hole’ for cases.) Having multiple cases waiting on an appeal is a symptom of the problem, not an explanation. At one time appeals (to the DAB) were made after ORI had issued a public finding. What has changed to keep them secret?
Why should effective and timely resolution of ORI cases matter to Retraction Watchers? The answer to that question resides in correction to the literature. How many institutions get their bad actor to leave via a sweetheart deal that defers corrections of the literature to an ORI decision?
The OIG approach, such as the secretive process at NSF, would be a poor model to apply to the massive NIH funded research program. It is certainly not one that would ensure transparency through the retraction process that was linked to a PHS finding.
Sorry about the venting. Thanks RW for asking these questions.
Whenever I hear phrases such as “measuring workflow”, I just assume no actual work is being done by anyone.
Likely so. The telling langauge may emanate from her faceless ‘career-o-crat’ handlers ‘upstairs’ in HHS, to justify whatever changes they want her to make.
I’d feel far more optimistic if 1) there was any sense that they had input from the research community, and 2) that there was not a sense HHS will inevitably tear apart the unique accumulation of statistical-, image-, and computer-forensics skills (for looking at questioned research) that DIO created in ORI. IMHO, That experience does not exist anywhere else.
Thanks John!
Last month I wrote an article based on interviews with a half-dozen current and former staff. My reporting reveals problems that are deeper, pervasive and more serious than described in this Q&A.
Sorry: meant to post the link to the full article:
http://www.hcca-info.org/Resources/HCCAPublications/ReportonResearchCompliance.aspx
Click for the free January issue.
I hit a paywall, coming in June
I sent Dr Partin a copy of my book “Hidden Data: The Blind Eye of Science”. She did not have the courtesy to acknowledge or thank me for it. So much for that. For anyone who is interested, it is available on Amazon in print and downloadable formats.
I just don’t know if a legal approach is the right approach. I think it is a generational gap/science culture issue where best practices are young, and the efforts to mandate or integrate an “RCR culture” have popped up and been quashed repeatedly for decades. Something bigger, more internal needs to happen at the individual institutional level for this to be addressed and to avoid case of RM. This, of course, is very challenging but not impossible.
You are quite right.
RCR is an exhortative approach that does not really address the (unresponsive) 1% who generate the bulk of ORI’s business. (Note the grim implication of the fact that the annual retraction rate is still 20 times less than this small value. Ditto the 70% (?) of ORI’s operating budget that is spent by the education branch. iMHO that goes toward matters that might increase awareness or chase causative theories, neither of which reduce the incidence of misconduct.
ORI RCR has been stuck on promoting values rather than identifying small practical changes that would ameliorate the effect misconduct when it happens (as it will). ORI has done nada-to-nil to publish the individual practical lessons, distilled from its voluminous case experience, regarding how small changes in conduct of research in the digital age might help confront the problem. There is more money in advocating exhortative RCR.
So why is government promulgating values for the research community? Short Answer: $30B. Cynically, and from a grants management perspective, the fundamental reason the government must remain in the research misconduct business is “to ensure proper stewardship of public funds.” (Mere mortals can’t make this langauge up!). Every year the NIH funding circus parade comes to town; in theory, ORI are the small few carrying brooms, walking behind the interesting collection of jugglers and larger wildlife to make sure the parade is welcome next year. That may be why ORI is funded as direct percentage from NIH appropriation rather than specific appropriation to the separate OASH agencies. Unfortunately, OASH Has control over how and where those funds are spent.