Two years after questions surfaced about work by former Harvard psychology professor Marc Hauser, an official government report is finally out.
It’s not pretty.
The findings by the Office of Research Integrity were first reported by the Boston Globe, which was also first to report the issues in Hauser’s work. They’re extensive, covering misconduct in four different NIH grants (we’ve added some links for context):
Respondent published fabricated data in Figure 2 of the paper Hauser, M.D., Weiss, D., & Marcus, G. “Rule learning by cotton-top tamarins.” Cognition 86:B15-B22, 2002, which reported data on experiments designed to determine whether tamarin monkeys habituated to a sound pattern consisting of three sequential syllables (for example AAB) would then distinguish a different sound pattern (i.e., ABB). Figure 2 is a bar graph showing results obtained with 14 monkeys exposed either to the same or different sound patterns than they were habituated to. Because the tamarins were never exposed to the same sound pattern after habituation, half of the data in the graph was fabricated. Figure 2 is also false because the actual height of the bars for the monkeys purportedly receiving the same test pattern that they had been habituated to totaled 16 animals (7.14 subjects as responding and 8.87 subjects as non-responding).
Respondent retracted the paper in 2010 (Cognition 117:106).
• In two unpublished experiments designed to test whether or not tamarin monkeys showed a greater response to certain combinations of unsegmented strings of consonants and vowels than others, Respondent falsified the coding of some of the monkeys’ responses, making the results statistically significant when the results coded by others showed them to be non-significant. Respondent acknowledged to his collaborators that he miscoded some of the trials and that the study failed to provide support for the initial hypothesis.
This research was never written up for publication.
• In versions of a manuscript entitled “Grammatical Pattern Learning by Human Infants and Monkeys” submitted to Cognition, Science, and Nature, Respondent falsely described the methodology used to code the results for experiments 1 and 3 on “grammar expectancy violations” in tamarin monkeys either by claiming coding was done blindly or by fabricating values for inter-observer reliabilities when coding was done by only one observer, in both cases leading to a false proportion or number of animals showing a favorable response.
Specifically, in three different experiments in which tamarin monkeys were exposed first to human voice recordings of artificial sounds that followed grammatical structure and then exposed to stimuli that conformed to or violated that structure, Respondent (1) provided an incorrect description of the coding methodology by claiming in the early versions of the manuscripts that “two blind observers” coded trials and a third coded trials to resolve differences, while all of the coding for one experiment was done just by the Respondent, and (2) in a revised manuscript, while Respondent no longer mentioned “two blind observers, he claimed that “Inter-observer reliabilities ranged from 0.85 to 0.90,” a statement that is false because there was only one observer for one of the experiments.
Furthermore, in an earlier version of the manuscript, Respondent falsely reported that “16 out of 16 subjects” responded more to the ungrammatical rather than the grammatical stimuli for the predictive language condition, while records showed that one of the sixteen responded more to grammatical than ungrammatical stimuli, and one responded equally to grammatical and ungrammatical.
Respondent and his collaborators corrected all of these issues, including recoding of the data for some of the experiments prior to the final submission and publication in Cognition 2007.
• In the paper Hauser, M.D., Glynn, D., Wood, J. “Rhesus monkeys correctly read the goal relevant gestures of a human agent.” Proceedings of the Royal Society B 274:1913-1918, 2007, Respondent falsely reported the results and methodology for one of seven experiments designed to determine whether rhesus monkeys were able to understand communicative gestures performed by a human.
Specifically, (1) in the “Pointing without food” trial, Respondent reported that 31/40 monkeys approached the target box while the records showed only 27 approached the target (both results are statistically significant), and (2) there were only 30 videotapes of the “Pointing without food” trials, while Respondent falsely claimed in the paper’s Materials and Methods that “each trial was videotaped.” Respondent was not responsible for the coding, analyses, or archiving but takes full responsibility for the falsifications reported in the published paper. Respondent and one of his coauthors replicated these findings with complete data sets and video records and published them in Proceedings Royal Society B 278(1702):58-159, 2011.
• Respondent accepts responsibility for a false statement in the Methodology section for one experiment reported in the paper Wood, J.N., Glynn, D.D., Phillips, B.C., & Hauser, M.D. “The perception of rational, goal-directed action in nonhuman primates.” Science 317:1402-1405, 2007. The statement in the paper’s supporting online material reads that “All individuals are . . . readily identifiable by natural markings along with chest and leg tattoos and ear notches.” In fact, only 50% of the subjects could be identified by this method, thus leading to the possibility of repeated testing of the same animal.
Respondent and one of his coauthers replicated these findings with complete data sets and video records and published them in Science 332:537, 2011 (www.sciencemag.org/cgi/content/full/317/5843/1402/DC2 – published online 25 April 2011).
• Respondent engaged in research misconduct by providing inconsistent coding of data in his unpublished playback experiment with rhesus monkeys exploring an abstract pattern in the form of AXA by falsely changing the coding results where the prediction was that habituated animals were more likely to respond to an ungrammatical stimulus than a grammatical one. After an initial coding of the data by his research assistant, in which both Respondent and assistant agreed that an incorrect procedure was used, the Respondent recoded the 201 trials and his assistant coded a subset for a reliability check.
The Respondent’s codes differed from the original in 36 cases, 29 of them in the theoretically predicted direction, thereby producing a statistically significant probability of p = <0.01. Respondent subsequently acknowledged to his collaborators that his coding was incorrect and that the study failed to provide support for the initial hypothesis. This research was never written up for publication.
Hauser “neither admits nor denies committing research misconduct” but has agreed to a three-year period, starting August 9, 2012, in which he will have any of his research that is funded by the Public Health Service — that’s the parent body of NIH — supervised and certified. He also can’t do any peer review for the PHS, nor serve on any PHS committees.
So after over four years of investigation, all that the scientific community learns about what happened is a meager 8 pages of information on this case of misconduct. How about all the other papers (co)authored by Hauser? Were they even studied for potential misconduct or bias and if so what were the findings? Harvard does not appear to care a lot about the scientific damage this case has caused. Hauser’s work continues to be cited 500+ times a year but the field is kept in the dark on whether his other results are trustworthy. In this sense, the Stapel enquiry is much better, because it covers all of his papers.
Examine all the papers is the way to go. I recently went through 40 papers by a Harvard professor.
It is not as difficult as many might imagine. The issue was the data. You look at the data. On the page you can see cases of data reuse and the same data being used to represent other things.
The most important thing is to get a lead. One addled paper means that there are likely more.
http://www.science-fraud.org/?p=370
Sure as eggs are eggs there were:
http://www.science-fraud.org/?p=438
http://www.science-fraud.org/?p=513
I have no idea who runs the “science fraud blog”. Whoever it is, they (number unknown) are performing a public service.
Yes and no. They are making accusations public, but this causes a problem when investigations have to be done (which, in all fairness they do) and the guilty scientist, knowing the jig is up before hand, destroys all the incriminating proof. The people at Science -Fraud would be doing a better service by officially reporting their accusations.
In reply to Sam September 5, 2012 at 3:07
“investigations have to be done” aren’t we allowed to look at publications? Publication does mean something that the public can look at.
If the authors destroy the incriminating proof they will have no primary data so the articles should be retracted in that case too. The burden of proof rests with the authors.
Fernando is right. Papers are in the public domain and fraudsters take public resource (tenured positions, grants) that is destined for scientists. They are also entrusted, if they work in a universit, with education!! So pointing out, for example, that data are duplicated between papers describing different experiments is not different from blogging your scientific opinion (or the conclusion of your lab’s journal club!) on a real paper.
Moreover, a theme here and on other sites is that the institutional response is rarely vigorous, so fraudsters seem to perpetuate for a good while. If nothing was placed into the public domain, we would probably believe that there isn’t a problem.
According to the Boston Globe article cited by Ivan, there will be no general review beyond what has already been done. Given the expense and difficulty of the studies, I suppose that’s inevitable. IIRC, Hauser’s knowledge of, and access to, tamarins was also almost unique — that being one of the reasons it took a long time to detect problems. Finally, experimental primatology in general is under some political pressure at the moment. Even if Harvard had the money, time, and skills, I doubt they’d see this as the best place to spend accumulated political capital. I don’t know, and have no opinion about, whether this was the right call by Harvard. But it was legitimately a hard call. The resources needed to review and/or replicate this corpus of work (on what may well be a model system of marginal worth) would have to be taken from somewhere else.
I’m sure this is a real mess for people in your field, and I really sympathize. But think of this as basic World War I-style triage. Some cases can be bandaged up and sent back to work; some can be saved by hard work and maybe an amputation; and some can only be made a bit more comfortable at the end. Isn’t this one of those last cases?
Regarding the issue of data that is difficult to obtain… Imagine the case of a patient with a rare form of brain damage. There is a journal called “NeuroCase” that specializes in this. It’s not a high impact journal, but it would be impossible to replicate any of the studies BY DEFINITION. You can always argue that the patient’s brain has reorganized, if someone tries (assuming one is persistent enough to go through the hassle of getting all the ethical approvals and privacy protection obstacles).
The submission of false data in a grant application is a federal crime punishable by imprisonment and/or a fine. In 2006 Eric Poehlman, a research scientists at the University of Vermont,was sentenced to a year in prison and a $100,000 fine for having submitted false data in grant applications.
Dr. Hauser’s actions may not trigger prosecution, but some potential Respondents might be deterred by the knowledge that the penalty for misconduct can go beyond exclusion from federal funding for x years.
I believe it’s a felony. Making false statements on grant reports is a felony.
Do you by any chance have a US Code section for that?
Search: grant application gives too many hits. Any ideas?
I don’t, but I remember reading that many times when submitting progress reports for federal agencies (I mean, the site would have a message saying that before you clicked the “submit” button; no ambiguity there).
@John Mashey: Start with 18 USC sec. 1001. There are a great many more specific provisions, but 1001 is always applicable.
I have seen and reported on false information provided in grant applications (ghost papers in CV) in Brazil. Well, the result was as expected — none, no one even wanted to hear about it. At least that one didnt get his funds, but surely many do succeed.
Social and cognitive psychology (animal or human) have a long way to go to be considered “science” and the Hauser and Stapel affaires are only high profile examples of this. Confirmation bias is as reprehensible as deliberate fraud. Perhaps it is time for scientists – like engineers or physicians – to be liable for their misconduct and be subject to litigation for tort. They surely breach many layers of trust and/ or “good faith”.
Obviously, you have no idea what you are talking about. Experimental psychology is as solid a discipline as any other, it just deals with more complex systems. Plus, there are ethical limitations on the kinds of studies that can be done to reverse engineer these biological systems (unlike in physics).
Your comment has several layers and you make a good point in the end. However, concluding that social and cognitive psychology are not science from the two high profile cases mentioned is a very good example of scientific discovery made in, surprise, surprise, those two disciplines! You’re generalising from the misconducts of two people. Besides, their misconduct has nothing to do with social or cognitive psychology being a science. It has to do with how the academic system works, with the servile attitude, the leader cult, and large egos that are attracted by that kind of easy fame and control. Psychology is much of a science as any other science, if we’re talking about systematic scientific research. However, I must add that often psychology has been to quick to jump into experimentation and has stayed away from observation (exactly for the fear of not being considered science). Anyway, the scientific method is the work tool of psychology research. Just because you don’t understand it, it doesn’t change that fact. Misconduct can and will pop up even with the scientific method…that’s human nature! And I’m sure you can find solid evidence of that in some healthy scientific findings within social or cognitive psychology. But if in doubt, you can always try to replicate the findings as that is the only sure way to test and advance scientific knowledge!
Well said, Rita.
This is so bad! Makes us all look like unreliable, cheating assholes! Sorry, my brain is now no longer trying to formulate a sentence that displays intellectual ability. It’s exhausting reading about these things!
Am I the only one who is surprised he only got 3 years SUPERVISED research with grants in the future? People have done a LOT less and got harsher sanctions (with just debarment) than he got.
Well, he did lose his job as well.
I personally think keeping consequences low for scientific cheats and constructing pathways back is a positive idea – if only that it is likely to create an environment in which whistleblowers are more likely to be able to continue in science. As it is I think you would find that far more whistleblowers are forced out of science than cheats. If the stakes were not so high perhaps outcomes would not be so bad? Think of it like the Laffer curve or supply-side science integrity.
Cheats are going to fight tooth and nail to avert disclosure and generally they have the skills and networks to successfully game the system while the poor naive whistleblower tries to play everything by the book – and then ends up being labelled a stalker because he posts too many emails on RW!
I thought he resigned?
Perhaps you’re right about the keeping consequences low for science cheats, but this guy was in it for a lon time. Who knows how long he’s done it?
I still think his punishment was a slap on the wrist.