Under threat of a lawsuit, an education journal changed its mind about publishing a paper that it had already accepted after peer review.
Last summer, Education Policy Analysis Archives, published by the Mary Lou Fulton Teachers College at Arizona State University, informed San Francisco State University professor Stanley Pogrow that the journal would be publishing his paper criticizing a widely used reform intervention for schools in poor districts called Success For All.
According to Success For All‘s website, the program is currently used in more than 1,000 schools in 48 states and received just over $10.5 million in grant funding in 2015. In 2010, the program was one of four recipients of the U.S. Department of Education’s $50 million Investing in Innovation Scale‑up grant.
Yet when Success For All’s co-developer, Robert Slavin at Johns Hopkins University, read a pre-publication draft of the paper, he threatened the journal with legal action if they published it. According to Slavin, the manuscript contained “libelous” and “defamatory” statements.
Subsequently, ASU declined to publish the accepted paper, and instead told Pogrow they would only publish a revised piece on the methodology used to evaluate which school interventions are effective — and thus should receive public funds. The revised paper does not directly mention Success For All or Slavin in the text (although it cites past articles by Pogrow criticizing the program). The revision, which Pogrow agreed to, “gutted the article,” he told us.
That revised paper, “The Failure of the U.S. Education Research Establishment to Identify Effective Practices: Beware Effective Practices Policies,” was published January 23, 2017.
Pogrow and Slavin have previously sparred in the literature; Pogrow is developer of a competing educational program to Success For All, listed on the first page of the EPAA paper as a conflict of interest:
Conflict of Interest Statement: Stanley Pogrow is the developer of the Higher Order Thinking Skills (HOTS) project, a specialized thinking development approach for Title I and Learning Disabled students in grades 4-8.
Pogrow initially submitted his manuscript to EPAA in August 2015, according to a copy of the original version, which Pogrow forwarded to us. After a round of blind peer review and revisions, EPAA accepted the paper in June 2016.
The manuscript was originally titled, “The 25-year Failure of Success for All: A Case Study of the Large-Scale Failure of the U.S. Education Research Establishment.”
Here’s is Pogrow’s description of the original paper:
The 43 page article sets forth my arguments, based on evidence I cite, that (1) in fact the program had not actually been successful, (2) that it only appeared to be effective in studies in top research journals because of clever but misleading presentation of statistics, and (3) that in my view there is a rampant conflict of interest among researchers between being both a vendor of programs and operating government funded centers to identify effective programs. The article also documented how the peer-review process of the government and top research journals that continued to publish and fund Success For All was tainted by the panelists’ gross ignorance of the widely available contrarian research. If FDA review panels approved a drug under such circumstances there would be public outrage. Most importantly, the article documents the basis for my opinion that while Success For All received tremendous support from the research community as the #1 demonstration of the power of research to improve practice, it was not only ineffective, but that it was also a highly regressive approach that stymied the development of the most vulnerable students.
Based on email exchanges forwarded to us by Pogrow between himself and Audrey Amrein-Beardsley, a professor at ASU and lead editor of EPAA, the journal was planning to publish the piece in January 2017 alongside responses from others on the article, plus a final response written by Pogrow. The journal was also planning a publicity campaign for the paper.
Amrein-Beardsley and Pogrow did discuss legal questions about the paper early on, said Pogrow, yet Amrein-Beardsley did not seem to have concerns. In an email dated September 8, 2016, she wrote to him:
I too am confident that it is not assailable legally.
According to the email thread, Amrein-Beardsley planned to send Pogrow’s manuscript out to respondents in September 2016. One of those included Slavin, who confirmed to us that he responded to Amrein-Beardsley with an email outlining his concerns. Slavin’s email, reviewed by RW, asked Amrein-Beardsley to reconsider publication to avoid legal action.
We asked Slavin to clarify why he felt the paper should not be published. He replied:
I understand and value the tradition of scholarly debate, and would never have responded to Dr. Beardsley if I merely disagreed with the paper. But I felt that statements in the paper were libelous, and therefore should not be published. Pogrow maintained that I attempted to influence legislation and educational policy to favor our Success for All program. I suppose he has a right to make this claim. But he further stated that I did this to enrich myself, personally. This is defamatory. Success for All was disseminated by Johns Hopkins University until 1997, and then spun off into its own non-profit (501 (C)(3)). Other than a portion of my salary, always set to be identical to my Johns Hopkins salary, I have never made any money from sales of Success for All. At the moment, I do not receive any funding whatever from my work at Success for All, where I am Chairman of the Board. To state as a matter of fact that I am seeking to influence the field of education for money is both factually wrong and defamatory, and this, as I understand it, is the definition of libel.
According to Pogrow, here’s what happened next:
The editor indicated to me that she had sent the threatening letter and article over to the university counsel’s office, but that she had re-reviewed every word of the article and that it was solid and was confident that there would be no problem. Several days later she called me back and said that the article could not be published. She told me that the university counsel’s office had hired a consultant who said that there were “some red flags.” (I was never told what the red flags were.) As a result the dean of the college would not let the piece be published. I told the editor that I would be glad to remove any specific language that was a “red flag.” However, her boss was unyielding. It was decided that I could do a revised piece that only focused on general methodology issues but could not discuss Success For All at all, the conflicts of interest, or Slavin.
Both Amrein-Beardsley and Carole Basile, dean of ASU’s Mary Lou Fulton Teachers College, declined to comment on the events.
Pogrow eventually re-wrote the paper to focus on general methodology, though he was unhappy to do so. He told us:
This gutted the article and severely diminished its importance, and represented ASU caving in to fear and not allowing the exposing what I argue is a major education research scandal that has endured for decades. The blind reviewers agreed that I had made such a case in a professionally responsible manner. I would have been more than happy to make changes to the sentences that worried the ASU lawyers.
In the email thread between himself and Amrein-Beardsley, he asked to include a statement on the cover page:
What would be appropriate and truthful on the cover page would be to keep the existing statements about the dates when the manuscript was received, revisions received, and accepted as on the original, and then add a statement something like:
Due to a legal threat of a defamation suit by an individual whose research was critiqued in the article, this article is a shortened version of the accepted manuscript with the critiques removed.
The final, published article did not mention the legal threat, just this:
This article is a substantially shortened version of the original, which went through two rounds of peer-review.
In late 2016, after a publication date had been set for the revised article, Pogrow consulted lawyer Kenneth White, a defamation specialist, to review his original manuscript and Slavin’s defamation threat. In a letter to Pogrow in December, White (whose blog may be familiar to our readers) concluded:
In short, though your article offers opinions and conclusions about Professor Slavin’s program, it does so based on disclosed facts and sources. That is the epitome of a protected opinion, particularly in the scholarly context. I am frankly shocked that a reputable institution like Arizona State University would yield to such a vague and transparently meritless legal threat, particularly when public funds are at issue and the issue is central to the education of impoverished students.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
As the author of the suppressed article Professor Slavin’s explanation for why he threatened a defamation suit is in my opinion unjustified. He claims that he threatened Education Policy Analysis Archives (EPAA) with a defamation suit if it published my article entitled “The 25-year Failure of Success for All: A Case Study of the Large-Scale Failure of the U.S. Education Research Establishment” because I accused him in it of personally enriching himself. That is not true. I never accused him of enriching himself. In fact, I specifically indicated in the article that I would not speculate on his motives. Indeed, such accusations would never have been allowed in any quality research journal such as EPAA. I do not care what his motives were or what his bank account is. I only care about whether his program that was widely used and continues to be promoted as a successful reform for inner city high-poverty schools was actually effective and whether the published research claiming success was valid—and if they were not, to consider what the devastating consequences were for efforts to improve inner city schools and for the profession of educational research. I believe the reasonable inference from Professor Slavin’s actions is that he is attempting to suppress questioning of his findings. “
In the suppressed article I called for the publications that published research about Success for All to explore retracting that research. Those who conducted the rigorous blind review of my article before Prof. Slavin’s threats agreed that it contained sufficient evidence to warrant consideration of retraction.
The proper scientific way to adjudicate what has happened and to stand up for academic freedom is for the American Educational Research Association (AERA) to publish my argument and the evidentiary basis for retracting the research it published in its journals claiming that Success for All was effective. Professor Slavin should then be provided with an opportunity to respond (as the editor of EPAA tried to get him to do). Then let an impartial panel of top researchers weight the evidence on both sides. However, If the evidence I put forth continues to be suppressed it will encourage others whose educational research is criticized to respond with the threat of a defamation suit.