Andrew Gelman, a statistician at Columbia University and a friend of the blog, has corrected a 2008 paper in the blunt way you’d expect him to.
Here’s the notice in the Annals of Applied Statistics:
In the paper, “Should the Democrats move to the left on economic policy?” AOAS 2 (2), 536-549 (2008), by Andrew Gelman and Cexun Jeffrey Cai, because of a data coding error on one of the variables, all our analysis of social issues is incorrect. Thus, arguably, all of Section 3 is wrong until proven otherwise. We thank Yang Yang Hu for discovering this error and demonstrating its importance.
In a post called “Retraction watch” — hey now! — Gelman writes:
“Thus, arguably, all of Section 3 is wrong until proven otherwise.” As with retractions in general, it makes me wonder about the rest of this guy’s work.
Dr. Anil Potti would be
pooping in his pantsspinning in his retirement.
Gelman and Cai’s original paper concluded:
Under our model, it turns out to be optimal for the Democrats to move slightly to the right but staying clearly to the left of the Republicans’ current position on economic issues.
Readers can of course judge for themselves how that model worked for U.S. elections since 2008.
It is not clear to me why this is a Correction rather than a Retraction. Section 3 (stated to be wrong) is the central core of the paper – without section 3, this paper is really not a candidate for publication.
Major Correction vs Retraction is a frequent debate on this site, I was wondering why this was not brought up as an issue here ?
I went to the journal web site. There is no link to – or indication of – the Correction. The Correction is also not indexed under the authors name on the web site. Instead you need to browse the letter section to discover the Correction.
http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aoas/1215118527
The URL to get the letter is currently
http://imstat.org/aoas/next_issue.html
Scroll down to the end of the page to see
Correction to “Should the Democrats move to the left on economic policy?”
We’ll see how long it takes to get a link to this letter on the page for the original article, and hopefully a newly-marked version of the paper pdf indicating the “correction”.
Indeed Section 3 begins with
“3. Empirical data on voter and candidate positions on issues.
The analysis presented in the preceding section is interesting, counterintuitive, and potentially appealing if you think it would be desirable for the two parties to be further apart, to present a clearer choice to voters. We test it using voters’ placements of themselves and the candidates on economic and social issues in the 2004 National Election Study.”
so now we know why it was counterintuitive (it was incorrect!) but why that is interesting or potentially appealing remains open to speculation.
Thus Roger is correct in wondering why the whole paper is not retracted. Presumably Gelman thinks the math and statistical model concepts are correct, but a simple error in the data was to blame for the incorrect analysis results. Hmmmm, where did we last see that?
This Reaction Watch entry bothers me on multiple levels – in addition to the issues noted in my two entries above. A short list of additional issues are:
1) The statement “Thus, arguably, all of Section 3 is wrong until proven otherwise.” is disingenuous. Since the author states that the analysis is wrong because of an error – this analysis will always be wrong. A subsequent study that might come to the same conclusions will not make this study correct.
2) Why is the Correction signed by only one author ? Does the other author not agree ?
3) I found the blog reference (both Gelman & Oransky) to Anil Potti incomprehensible. What is the purpose of referring to [someone] with no obvious connection to this Correction ?
My impression is that “a friend of the blog” has been given an uncritical pass. The “blunt way you’d expect” appears meritorious until you consider the opaque nature of this Correction (why not a retraction? why is it signed by only one author? why is no information provided about the source of the mis-coding error? why no link to the original manuscript by the journal? why the gratuitous reference by the author to Potti?). There appears to me to be room for some improvement here.
Roger: I will respond to all your comments.
Your comment 1:
My reply: I sent a note to the editor of the journal saying that “I’d like to publish some sort of erratum notice regarding my paper.” He told me to submit it as a Correction, so that’s what I did.
Your comment 2:
My reply: The editor told me the correction is scheduled to appear in a future issue of the journal. The journal is both print and online, and accepted items are not published immediately. That’s why they have a page for items that are scheduled to appear.
Your comment 3:
My reply:
I looked up “disingenuous” in the dictionary and it says “Not candid or sincere, typically by pretending that one knows less about something than one really does.” Given that you have expressed this attitude about me, I suspect it will be difficult for me to respond in any way that would satisfy you. Nonetheless, I will try.
I do not find the statement, “Thus, arguably, all of Section 3 is wrong until proven otherwise” to be insincere. My statement is descriptive and precise. Our paper has four sections. Sections 1 and 2 are unaffected by the data, and Section 4 is expressed conditional on Section 3. Of course I could’ve just written that the paper is wrong (by the rules of logic, if C is wrong, then “A and B and C and D” is necessarily wrong) but that would be less informative. Also, I wrote “arguably . . . until proven otherwise” because it is possible that a reanalysis with correctly-coded data would result in similar results. It also is possible that some of the conclusions in Section 3 will remain valid while others would fall. I don’t know. Again, I could’ve simply written “Section 3 is wrong,” but that would’ve been misleading, as readers of the journal are concerned with the correctness of the substantive claims. When I say the claims cannot be trusted, this is different from saying that I know the claims are wrong. I endeavored to make the correction notice as accurate as possible. (This is also my response to Steven McKinney’s comment.)
As to the other items: My coauthor is a former student who I have not seen in awhile. The source of the error was that we miscoded a question from the National Election Study. And I mentioned Dr. Anil Potti on my blog because I thought it was funny. I think that pretty much covers it.
“My reply: I sent a note to the editor of the journal saying that “I’d like to publish some sort of erratum notice regarding my paper.” He told me to submit it as a Correction, so that’s what I did.”
Obviously the decision to correct as opposed to retract was approved the by editor. I believe the original posters were trying to make the case that the editor was wrong, and that the uncertainly regarding the results in Section 3 should have resulted in a full-fledged retraction.
Different fields, different terminology. In biology they call them retractions, in statistics they call them corrections. I published a false theorem once. When I learned of the mistake, I told the journal and they published a correction. It was the same as a retraction, they just called it a correction.
“Different fields, different terminology. In biology they call them retractions, in statistics they call them corrections.”
Wait a minute. The editor of the the Statistics journal CDSA certainly used the word “retracted” to refer to a problematic paper:
“The journal publisher’s legal team “has decided to retracted the study,” said CSDA journal editor Stanley Azen of the University of Southern California, following complaints of plagiarism.”
(See: http://www.retractionwatch.com/2011/05/17/controversial-paper-critiquing-climate-change-science-set-to-be-retracted-because-of-plagiarism/)
Note that I’m not defending Wegman’s work, or suggesting that you’re guilty of any kind of misconduct. However, clearly the concept of “retraction”, and the associated term, is used in the field of Statistics as well.
Sorry Goofball – this is a “friend of the blog”. We are not allowed to question whether this is a Correction, a Partial Retraction or a Retraction. Who knows what this is? What is the meaning of is? The author is “blunt the way you’d expect”, so why do you need more ?
Perhaps you should report this to “Retraction Watch” as an example of how a Retraction should not be done.
Ugh. I guess it serves me right for checking on these comments. Don’t worry, I won’t do it again!
Please let me state that we all make mistakes, and you should be commended for correcting an error that anyone one of us could have made (any many of us perhaps have made). I just think that the correction versus retraction concern is valid. People outside of Statistics may see this “corrected” manuscript and think that the error has been “fixed”, which may not be the case here.
However, you may see this as merely a semantic issue of little concern. (And you may not even be reading this.) Oh well.