It turns out a 2014 paper that found a surprising pattern of plant migration in response to global warming was not so surprising after all — it’s been retracted by the authors due to a mistake in the statistical analysis.
Most studies on migrating populations have found that species around the globe move north to escape the rising temperatures. But the authors of the 2014 paper in Global Change Biology found the opposite — according to their analysis, many plant species in Western North America had been migrating south, toward warmer climates.
The lead author Melanie Harsch told Science in 2014 that the team had been suspicious of the analysis, and so ran it twice:
“Initially, we thought there was something wrong with our analysis—species distributions are expected to shift upward, not downward,” says team leader and plant ecologist Melanie Harsch. “But we redid the analysis and we got the same results.”
In the end, the authors reasoned the plants were moving south to where there was more water available. That explanation, however, ultimately didn’t hold water — according to the notice, a “coding error” had invalidated the results.
Here’s the notice for “Species distributions shift downward across western North America”:
The above article, published online on 18 August 2014 in Wiley Online Library (wileyonlinelibrary.com), has been retracted by agreement between the authors, Dr Melanie Harsch and Associate Professor Janneke Hille Ris Lambers, journal Editor-in-Chief, Professor Stephen Long, and John Wiley & Sons Ltd. The retraction has been agreed for the following reasons: a coding error affected the results and therefore invalidated the broad-scale conclusions presented in the article. The article presented broad-scale patterns of species distribution shifts in response to recent climate change. Unfortunately, it has since been found that one approach used to account for sampling bias, the null model approach, was affected by the coding error. Following the identification of the coding error, we are therefore retracting the article. We thank Drs Adam Wolf and William Anderegg for bringing this issue to our attention.
Adam Wolf, a biologist at Princeton University, sent us some background on what happened:
The short story is that we had done some research on a fairly similar topic, and were rejected from the same journal earlier in the year even though we had put the data through alot of careful checks to make sure of our analysis. We were annoyed to see they had submitted something basically the same, but with a more dramatic finding, and had gotten published in that journal. Looking at the supplementary material I realized there is no way they could have gotten their results without some flaw in the code. Basically: they found that climate forces species to shift uphill, but human sampling of plants in their data had also shifted uphill, so the two are confounded. They claimed that they removed this source of bias, but it was evident from their supplementary figures that they hadn’t. The editors brokered a deal where they shared their data with us, and I re-did their analysis and found a different result. Next thing I found out is they decided to retract completely. They said it was the first time that journal had ever issued a retraction!
We asked him if we thought the original conclusions, that species move downhill in response to global warming, made sense:
It’s possible for species to move all sorts of directions, based on all sorts of drivers. Invasions, acid rain and ozone, land use change (forestry and ag, urbanization), and others. Our conclusion is there is some evidence for some species to move uphill, plenty of species not moving uphill (especially endemics with restricted range), many species moving up hill but probably not because of climate change, and generally so much ambiguity in the data that it’s hard to detect or attribute specific observations to specific mechanisms.
We’ve reached out to Harsch, Anderegg, and the journal, and will update if we hear back.
The news article in Science about the research now includes a note saying the study has been retracted.
Update 11:30 a.m. EST 4/13/15: Melanie Harsch emailed us the following statement, and asked that we attribute it to both her and co-author Janneke Hille Ris Lambers:
We retracted the paper after identifying a coding error in the null model analysis that invalidated one of the major conclusions of the paper. We first became aware that there might be a problem with the analysis when we received a draft of the response letter by Wolf and Anderegg (to whom we had sent the data to analyze upon request). We then tried to replicate their methodology to determine why there was a discrepancy. We were dismayed to find an error in our code and to discover that this error was responsible for our counterintuitive prediction that the majority of flora across western North America are moving to lower elevations. At this point, we immediately contacted the journal and worked with the editors to arrange retraction. We are currently reanalyzing the data and will resubmit a revised manuscript. We will make the data and code publicly available.
Hat tip Rolf Degen
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post.
In stories like this, it is far more useful to explain WHAT SPECIFICALLY the “coding error” was. This is not a statistical problem, likely. It is more likely to be a data management problem. Data management is a huge and underemphasized problem. It is at the heart of so many of the recent academic and research fraud scandals. Would it be possible to state EXACTLY WHAT kind of “coding error” occurred?
My guess is that it was an error in the R code (or whatever) that they used to do the statistical analysis. That seems pretty obvious from “coding error”.
Yes, it was presumably an error in the R code. And BTW, there’s no question of fraud or scandal here, so I don’t see why you even raise those terms in passing. The authors were careful, and unfortunately an honest mistake got through. It happens. When the mistake was discovered, the authors quickly retracted the paper.
I am very surprised by the comments that mention the coding error to be in R, and that this would not be a big deal, or a question of fraud or scandal.
As the write-up says, the authors presented work that showed the opposite of the majority of current literature on the subject. And if the conclusion is based on statistical analysis, I would most certainly expect the authors to very diligent when performing this analysis. If the results of the analysis was surprising (which it should have been) checking code should be the first thing to do. and it’s really not that difficult.
To me, there are mistakes on sides of the authors as well as the journal.
The authors should have run the statistical analysis with a known training given the surprising results of their own data.
The journal and reviewers had the code available to them (it is requirement for manuscript submission) but apparently, and sadly not surprisingly, this was not verified.
The authors said themselves that they were surprised by their result and so double-checked all their code. The authors here did *exactly* what you say they should’ve–why aren’t you praising them?
Neither I nor other commenters said the coding error isn’t a big deal. It’s absolutely a big deal in the sense that it’s an error that greatly affected the results–which is why, as soon as the error was discovered, the authors did the right thing and retracted the paper. For which they ought to be praised.
These authors are well-trained. They know what they’re doing. People who know what they’re doing and who take exactly the precautions everyone on this blog thinks they should take *still* make honest mistakes every once in a while.
When one gives inanimate objects human powers, this is termed personification. Trees, shrubs and other plants don’t just uproot and trek south, north or any other direction. Humans migrate. Birds migrate. Fish migrate. But not plants. So, IMO, the concet of “migrating” plants is a myth. Seeds get dispersed, or pollen gets transferred. Humans may transport plants, but plants don’t just migrate. Their goegraphic distribution may change, but populations don’t just walk around. They die off and they reestablish in more favorable conditions. This personification of plants reminds me of a recent case highlighted at PubPeer:
https://pubpeer.com/publications/FF697CF8CD514E7A437EBC24D245AD#fb23537
The authors are using standard terminology in the field. Every field has jargon. Their use of the term “migration” doesn’t cause any confusion for anyone the least bit familiar with the field.
Maybe it doesn’t cause confusion exactly, but I argue that the use of “migration” to describe changes in geographic distribution (e.g. upslope, or poleward shifts) is not ideal. Are we to refer to all changes in species distributions as migration? Furthermore, I study birds that undergo seasonal elevational migration AND whose geographic ranges are shifting upslope due to climate change. We can’t very well refer to both of those phenomena as elevational migration (as is currently done) and expect anyone to not get confused!
Um, if it doesn’t cause any confusion, then it doesn’t cause any confusion. And if jargon that causes no confusion when used with plants would cause confusion if used with birds, well, so what? There are only so many words in the world. They get used with different meaning in different contexts. And the same meaning gets conveyed with different words in different contexts.
Still waiting for someone to come up with a *good* excuse to criticize these authors…
You’re just arguing semantics. These researchers are obviously not suggesting that plants get up and walk around.
When I was a botany undergraduate, we learned that “plants are pretty much like animals except that they don’t move”. Glad to see you reinventing 35 year-old undergraduate humour!
I would be curious to hear the authors’ opinions about crops and wild plants within California as it faces a veritable second “Grapes of Wrath” situation:
http://www.vox.com/2015/4/10/8379221/california-drought-water-crisis
One of the issues dogging science at the moment is the lack of reproducibility of a lot work that is being published. This is not necessarily because the work isn’t reproducible; rather, it is because the authors do not provide their raw data for others to process. The failing in the system is illustrated by this sentence,
“The editors brokered a deal where they shared their data with us,”
It should not be necessary to broker any deals, the raw data should be available for all.
Science can’t be self correcting if there isn’t any access to the data to correct.
Very good point. If Global Change Biology (GCB) required the data to be open to begin with, then brokering the deal with the editors just to access the data would be unnecessary. If not for Adam Wolf persistence, who brokered the deal, this GCB paper would have likely stayed intact and no “coding error” would have been discovered.
Even better, if GCB made open the refereeing process that resulted in the rejection of Wolf’s paper but the acceptance of Harsch’s and Lambers’
We retracted the paper after identifying a coding error in the null model analysis that invalidated one of the major conclusions of the paper. We first became aware that there might be a problem with the analysis when we received a draft of the response letter by Wolf and Anderegg (to whom we had sent the data to analyze upon request). We then tried to replicate their methodology to determine why there was a discrepancy. We were dismayed to find an error in our code and to discover that this error was responsible for our counterintuitive prediction that the majority of flora across western North America are moving to lower elevations. At this point, we immediately contacted the journal and worked with the editors to arrange retraction. We are currently reanalyzing the data and will resubmit a revised manuscript. We will make the data and code publicly available.
People make mistakes and I think it’s great that the authors are willing to make comments on their research. Plant migration is so cool…
Peer review is important and I can think of a dozen accepted scientific facts that took decades for mainstream academia to agree to and dozens that took decades if not centuries to disprove.
104 years ago Rutherford’s atomic model (although basically wrong) became “our” science. It blows me away how much “science” has happened since. Basically everything in our modern world was created in the last 100 years. Have faith that science will find a way… Sure hope I get to see more of it.
🙂