Two researchers at the University of Saskatchewan in Canada have retracted a paper that came to fairly common-sense conclusions about bike safety.
In the September 2014 issue of the Journal of Transport and Health, the authors concluded that slippery road surfaces, night-time biking, and higher speed limits were all associated with higher probabilities of a bicycle accident.
Despite these logical conclusions, the authors discovered a statistical error that “would significantly change the discussion,” according to the retraction notice.
It’s unclear what part of the paper was affected by the statistics problem.
Here are the highlights at the top of the abstract:
Slipper road surfaces with gravel, sand, oil, and ice were associated with bicycle collisions.
Cycling at night increased the odds of collision when compared to cycling in the day.
Vehicle speed limits over 70 km/h increased the odds of bicycle collision when compared to speed limits of less than 60 km/h.
Intersections were not associated with an increased odds of bicycle collision when compared to other road structures.
Here’s the notice for “Analyzing road surface conditions, collision time, and road structural factors associated with bicycle collisions from 2000 to 2010 in Saskatoon, Saskatchewan” (no citations, according to Google Scholar):
This article has been retracted: please see Elsevier Policy on Article Withdrawal (http://www.elsevier.com/locate/withdrawalpolicy).
This article has been retracted at the request of the authors.
In the original paper the reference category for the logistic regression was inadvertently reversed. The reversal of the reference category affects the interpretation of all regression coefficients reported in the paper and would significantly change the discussion. The authors regret the inconvenience caused by this error.
We emailed the authors and journal editor, and will update if we hear back.
Hat tip: Rolf Degen
Update 2/26/2015 2 p.m. EST: Editor Jenny Mindell emailed us with more details:
The authors inadvertently made a simple but fundamental mathematical error.
This inverted the results to produce conclusions that were plausible but wrong.
As soon as they realized, one of the authors alerted the journal, which followed COPE guidelines and retracted the article.
Update 3/20/15 3:44 p.m. EST: Author Daniel Fuller sent us more information:
We contacted the journal to pull the paper when a colleague identified a potential coding error in the statistical analysis. We confirmed that there was an error in the code and contacted the journal editor. The editor asked us to resubmit the paper with the changes but thought the results were too different to publish a correction.We have resubmitted the corrected paper to the same journal as part of a special issue.
This unfortunately seems to be a major error, completely invalidating the conclusions. The retraction statement “The reversal of the reference category affects the interpretation of all regression coefficients…” seems to imply that all conclusions should be reversed as well and should read something like:
– Slippery Road surfaces decrease bicycle crashes.
– Cycling at night is safer than cycling during day time.
– Lower speeds lead to more bicycle collisions.
– Bicycle collisions tend to occur more at intersections as compared to other road structures.
These ‘reversed’ conclusions appear to be supported by the raw data in Table 1 of the retracted article.
In fact all these conclusions are completely meaningless (with the possible exception of the last one), as they do not include the number of cyclists that are on the road under these conditions.
Perhaps because of the “common sense” of their original conclusions, the authors failed to check whether these conclusions made “actual sense” with the data…
Sucks that they made the mistake, but kudos to them for issuing an immediate retraction. To err is human, to cover up a critical error, on the other hand, is wrong. And I imagine the authors will be ensuring all stats work is double checked before they publish again.