It’s pretty impressive to publish two peer-reviewed papers on complicated vaccination models while you’re still in high school. So it’s not surprising that Nathan Georgette, who grew up outside of Jacksonville, Florida, earned a prestigious fellowship from the Davidson Institute for Talent Development.
But perhaps even more impressive is realizing you’ve made a fundamental error in one of those studies, and retracting it while you’re still a college senior at Harvard.
Georgette’s first paper, published in the Internet Journal of Epidemiology in 2007, the year he turned 16, was titled “The Quantification Of The Effects Of Changes In Population Parameters On The Herd Immunity Threshold.” Here’s the abstract:
The purpose was to develop a novel population parameter based (PPB) equation for the herd immunity threshold that incorporates the effects of population dynamics and immunization on the infectiousness of a disease and to analyze these effects. Previous research has not attempted this specific method. The researcher sought to improve cost effectiveness of outbreak response in resource-poor areas. This was achieved by solving for a PPB equation for the basic reproductive number and developing the threshold equation. The researcher applied this equation to three actual measles outbreaks. The PPB equation demonstrates that, using data from the 2003 Marshall Islands measles outbreak, gradual immunization decreases both the effective and the basic reproductive numbers when compared to pulse immunization (from 3.48 to 3.05 and 18.38 to 17.47 respectively). This decreases the potency of the outbreak, thus reducing the associated morbidities, mortalities, and costs.
The paper, Georgette tells Retraction Watch, was “a key part” of his submission to the Davidson Fellowships program.
So how did a 16-year-old kid in Florida develop an interest in outbreaks in poor countries? In 2009, in an interview about being valedictorian of his high school class, he told the Florida Times-Union:
I’ve always had an interest in global issues. In 10th grade, I found on the Internet that hundreds of thousands of children are dying from these diseases that can be cured for $1. … I wanted to see if I could learn more about it. I thought maybe I could somehow help in some small way.
Two years after publishing that paper, and having already applied for the Davidson fellowship, Georgette — now a high school senior — published another one, this time in PLoS ONE, in January 2009. Here’s the abstract of “Predicting the Herd Immunity Threshold during an Outbreak: A Recursive Approach:”
Background
The objective was to develop a novel algorithm that can predict, based on field survey data, the minimum vaccination coverage required to reduce the mean number of infections per infectious individual to less than one (the Outbreak Response Immunization Threshold or ORIT) from up to six days in the advance.
Methodology/Principal Findings
First, the relationship between the rate of immunization and the ORIT was analyzed to establish a link. This relationship served as the basis for the development of a recursive algorithm that predicts the ORIT using survey data from two consecutive days. The algorithm was tested using data from two actual measles outbreaks. The prediction day difference (PDD) was defined as the number of days between the second day of data input and the day of the prediction. The effects of different PDDs on the prediction error were analyzed, and it was found that a PDD of 5 minimized the error in the prediction. In addition, I developed a model demonstrating the relationship between changes in the vaccination coverage and changes in the individual reproduction number.
Conclusions/Significance
The predictive algorithm for the ORIT generates a viable prediction of the minimum number of vaccines required to stop an outbreak in real time. With this knowledge, the outbreak control agency may plan to expend the lowest amount of funds required stop an outbreak, allowing the diversion of the funds saved to other areas of medical need.
About a month after the paper went online, Georgette realized there were typos in two equations, so the journal ran a correction.
Fast forward to this past summer, by which point the paper had been cited once. Georgette tells us:
…I had time to perform an in-depth (and more enlightened) review of my high school publications following a course in Ordinary and Partial Differential Equations. Once I had concluded that the PLoS ONE paper’s flaw was fundamental (in Aug 2012), I contacted the editorial board of the journal. From Aug to Oct, I was in correspondence with the PLoS ONE board to produce and finalize the retraction.
The retraction, Georgette notes, was unrelated to the earlier corrections. Here’s what happened. The Internet Journal of Epidemiology paper, he says,
…made an assumption that I did not recognize at the time. Any mathematical model of a natural phenomenon makes assumptions; these assumptions become errors when they are unknowingly violated (as is what happened with the PLoS ONE paper). The Internet J Epi paper didn’t violate its own assumption in its mathematical development and implementation. That said, I am planning to correct the Discussion section of the Internet J Epi paper to clarify the scope and possible applications of the model developed therein.
Here’s the retraction notice:
The author wishes to retract this publication due to a mathematical flaw that undermines the article’s methods and conclusions.
This paper builds upon the author’s previous publication in Internet J Epidemiology [1]. In that previous paper, the author derived a mathematical model for the basic reproduction number from a modified SIR system of differential equations. During this derivation, an integration step was performed that implicitly assumed the per-susceptible-person rate of immunization, ρ, is constant for a given epidemic. At the time, the author did not recognize the existence of this assumption and unknowingly violated it while building upon [1] to develop this current paper: the author founded this work on a description of ρ as changing through time. This violation of an underlying assumption renders invalid the results of the current paper.
The author apologizes to the readers and editors of PLOS ONE for this error and the delay in its recognition.
[1] Georgette N (2007) The Quantification Of The Effects Of Changes In Population Parameters On The Herd Immunity Threshold. Internet J Epidemiology 5(1).
Note that this fundamental flaw wasn’t caught by peer reviewers.
In 2009, Georgette told the Florida Times-Union:
I have three main ideas: a medical researcher for infectious diseases, practice medicine in the field or go into human rights law with a focus on health care. I believe everyone in the world should have equal access to health care.
Based on the kind of rigor and transparency with which he handled these errors, we’d say he has a good chance of success in any of those fields.
Good on Nathan. Really smart kid. Published papers young. Later on, recognised errors in his own published work, realised he wasn’t as smart as first thought. Did the right thing – initiated correction of errors and retraction of own paper. Not many killed. Nathan comes out with big brain and personal and scientific integrity all intact. Good on you, Nathan. If only others followed your honourable example. Too bad about the disingenuous efforts at the University of Sydney with the high-profile but bogus Australian Paradox papers: http://www.australianparadox.com/pdf/AUSTRALIAN-PARADOX-101-SLIDESHOW.pdf
Good on him.
I still think he should be grounded for a week…
So nobody is concerned that he wins a fellowship on the back of a now retracted paper? What about all the other kids who missed out because of this “whizz kid”?
The first (2007) paper, which was the basis for the fellowship application, was not retracted. From what I’ve understood, the post-application PLoS ONE article is retracted because math used in that paper is not applicable to problem.
PLoS One again…it seems to me that there is an increase in the number of PLoS ONe papers getting retracted these days.
Agree with Dave on the point he has mentioned above.
Completely disagree with concerns about Mr. Georgette. The error that he made, although simple, is not obvious: it is the sort of problem that could also easily afflict a better trained researcher, and could easily be overlooked by reviewers. In fact, you could say that the error he made is emblematic of the general problem of theoretical work — so easy for a typo or a simple math error to creep into your elegant structure of reasoning, and to remain undetected if you don’t have experimental cross-checks of some sort.
So I say: let us salute Mr. Georgette for his integrity in calling himself out for the error, rather than simply letting it be buried by time. Moreover, even had the other paper not existed, I think the scholarship folks should be proud they’re funding him: you want your recipient to be somebody with drive and initiative who will benefit greatly from scientific training, and this certainly appears to be the case with Mr. Georgette.
If indeed he realized his error after taking “ordinary differential equations 101”, then no “better trained researcher” would be likely to commit such elementary mathematical error. This shows the problem in general with “whizz kids” doing theoretical work. It takes time, practice, and lot of classes to acquire and master those skills. Then, if you don’t have the proper mathematical training and you send to low-rung, broad-spectrum pay-to-publish journals like this, it’s easy to screw up the math and get it past the reviewers. I’m doubtful this would’ve passed peer review at the of the major mathematical biology journals.
to Anonymous:
>I’m doubtful this would’ve passed peer review at the of the major mathematical biology journals.
Actually, it’s pretty likely that it would. I’ve seen much more serious errors slip by the reviewers at such journals. As long those are honest errors (& immediately corrected/acknowledged after being discovered), this is completely ok – just a way the science is done.
I think you (& several others) are way too harsh on this kid. He showed a lot more integrity here than many many well regarded professional researchers. Please back off.
Calling the mistake “difficult to spot” is disingenuous. If you assume some quantity to be constant in order to make solving a differential equation easier and a few lines down the road you claim this quantity to change, this is just careless to the extreme. Given the author’s age, there is no need to bash his proverbial head against the wall for this, though. But both the editors and reviewers do have some explaining to do. The only thing I find questionable is the common practice of using eloquence to misrepresent the true nature of a problem. Very frequently, fraudsters, as apparent from many retraction notices published on RW, use eloquence in an attempt to deceive the public as to the true nature of their deeds. In this particular case there is no fraud involved, but still the eloquence trick is used nonetheless.
Nothing personal against this author’s age, but why do we need to have minimum age for many aspects in life drinking alcohol, driving etc – but not for publishing a research article in a peer-reviewed journal. Experience doesn’t have any value in science. Whiz kids…
> If you assume some quantity to be constant in order to make solving a differential equation
> easier and a few lines down the road you claim this quantity to change, this is just careless
> to the extreme.
You clearly did not actually look at the paper.
Would you care to point out exactly where you believe the obvious error has occurred, and say how long it took you to find it?
I suspect it just looks that way given the sheer volume of papers they’re publishing these days. My suspicion would be that their ‘retraction index’ hasn’t changed, although you’d have to crunch the numbers obviously.
Young Mr Georgette’s is a tremendous and affirming case to share, thank you for the effort behind its writing.
Its presentation reminds me of a structural question for you and your colleagues at RW, regarding the top level page of the RW site:
Could utility/initial impressions be helped by a slight change to the initially visible summary? In general, the first page might also standardize routinely sought information for those without time to dip into the whole article—e.g., an up-front small html table with the key citation (for immediate recognition by persons in the field), link to the original publication and retraction, a pithy running title, etc.). In the “whiz kid” case, the running title or some other standard element could have made clear at the top level of viewing, the very positive nature of the Georgette case.
I am aware that the challenge of such changes is in the details and implementation (especially when the article is more complex or non-standard in character), but wished regardless to express the article affirmation and this general encouragement.
Please also see a brief comment at the 1 October PNAS “Majority of retractions…” article.
The problem with PlosOne is that it is very reviewer dependent. The editors do not have the breadth of knowledge to properly review the reviewers. This can make publishing in PlosOne either very easy or extremely difficult, depending on the reviewers the editors choose.
As for this case, the kid asked to have his own paper retracted, which is far more than most Prof. would ever do. Most senior scientists would defend their bad work till their last breath. Give the kid respect for doing the right thing. The reviewers and editors are at fault for letting the work get published.
I am sure Nathan learned his lesson. I doubt the adults involved learned anything.
“Most senior scientists would defend their bad work till their last breath.” That does not mean they would do so in bad faith (you seem to imply they would). There is always ambiguity, and there is nothing wrong for people to defend their position, while more conclusive evidence is collected.
Actually, Jon, there is NOT always ambiguity. Or at least I challenge you to find ambiguity in the basic but eye-popping errors in the high-profile Australian Paradox paper, at Slides 9 and 10 at http://www.australianparadox.com/pdf/AUSTRALIAN-PARADOX-101-SLIDESHOW.pdf
I don’t know about you but for me it’s hard to imagine that this is the work of professional scientists, let alone two of the University of Sydney’s best and brightest.
Some have wondered whether the fact that the lead author and the “Guest Editor” of the publishing journal are the same person (!) might have something to do with how the “deeply flawed” paper was published in 2011 in the first place?
And then, extraordinarily, confirmed as fault-free in the Australian Paradox Revisited paper in 2012.
The latest fascinating development is that Australia’s sugar industry has funded the production of a shall-we-say-unconvincing sugar series that – I was shocked, shocked, to see – was used to try to support the Australian Paradox paper – with its unlikely but twice peer-reviewed conclusion of “an inverse relationship” between sugar consumption and obesity! – from correction or retraction: http://www.australianparadox.com/pdf/New-nonsense-based-sugarreport.pdf
Indeed, barely five minutes after this “new” series was produced, the Deputy Chairman of the University of Sydney’s Nutrition Research Foundation turned up to claim “vindication” for his underperforming colleagues, and victory for the University of Sydney in the Great Australian Paradox Dispute of 2012: http://www.australianparadox.com/pdf/university-sydney-falsely-declares-victory.pdf
Champion stuff. The whole episode would be quite a hoot for everyone, except for the fact that scientists I think still are not allowed to misrepresent the available facts in formal scientific papers?
The interesting question now is whether the repeated formal misrepresentation of the available facts on added sugar in the development of obesity was the result of persistent negligence or basic scientific fraud? (Slides 8-10, 17 and 36-44 ).
Did I mention that the University’s still-undisclosed conflict of interest in running a low-GI food-stamping enterprise while promoting the (scientifically unlikely but) strong public view that super-low GI (19) fructose – the “sweet poison” half of added sugar – is harmless adds another unsettling element to the Australian Paradox fiasco: http://www.australianparadox.com/pdf/Sydney-Uni-conflict-interest-030712.pdf
Especially when a growing nucleus of global scientists think that added sugar/fructose is a key driver or global obesity and diabetes: http://www.nytimes.com/2011/04/17/magazine/mag-17Sugar-t.html?pagewanted=all&_r=0 and http://well.blogs.nytimes.com/2012/10/22/in-fight-against-obesity-drink-sizes-matter/
If you are health-conscious, then perhaps check out the sugary food products that the University of Sydney for a fee endorses as “healthy”, on page 10 of http://www.gisymbol.com/cmsAdmin/uploads/Glycemic-Index-Foundation-Healthy-Choices-Brochure.pdf
The University of Sydney’s pricing structure is outlined here: http://www.foodhealthdialogue.gov.au/internet/foodandhealth/publishing.nsf/Content/D59B2C8391006638CA2578E600834BBD/$File/Resources%20and%20support%20for%20reformulation%20activities.pdf
Readers, I’d be interested to hear your views on whether or not the University of Sydney’s bogus Australian Paradox papers should be corrected or retracted?
Maybe I’ve lost the plot on this topic. Does anyone have a view on how this episode might all fit together without involving anything untoward?
Finally, the paper itself has been used to spearhead the food industry’s campaign for the Australian Government’s revised national nutrition guidelines NOT to toughen official advice against added sugar: http://www.smh.com.au/national/health/research-causes-stir-over-sugars-role-in-obesity-20120330-1w3e5.html
PlosOne is crap, that is the sad truth.
Damn right it is.