About these ads

Retraction Watch

Tracking retractions as a window into the scientific process

High school whiz kid retracts PLoS ONE herd immunity paper

with 19 comments

Georgette speaking at the Davidson Institute’s award ceremony, Library of Congress, 2008

It’s pretty impressive to publish two peer-reviewed papers on complicated vaccination models while you’re still in high school. So it’s not surprising that Nathan Georgette, who grew up outside of Jacksonville, Florida, earned a prestigious fellowship from the Davidson Institute for Talent Development.

But perhaps even more impressive is realizing you’ve made a fundamental error in one of those studies, and retracting it while you’re still a college senior at Harvard.

Georgette’s first paper, published in the Internet Journal of Epidemiology in 2007, the year he turned 16, was titled “The Quantification Of The Effects Of Changes In Population Parameters On The Herd Immunity Threshold.” Here’s the abstract:

The purpose was to develop a novel population parameter based (PPB) equation for the herd immunity threshold that incorporates the effects of population dynamics and immunization on the infectiousness of a disease and to analyze these effects. Previous research has not attempted this specific method. The researcher sought to improve cost effectiveness of outbreak response in resource-poor areas. This was achieved by solving for a PPB equation for the basic reproductive number and developing the threshold equation. The researcher applied this equation to three actual measles outbreaks. The PPB equation demonstrates that, using data from the 2003 Marshall Islands measles outbreak, gradual immunization decreases both the effective and the basic reproductive numbers when compared to pulse immunization (from 3.48 to 3.05 and 18.38 to 17.47 respectively). This decreases the potency of the outbreak, thus reducing the associated morbidities, mortalities, and costs.

The paper, Georgette tells Retraction Watch, was “a key part” of his submission to the Davidson Fellowships program.

So how did a 16-year-old kid in Florida develop an interest in outbreaks in poor countries? In 2009, in an interview about being valedictorian of his high school class, he told the Florida Times-Union:

I’ve always had an interest in global issues. In 10th grade, I found on the Internet that hundreds of thousands of children are dying from these diseases that can be cured for $1. … I wanted to see if I could learn more about it. I thought maybe I could somehow help in some small way.

Two years after publishing that paper, and having already applied for the Davidson fellowship, Georgette — now a high school senior — published another one, this time in PLoS ONE, in January 2009. Here’s the abstract of “Predicting the Herd Immunity Threshold during an Outbreak: A Recursive Approach:

Background

The objective was to develop a novel algorithm that can predict, based on field survey data, the minimum vaccination coverage required to reduce the mean number of infections per infectious individual to less than one (the Outbreak Response Immunization Threshold or ORIT) from up to six days in the advance.

Methodology/Principal Findings

First, the relationship between the rate of immunization and the ORIT was analyzed to establish a link. This relationship served as the basis for the development of a recursive algorithm that predicts the ORIT using survey data from two consecutive days. The algorithm was tested using data from two actual measles outbreaks. The prediction day difference (PDD) was defined as the number of days between the second day of data input and the day of the prediction. The effects of different PDDs on the prediction error were analyzed, and it was found that a PDD of 5 minimized the error in the prediction. In addition, I developed a model demonstrating the relationship between changes in the vaccination coverage and changes in the individual reproduction number.

Conclusions/Significance

The predictive algorithm for the ORIT generates a viable prediction of the minimum number of vaccines required to stop an outbreak in real time. With this knowledge, the outbreak control agency may plan to expend the lowest amount of funds required stop an outbreak, allowing the diversion of the funds saved to other areas of medical need.

About a month after the paper went online, Georgette realized there were typos in two equations, so the journal ran a correction.

Fast forward to this past summer, by which point the paper had been cited once. Georgette tells us:

…I had time to perform an in-depth (and more enlightened) review of my high school publications following a course in Ordinary and Partial Differential Equations. Once I had concluded that the PLoS ONE paper’s flaw was fundamental (in Aug 2012), I contacted the editorial board of the journal.  From Aug to Oct, I was in correspondence with the PLoS ONE board to produce and finalize the retraction.

The retraction, Georgette notes, was unrelated to the earlier corrections. Here’s what happened. The Internet Journal of Epidemiology paper, he says,

…made an assumption that I did not recognize at the time.  Any mathematical model of a natural phenomenon makes assumptions; these assumptions become errors when they are unknowingly violated (as is what happened with the PLoS ONE paper).  The Internet J Epi paper didn’t violate its own assumption in its mathematical development and implementation.  That said, I am planning to correct the Discussion section of the Internet J Epi paper to clarify the scope and possible applications of the model developed therein.

Here’s the retraction notice:

The author wishes to retract this publication due to a mathematical flaw that undermines the article’s methods and conclusions.

This paper builds upon the author’s previous publication in Internet J Epidemiology [1]. In that previous paper, the author derived a mathematical model for the basic reproduction number from a modified SIR system of differential equations. During this derivation, an integration step was performed that implicitly assumed the per-susceptible-person rate of immunization, ρ, is constant for a given epidemic. At the time, the author did not recognize the existence of this assumption and unknowingly violated it while building upon [1] to develop this current paper: the author founded this work on a description of ρ as changing through time. This violation of an underlying assumption renders invalid the results of the current paper.

The author apologizes to the readers and editors of PLOS ONE for this error and the delay in its recognition.

[1] Georgette N (2007) The Quantification Of The Effects Of Changes In Population Parameters On The Herd Immunity Threshold. Internet J Epidemiology 5(1).

Note that this fundamental flaw wasn’t caught by peer reviewers.

In 2009, Georgette told the Florida Times-Union:

I have three main ideas: a medical researcher for infectious diseases, practice medicine in the field or go into human rights law with a focus on health care. I believe everyone in the world should have equal access to health care.

Based on the kind of rigor and transparency with which he handled these errors, we’d say he has a good chance of success in any of those fields.

About these ads

Written by Ivan Oransky

October 19, 2012 at 4:22 pm

19 Responses

Subscribe to comments with RSS.

  1. Good on Nathan. Really smart kid. Published papers young. Later on, recognised errors in his own published work, realised he wasn’t as smart as first thought. Did the right thing – initiated correction of errors and retraction of own paper. Not many killed. Nathan comes out with big brain and personal and scientific integrity all intact. Good on you, Nathan. If only others followed your honourable example. Too bad about the disingenuous efforts at the University of Sydney with the high-profile but bogus Australian Paradox papers: http://www.australianparadox.com/pdf/AUSTRALIAN-PARADOX-101-SLIDESHOW.pdf

    roryrobertsonformerfattie

    October 20, 2012 at 4:04 am

  2. Good on him.

    • I still think he should be grounded for a week…

      chirality

      October 20, 2012 at 8:40 am

  3. So nobody is concerned that he wins a fellowship on the back of a now retracted paper? What about all the other kids who missed out because of this “whizz kid”?

    Dave

    October 20, 2012 at 12:31 pm

    • The first (2007) paper, which was the basis for the fellowship application, was not retracted. From what I’ve understood, the post-application PLoS ONE article is retracted because math used in that paper is not applicable to problem.

      Sylvain Bernès

      October 20, 2012 at 1:02 pm

  4. PLoS One again…it seems to me that there is an increase in the number of PLoS ONe papers getting retracted these days.
    Agree with Dave on the point he has mentioned above.

    Ressci Integrity

    October 20, 2012 at 7:09 pm

    • Completely disagree with concerns about Mr. Georgette. The error that he made, although simple, is not obvious: it is the sort of problem that could also easily afflict a better trained researcher, and could easily be overlooked by reviewers. In fact, you could say that the error he made is emblematic of the general problem of theoretical work — so easy for a typo or a simple math error to creep into your elegant structure of reasoning, and to remain undetected if you don’t have experimental cross-checks of some sort.

      So I say: let us salute Mr. Georgette for his integrity in calling himself out for the error, rather than simply letting it be buried by time. Moreover, even had the other paper not existed, I think the scholarship folks should be proud they’re funding him: you want your recipient to be somebody with drive and initiative who will benefit greatly from scientific training, and this certainly appears to be the case with Mr. Georgette.

      Reviewer #3

      October 21, 2012 at 9:53 am

      • If indeed he realized his error after taking “ordinary differential equations 101″, then no “better trained researcher” would be likely to commit such elementary mathematical error. This shows the problem in general with “whizz kids” doing theoretical work. It takes time, practice, and lot of classes to acquire and master those skills. Then, if you don’t have the proper mathematical training and you send to low-rung, broad-spectrum pay-to-publish journals like this, it’s easy to screw up the math and get it past the reviewers. I’m doubtful this would’ve passed peer review at the of the major mathematical biology journals.

        Anonymous

        October 21, 2012 at 1:08 pm

      • to Anonymous:
        >I’m doubtful this would’ve passed peer review at the of the major mathematical biology journals.

        Actually, it’s pretty likely that it would. I’ve seen much more serious errors slip by the reviewers at such journals. As long those are honest errors (& immediately corrected/acknowledged after being discovered), this is completely ok – just a way the science is done.

        I think you (& several others) are way too harsh on this kid. He showed a lot more integrity here than many many well regarded professional researchers. Please back off.

        yet another reviewer

        October 21, 2012 at 3:35 pm

    • Calling the mistake “difficult to spot” is disingenuous. If you assume some quantity to be constant in order to make solving a differential equation easier and a few lines down the road you claim this quantity to change, this is just careless to the extreme. Given the author’s age, there is no need to bash his proverbial head against the wall for this, though. But both the editors and reviewers do have some explaining to do. The only thing I find questionable is the common practice of using eloquence to misrepresent the true nature of a problem. Very frequently, fraudsters, as apparent from many retraction notices published on RW, use eloquence in an attempt to deceive the public as to the true nature of their deeds. In this particular case there is no fraud involved, but still the eloquence trick is used nonetheless.

      chirality

      October 22, 2012 at 4:01 am

      • Nothing personal against this author’s age, but why do we need to have minimum age for many aspects in life drinking alcohol, driving etc – but not for publishing a research article in a peer-reviewed journal. Experience doesn’t have any value in science. Whiz kids…

        minimum age

        October 22, 2012 at 7:26 am

      • > If you assume some quantity to be constant in order to make solving a differential equation
        > easier and a few lines down the road you claim this quantity to change, this is just careless
        > to the extreme.

        You clearly did not actually look at the paper.

        Would you care to point out exactly where you believe the obvious error has occurred, and say how long it took you to find it?

        Reviewer #3

        October 23, 2012 at 12:20 pm

    • I suspect it just looks that way given the sheer volume of papers they’re publishing these days. My suspicion would be that their ‘retraction index’ hasn’t changed, although you’d have to crunch the numbers obviously.

      Booker

      October 22, 2012 at 6:22 pm

  5. Young Mr Georgette’s is a tremendous and affirming case to share, thank you for the effort behind its writing.

    Its presentation reminds me of a structural question for you and your colleagues at RW, regarding the top level page of the RW site:
    Could utility/initial impressions be helped by a slight change to the initially visible summary? In general, the first page might also standardize routinely sought information for those without time to dip into the whole article—e.g., an up-front small html table with the key citation (for immediate recognition by persons in the field), link to the original publication and retraction, a pithy running title, etc.). In the “whiz kid” case, the running title or some other standard element could have made clear at the top level of viewing, the very positive nature of the Georgette case.

    I am aware that the challenge of such changes is in the details and implementation (especially when the article is more complex or non-standard in character), but wished regardless to express the article affirmation and this general encouragement.

    Please also see a brief comment at the 1 October PNAS “Majority of retractions…” article.

    Prof. Puuttuu Nykiminen-Polvi

    October 21, 2012 at 9:27 pm

  6. The problem with PlosOne is that it is very reviewer dependent. The editors do not have the breadth of knowledge to properly review the reviewers. This can make publishing in PlosOne either very easy or extremely difficult, depending on the reviewers the editors choose.

    As for this case, the kid asked to have his own paper retracted, which is far more than most Prof. would ever do. Most senior scientists would defend their bad work till their last breath. Give the kid respect for doing the right thing. The reviewers and editors are at fault for letting the work get published.

    I am sure Nathan learned his lesson. I doubt the adults involved learned anything.

    John

    October 23, 2012 at 5:00 am

  7. PlosOne is crap, that is the sad truth.

    Jon Beckmann

    October 23, 2012 at 7:21 am


We welcome comments. Please read our comments policy at http://retractionwatch.wordpress.com/the-retraction-watch-faq/ and leave your comment below.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 34,510 other followers

%d bloggers like this: