A retraction from a high-profile group uncovered a technical limitation involving a widely used reagent.
Some quick background: the sequence hypothesis central hypothesis dogma of biology states that DNA gets transcribed to RNA that gets translated into proteins. Some RNAs, however, don’t code for proteins, but instead help to regulate gene expression. These microRNAs are tiny in size, but can regulate gene expression across animal and plant kingdoms.
In September 2011, the Molecular Cell published an entire issue with regulatory RNA as its theme. V. Narry Kim, associate professor at Seoul National University and high-profile microRNA researcher contributed a study that her group has now retracted just months later on June 29.
The problem? A reagent used to purify miRNAs favors some miRNAs and fails to isolate those rich low in guanine and cytosine — two of the four building blocks of RNA — or those with few secondary folding structures.
In this paper, we reported that the levels of specific microRNAs (miRNAs) decrease when cells are grown at low density or when cells are detached from a culture dish. Based on our results, we proposed that some miRNAs are selectively destabilized depending on the adhesion status of the cells. However, in subsequent studies, we discovered that structured miRNAs with low GC content, such as miR-141, miR-29b, miR-21, miR-106b, miR-15a, and miR-34a, are selectively lost during sample preparation rather than degraded in the cell. The small RNA loss occurs when a small number of cells is used for RNA preparation using the standard TRIzol protocol (see Kim et al. [pp. 893–895] in this issue for details). These findings provide an alternative explanation for our original data. While the original data are all reproducible, we are retracting the paper because we feel the main conclusions have been compromised. We apologize for any inconvenience this may have caused.
The group took their observation that led to the retraction one step further and conducted experiments to show how different reagents will isolate different populations of RNAs from small sample sizes. This is important since research before assumed that protocols isolated all kinds of RNA equally.
Along with the retraction notice, the group published a three-page letter to the editor with one figure that concludes:
Our current analyses indicate that structured small RNAs with low GC content are recovered inefficiently when a small number of cells is used for RNA isolation with TRIzol.
A spokesperson for Cell, the publisher of Molecular Cell, declined to comment. And we haven’t had a response from Life Technologies, which makes TRIzol. (Life Technologies, we should note, was the company that last year asked two of its employees to retract a paper because they’d submitted it without approval.)
Kim, however, was very forthcoming. She wrote us this email in response to our questions:
In brief, we found after publishing the original paper that our results were due to a technical problem associated with widely used Trizol reagent. When we used a different protocol (which is infrequently used in the field), we did not observe the same phenomenon. It turned out that small RNAs with low GC contents and/or secondary structure are lost from RNA purification when a small number of cells are used (miR-141 was a typical case). This is not due to a biological regulation, but a technical problem associated with Trizol reagent.
Our original data are reproducible regardless of scale. But we decided to retract the paper because the conclusion of the paper was mistaken. We interpreted the results based on an assumption that Trizol extracts all sorts of RNAs with equal yield. In fact, Trizol reagent has been used for decades to extract “total RNA” and has been the standard RNA extraction reagent. No one in the field has realized this problem before. My group found this problem from our own experiments without a help or notification from other groups.
After finding this problem, I contacted John Pham, an editor of Molecular Cell, who had handled our original paper. John was very surprised to hear the news but helped us enthusiastically in the process of retraction and publication of another article.
The new article was published as a Letter to the editor format, which is a type of peer-reviewed paper. The reviewers commended our new findings and were highly supportive of publishing the new article…I also received a number of supportive emails from scientists in the field regarding the publication of retraction.
Kim also had some thoughts on how retractions should be handled, and explained how this situation was very different from some other recent cases involving Seoul National University:
Although the whole process of finding the mistake and publishing a retraction was painful to my group, I strongly believe that scientists should not be afraid of retracting their own papers. Scientists inevitably live with a number of assumptions that may turn out to be wrong later. We also have a number of technical limitations which are unrecognized at the time of the study. So, making a mistaken conclusion is sometimes unavoidable. What is important is to be honest and to correct the mistake as soon as we realize it. I am very proud of my people who were smart enough to find and dissect the problem and brave enough to carry out additional experiments and retract the paper.
In this particular case, we and the reviewers felt that sharing the information is important because the technical issue is a general one and affects not only ours but also potentially many other people’s work. So we decided to publish the new article to warn the colleagues in the field.
This incident does not affect my research program. The particular project was just a part of my group’s research subjects. We continuously work on microRNAs to understand their regulation and function.
The SNU ethics committee was not involved in the retraction process at all. I did not notify them because there is no ethical violation here. My people and I have kept the highest standard of scientific integrity. This should be clearly distinguished from scientific misconduct. I believe that research community should respect and encourage honest retractions, instead of automatically associating retraction with misconduct.
We couldn’t agree more. Kudos to Kim for a transparent move that corrects the literature.
Reblogged this on .
Great!
The use of a retraction to correct an error in interpretation troubles me. As the authors themselves say “the original data are all reproducible”. If we come to expect articles to be retracted in all such cases, then we are dangerously reliant on the original authors for ongoing quality control of what is printed. Instead, we need to recognize that the scientific literature is littered with researchers who have reached the wrong conclusion based on valid data. And we should strive to build better infrastructure to track when some point of inference in an article is disputed by a later publication, no matter who does the disputiung. It just so happens in this case that the authors disagreed with themselves in a later letter.
“The original data are all reproducible” but, as the author explained, they are not reliable. Consequently, all the key conclusions based on these data were erroneous. It must have been very upsetting for the authors to discover the unexpected properties of the Trizol regent but to their credit they acted swiftly to fix the scientific record.
I think there is a small error in the text when it says that it “fails to isolate those rich in guanine and cytosine.” Later the letter to the editor is quoted as saying: “It turned out that small RNAs with low GC contents and/or secondary structure are lost from RNA purification when a small number of cells are used (miR-141 was a typical case).”
Fixed — thanks for pointing that out!
From this retraction vs other recent retractions, it appears that there is an inverse relationship between the amount of information available about a retraction and the culpability of the retractor.
I what to express my appreciation of Dr. Kim. The best scientists can do is to say exactly what they did, and to be sure the results of what they did are reproducible. After that, interpretations and conclusions often turn out to be wrong- if only because other relevant data appears. I think that a retraction is a good move because it is
noticed. There must be a lot of people out there checking on Trizol.
“the central hypothesis of biology states that DNA gets transcribed to RNA that gets translated into proteins”
That is incorrect. What you stated was the sequence hypothesis. “The Sequence Hypothesis. This has already been referred to a number of times. In its simplest form it assumes that the specificity of a piece of nucleic acid is expressed solely by the sequence of its bases, and that this sequence is a (simple) code for the amino acid sequence of a particular protein.”
“The Central Dogma. This states that once “information” has passed into protein it cannot get out again. In more detail, the transfer of information from nucleic acid to nucleic acid, or from nucleic acid to protein may be possible, but transfer from protein to protein, or from protein to nucleic acid is impossible. Information means here the precise determination of sequence, either of bases in the nucleic acid or of amino acid residues in the protein.”
Both quotes from: Crick, F.H.C. (1958) On protein synthesis. Symp. Soc. Exp. Biol. XII:138-163 and detailed about in a blog post by Larry Moran on Jan. 15, 2007 http://sandwalk.blogspot.com/2007/01/central-dogma-of-molecular-biology.html
Fixed based on Marshall Nirenberg’s phrasing of the central dogma, “DNA makes RNA makes protein.” Thanks for pointing this out. And, apropos of the mixup between “hypothesis” and “dogma,” from Horace Freeland Judson’s “The Eighth Day of Creation:”
Still not the Central Dogma. Nirenberg gets it wrong despite saying its Francis Crick’s Central Dogma. What you are stating is the sequence hypothesis of Crick. It is a positive statement, the amino acid sequence of a protein is encoded in DNA. The Central Dogma is a negative statement, it is a prediction that sequence information does not pass from protein to protein/nucleic acid (proteins are not the template to make new proteins nor nucleic acids, i.e. reverse translation is highly unlikely). Too many papers sell themselves as being a violation of the Central Dogma without understanding what the Central Dogma is. Too many cite Crick without actually reading what he wrote about the Central Dogma and Sequence Hypothesis.
“No one in the field has realized this problem before. My group found this problem from our own experiments without a help or notification from other groups.”
This appears to be one of those rare cases where the paper is retracted but the scientific community is learning something very useful from the researchers involved.
Text changed to refer to Crick’s original term for the DNA-RNA-protein direction of information transfer, the sequence hypothesis. Nirenberg interpreted this as the central dogma, as Ivan noted before.
My thanks (again) to the authors for bringing this problem to the attention of the readers. I suspect many researchers will be concerned, given how many work is done on microRNAs and how commonly Trizol is used as the extraction reagent. If this problem gets worse with smaller cell numbers, I can see problems arise when e.g. microdissected tumor samples are compared to larger pieces of normal tissue, a not unusual situation.
Maybe other people did notice before that there might be something amiss, but it just never was made public…
Noble but incompetent? It’s called a spike-recovery experiment.
If its incompetence, it is field wide competence.
It has been assumed (I think) that the amount of 2nd structure will have no influence on the yield of recovery. Perhaps this is not so.
It might be interesting for some bioinformatician to a meta-study on published miRNAs papers to see if RNAs with low G-C content are more likely to be reported as differentially regulated. Perhaps length might have an effect also.
I am not sure how you control such a problem – maybe rigorously controlling the amount of extraction reagent to cell number or perhaps per mg protein?
LGR, I’m not sure why a “reply” link didn’t come up on your message, so I’ll reply to myself.
It appears that the authors are describing pretty simple matrix effects that alter sample extraction efficiency. In Pharma development, we follow well-established analytical validation principles using standard experimental approaches (spike recovery and dilution linearity analysis) to avoid these traps in even the most basic experimental situations. What is reported in their retraction notice is so elementary that it wouldn’t warrant publication as a technical report in most analytical journals. First year grad school stuff, really.
Given that selective miRNA “expression” under conditions of varying matrix concentrations was central to the story, one would hope that the potential for such experimental artifiacts would be addressed more rigorously. There’s really no excuse for shoddy science like this. Perhaps the authors should rephrase their conclusions to state that the far-reaching implications of their studies are that people should validate their assays before jumping to conclusions?
Pharmapawn, just a technical comment: this blog is apparently set-up to allow only three levels. That is: comment, reply, reply-to-reply.
If you allow infinite replies, you can get some problems following the discussions, with some of the reply-to-reply-to-reply-to… comments ultimately
lo
ok
in
g
li
ke
t
hi
s
“It appears that the authors are describing pretty simple matrix effects that alter sample extraction efficiency”
It may look like that, but it isn’t. Although in fairness it is not massively different.
The more G=C content, the more secondary structure, the more secondary structure the greater the resistance to degradation. RNA purification is highly variable and highly susceptible to degradation that yields can vary widely even among a series of samples processed in parallel. So that results are virtually never expressed in physical quantities but in arbitrary units that are valid only for a single series, either expressed in terms of x-fold induction against a null condition and/or normalised against a stable housekeeping gene.
I have not worked with microRNAs myself so I won’t comment further. But my understanding is that it is extremely uncommon to perform the type of control that you describe and that Dr Kim undertook.
I guess it’s a matter of perspective and awareness. Every single quantitative biomarker assay I’ve developed over the past 25 years has taken into consideration the amount and quality of the sample matrix, and its effect on recovery of spiked internal standards.
Here, the authors are quantifying a small set of miRNAs, and claiming that slight differences in nucleotide sequence lead to differential biological regulation. Given the small number of specific analytes under consideration, and the relatively low cost of generating radiotracers or stable-labeled internal reference standards, I can only attribute their failure to monitor extraction efficiency to ignorance or impatience.
To be fair, it’s unlikely that a cursory glance through the “high impact” literature would lead the average academic biologist to examine analytical chemistry standards and practices. Still, I think it’s reasonable to expect that Dr. Kim be aware that miRNA extraction methods are known to yield quite different results. Kim certainly deserves kudos, though, for so effectively spinning these results into what is essentially a second paper characterizing an experimental artifact.
Believe it or not, these obvious expressions of bitterness and schadenfreude do have a point. The publications of development scientists, who do the yeoman’s work of clinical chemistry and drug development, are typically relegated to the stagnant backwaters of the literature. Without loads of sexy biology, it’s almost impossible for us to publish assay development and validation articles where they will be seen by the average reader of SCN family journals.
Basic biological research in Pharma is now essentially extinct, and the industry is almst completely dependent on academic labs and biotechs, where this kind of rushed garbage is now the norm. My Pharma job is now solely to move new drugs toward the clinic. Given that, I suspect that my publications will increasingly be focused on “mundane” aspects of analytical biochemistry, and will continue to be scorned and ignored by “cutting edge” labs like Dr. Kim’s who publish their artifacts and crappy analytics “high impact” journals like Molecular Cell.
I’ve given up on expecting to have a wider impact on the literature, but at least I’ve never had to correct or retract a paper…
I agree with tjvision above, this is not a paper that should be retracted as it is not fraudulent nor considered wrong at the time of publication. New information has come to light and a correction paper or note is required to add to our sum of knowledge. This is science. This new information now sits in the grey literature instead of as a proper paper.
A case of overreaction? As more people with no real clue about science join in, more cases such as this one will occur…
I really feel sorry for the authors. This kind of thing is a huge problem in lab science – at least in this case they’ve managed to figure out a source of the difference they observed, but lab methods are often based on an assumption about how they work. There’s any number of potential things that could go wrong and it’s essentially impossible to test for them all.
There’s reports of plastics in lab consumables leaching into solutions and influencing results, so that different results can be obtained from the same experiment conducted in glassware vs plastics. I’ve seen debate around whether a certain cytokine was pro-inflammatory or not, with some letters to the editor suggesting that contamination of the samples with left-over LPS (an inflammatory stimulus) from generating the cytokine with recombinant technologies from E Coli was in fact the source of the pro-inflammatory effect observed.
I suspect there’s a lot more artifacts out there published as genuine results, where the source of bias or error has simply never been figured out.
Actually, I use Trizol all the time (not for studying miRNA though). Have I tested it to make sure that all my mRNA is being equally extracted? No. I don’t even know how I would do that properly. I could ‘seed’ an experiment with a known amount of different mRNAs (although how I would get those I don’t know) and try out different extraction methods and see how the results compare. But which one would be right? Trizol is a standard reagent in the field, which has been in use for decades.
I agree with pretty much all you say.
I think miRNAs, being so short, have a greater potential for this kind of distortion than mRNAs.
If you wanted to test this, some kind of variant of this rough outline might work.
Do a very large and concentrated prep of mRNA – or pool a number of left over experiments.
Do an RT-PCR with standard curves on a number of mRNAs you work on + a couple of housekeeping genes. I think there are programs that calculate the energies of predicted secondary structures.
Then take 3 different concentrations of this prep and spike them into a new RNA prep cycle – either just in water or perhaps into a real cell lysate (cross species maybe). Repeat the RT assay and compare with both % percentage recovery and whether the ratio of each mRNA to each other and the housekeeping genes stays the same. My guess is that recovery will be low and ratios will be all over the place – but not necessarily in any particular pattern.
But if the point is that GC content (or some other aspect of RNA) affects its retention during extraction, does testing a single mRNA address that issue? Does testing a dozen?
I remember from many years ago a paper showing that alcohol precipitation efficiency is directly correlated to starting nucleic acid concentration. Thus, I expect the problem with Tri-reagent which has an alcohol precip. step to be a direct consequence of using small samples and not a general problem, and probably remediable with sufficient co-precipitant addition. In my experience, larger preps are always preferable. Another issue causing low RNA recovery is water in sample which should be avoided as much as possible (e.g. from excess frost accumulating when adding liquid N2).
I salute the honest researchers!
Try column-zol RNA kit, which combinesTri-reagent and RNA Tini-spin column. It will recover RNA<17nt. I have compared the kits from many different companies, and found out the column-zol RNA kit from this small company works the best…
http://www.enzymax.net/columns/RNA_isolation_column_zol.htm
I hope I can get commission for this:-)
It is interesting to note that the authors have corrected the anomaly. They should be commended for that. However, I can’t help wonder why these authors and others in the filed cannot take more precaution to arrive at a wrong conclusion.
We and some other investigators avoid extraction of total RNA from small numbers of cells. In fact, to avoid gene expression bias on microarrays, it is a good practice to pool RNA samples from for example several mice. I am aware that some investigations have been done in small pieces of tissues from individual mice, which could skew the results.
As Booker indicated it is not easy to monitor whether use of TRIZol skews the results. Like most others in the field, we use this reagent all the time with large amounts of starting material and we pool total RNA from 4-5 mice extracted individually. This is also the most important caveat in clinical research of microRNAs since total RNA is usually extracted from small numbers of peripheral blood mononuclear cells. Unfortunately, it is not easy to collect large numbers of these cells from individual patients. Additionally, it will be impossible to pool total RNA from different patients since no two patients with the same illness are genetically identical except in monozygotic twin pairs, which are rare. There is also no consensus as to what constitutes the “house keeping genes” when studying microRNAs in general.
My proposal is to formulate some guidelines for microRNA studies, as the MIQE standards strongly recommended for gene expression studies but not followed largely by the scientific community.
I’m still flabbergasted that a paper touting differences in miRNA expression that are based on cell plating density didn’t control for cell number and miRNA recovery in their extractions. I don’t have access to the full text article, so perhaps someone can fill me in on the details of the controls in the paper. Did the authors simply compare the yield of miRNA from a low density plate with some fractional volume of miRNA isolated from a single plate grown at higher density?
I realize that Trizol methods have been standardized over decades, and spike-recovery experiments aren’t typically done, but this paper is addressing the mechanism of low recovery for particular miRNA species. Those controls pretty much design themselves.
Our lab uses Trizol method for the past 25 years and we did notice the loss of miRNA during the ethanol precipitation. Last year, we found a tiny column (Tini Spin Column) from a small company, Enzymax and we followed their instruction and combined the column method withTrizol method. This allow us to use column to replace the ethanol precipitation and it worked very well:-) They are now selling the column-zol kit this year, but you can still buy bulk RNA tini or mini spin columns from them…..
” it is not easy to collect large numbers of these cells from individual patients.”
I guess this depends on one’s definition of “easy” and “large”?
http://www.scripps.edu/california/research/dna-array/pdfs/blood_technote.pdf
I stand by my point regarding the predictability of this mistake. Incompetence or impatience.
It is not the question of definition of “easy” and “large”. I am referring to clinical samples (blood) from various human patients. Unlike tissue cultured cells, collection of peripheral blood from patients is limited by a number of factors. This is the reality of the situation in clinical studies.
I’ve obviously not explained my point about cell yield adequately. It’s slightly OT, but here goes:
Blood collection from patients (even very sick patients) doesn’t get much easier than a vacutainer. If you’ll read the link above or search Google for CPT and Paxgene tubes, you’ll see that they are clinically validated and yield plenty of cells for nucleic acid isolation and protein analysis*. I used to use these tubes regularly for biomarker studies in support of GCP clinical trials. 8-16 mL of whole blood (one or two tubes) was all I needed for mRNA profiling and qPCR. One CPT tube also yields more than enough cells for immunophenotyping and most phospho-flow cytometry applications. Lastly, there was plenty of residual plasma in a single CPT tube for protein/peptide mass spec. I’ve even used these tubes for biomarker studies in rats.
Cell isolation can easily lead to loss in recovery of intact cells arising from differences in cell recovery after trypsinization, washing, and centrifugation. Given the role of ECM and other secreted factors in cell biology, it’s not unreasonable to imagine that densely and sparsely plated cells (or cells treated with different doses of drugs) might be differentially sensitive to shear forces, osmotic stress, trypsin, etc.. Differential sensitivity can mean differential lysis during processing. When I do ANY of these procedures, even with cultured cells, the cells are treated exactly the same way, and we carefully monitor analyte recovery and matrix effects in EVERY experiment. Dr. Kim’s experiments cried out for spike-recovery controls, but the authors let that particular Dingo slip into the night.
*I am only an end user, and have no commercial interest in these blood collection products.
I take my hat off to Dr Kim. I think that the retraction was not – strictly speaking – ethically and morally essential, as the data were valid in conjunction with the spelled-out methods, the impact of their decision on the broader miRNA community is substantially higher this way. I am sure this is not an altruistic move, neither was it lightly decided upon, but the fact remains that this is the most efficient way to bring this technical limitation to wide scrutiny. Thank you.
From a technical point of view, I concur with Yoel that precipitation is the common culprit of differential yields, although this is typically driven by RNA length not composition. Adding neutral carrier (e.g. linear acrylamide, glycogen or yeast/bacterial RNA) usually solves the problem. I have not yet read in detail Dr Kim’s methods, but would be curious to see if any carrier was involved. Methods that employ column-based purification following the Trizol extraction may exhibit less bias. For our own research I might even go as far as using miRNA microarrays for re-extracted RNA to evaluate bias. On a side note regarding column-based purification, when I first started working with miRNAs I was using column-based kits separating RNAs longer or shorter than ~200nt. I was concerned about the fate of mRNA-bound miRNAs and compared the yields of a synthetic oligo complementary to rRNA and that of an orphan sequence. Result? Much of the anti-rRNA went with its mother ship to the large fraction. Solution? Heat the sample prior to Trizol addition extraction. Easy.
Pharmapawn, you express bitter and somewhat arrogant sentiment against biologists who do not use enough controls for their methods. The truth is that if we did, no biological advances would ever be made. You are a chemist testing chemical methods. We are cell-, developmental and molecular biologists, so while we can rigorously test the cell-, developmental and molecular aspects of our work, we have to defer the chemical testing to Industry Pawns who plough through the relevant chemical controls. We have to assume that the reagents and kits do what they are meant to do. This arrangement is far from perfect, and there were several recent Nature Opinion letters speaking out against this kit-culture, including the need to educate the young generation to do experiments from first principles and my own call for more transparent reporting from kit-makers. Our lab caught an artifact-driven project before a high-impact publication, but it still hurt to put aside a year of laborious experiments. To flip your argument, are you required to do the metabolic, physiological and clinical controls for the drugs you analyse? Because we are.
As to “the stagnant backwaters of the literature”, it is full of biological and clinical papers, too. There is a rapidly growing recognition, even from the SCN family, that methods are crucial – hence Nature Methods, Nature Protocols, methods-dedicated sections in other journals (e.g. NAR) and a vast number of journals on computational methods. While I agree that the high-impact-culture is fundamentally flawed, it is not biased against method developers. Your bitterness is against the increasing mundaneness of your scientific work. Well, collaborate with academia! To be worthy of a high-impact publication you actually have to show that the novel method (even if meticulous) can have real-life application. Ask any bioinformatician. Mundane research gets published in mundane journals. I should know, I am a proud author in a range of journals.
Once again, kudos to Dr Kim’s team (I feel for the students!). Do the additional necessary controls and resubmit the original original paper. I am crossing my fingers!
I hit send, but I’m not sure where my other reply went, so I’ll make this brief and hope that my other comments come in eventually.
I sincerely hope that your amateur psychoanalysis isn’t reflective of your approach to experimental design, but your uninformed comments and logical leaps about my background and responsibilities suggest otherwise. I’m trained in molecular and cell biology, and have 15+ years in industry to go with my 11 in academia. I’ve co-authored in high impact journals, and “senior authored” papers in middle impact journals (mostly the JBC). Lately I’ve been doing methods development and non-publishable, proprietary research, so I’ve been relegated to the backwater technical journals. I also have two patent applications pending that characterize PBMC miRNA patterns for patient stratification and treatment response analyses, and have spent some time thinking about biochemical biases in sample processing.
I think I know enough about this topic to make informed comments, perhaps you should spend some time learning about industry research? 100% of my work is now done in collaboration with academic labs where I act as the primary drug development liason to my company. On the whole, their “top-tier” research is pretty pathetic, and needs many MORE controls to ensure that any stumbled-upon “biological advances” are permanent.
My comments on experimental methods may sound arrogant to you, but please take some time to re-read your own assessment of our respective contributions to Science, then think about arrogance. I can’t really imagine that there is any more “real-life” application of basic life science than my life’s pursuit of translational biomarkers for drug development. Academic affectations and personal attacks add very little to the discussion.
” …bitterness is against the increasing mundaneness of your scientific work. Well, collaborate with academia! To be worthy of a high-impact publication you actually have to show that the novel method (even if meticulous) can have real-life application.”
If you know of a more “real-life” application for basic cell and molecular biology research than translational research in drug development, I’d like to hear it. Pharma is dying anyway.
FWIW, I am formally trained as a cell and molecular pharmacologist. Again, that’s no excuse for ignorance of basic biochemistry and analytical chemistry practices. If one choses to blindly follow instructions that come in a reagent box, they perhaps shouldn’t expect to be treated kindly when experiments fail. I strongly suggest you reduce your reliance on the kindness of strangers and kit manufacturers.
As to the source of my bitterness, it might also help you to know that 100% of my research is now done in collaboration with academic labs. As a part of target validation (i.e., even before the first new candidate drug is tested on cells) my teams now spend the first six months or so of every project replicating the academic work and applying rigorous controls to every aspect of the project. Guess what? My bitterness doesn’t arise from the unfettered brilliance of academia. The abundance of poor quality data, even at top institutions, is frankly appalling. Don’t just take my word for it:
http://pipeline.corante.com/archives/2012/03/29/sloppy_science.php
I’m not sure how to document the bias of academic editors and reviewers against industry research. Perhaps you should go to an AACR meeting, where every academic speaker is now forced to reveal the Scarlet P on their lapels, and every Pharma scientist must self-flagellate for the sin of collecting an industry salary. In my experience, that attitude pervades academic publishing as well. I’ve recently had reviewers of an analytical methods paper question whether my employment represents a conflict of interest in the publication process. For a methods paper. There is a separate water fountain for industry scientists.
Academics often claim our controls are unnecessary luxuries. Tenured faculty can hide for years behind a curtain of ever-moving priorities and the guise of “Scientific Progress.” The latest sparkly technology or sexy theory is usually sufficient to distract people from the fact that previous work was pure bunk, at least long enough to get to the next grant cycle. In Pharma, millions of dollars are spent on each drug program before it ever gets into humans. There, the real life consequences of experimental compromises are quickly evident. Some controls are mandated by law (spike-recovery, matrix assessments in analytical methods), but all are borne of hard, expensive experience. Given dwindling resources for basic biomedical research, and the incredible waste that sloppy science like Dr. Kim’s generates, I can only hope that sloppiness will become increasingly rare, and rigorous controls more common. Retraction Watch is playing an important role in shaping that path.
I want to clarify the point that Pharmapawn made about limitation of blood collection.
Some times, it is not feasible to draw more than 5 ml of blood from certain patients, for example from those who received radiation treatment. So, we have to deal with what we have instead of discarding the specimen. One cannot repeat blood draw from all of the patients on other occasions. Even if one does, it is not under the same condition. These are the limitations that I mentioned.
Another issue is that we considered using the tubes that Pharmapawn mentioned. Although one can extract total RNA from all of the nucleated cells in the blood using these tubes, they are not useful in every case. Traditionally, studies were done using peripheral blood mononuclear cells isolated using Ficoll gradient which is labor intensive and many cannot obtain enough yields consistently. If a group has been studying GWAS, SNPs, gene expression, microRNA expression etc from blood samples of patients for many years, they must use the same protocol to obtain RNA. Otherwise the results are not comparable. One cannot start using Paxgene or any other tube in the middle of the study, which lasts many years in some cases. If one starts a brand new protocol, then it is possible to use these tubes to isolate RNA from all white blood cells. Before jumping into using these tubes, one must do a comparative study in their own lab and decide which kit really works best. I am skeptical about claims made by companies who market these products or because your Core facility who may not have direct experience with these kits recommends it. There are controversies about kits used for RNA stabilization and RNA isolation. We should realize these limitations and share our various view points in this blog.
Although Anna Git expressed great displeasure about the comments made by Pharmapawn in general about biological research, I do have to agree with Pharmapawn in one aspect. Simple things such as controlling for cell numbers etc are usually ignored by the reviewers and the editors of high impact journals. You would be surprised that some primers including those for house keeping genes that are in use for decades have not been validated according to MIQE guidelines. It was even commented that most of the qRT-PCR data published in high impact journals cannot be reproduced. Ironically, even though MIQE guidelines are required to be followed when publishing in most European journals, American journals have not implemented such a policy yet. The scientific community should embrace these requirements as soon as possible to avoid controversies and embarrassment.
The old ways are sometimes the good ways. I always advocated (forced) people in the lab to use cesium chloride gradients to make pure RNA. Go to a conference and come back and discover them using kits from Qiagen, because some rep made it seem easy. “Have you proven recovery, have you measured DNA contamination?” Blank looks – start over explaining why. I call it protocol drift, the unseen force that slowly pushes your lab standards onto the rocks. Go back to cesium chloride for RNA: use a mini ultracentrifuge.
I completely agree. Radioactive tracers are often the way to go instead of fluorescence detection. We occasionally use organic, PEG, or salt precipitations instead of affinity chromatography or Protein G beads, though convincing some of our recent graduates to even try is typically quite a battle. Ouchterlony double diffusion usually destroys Western blots for characterizing polyclonal antibodies and serum immunoreactivity, but I can count on one hand the number of scientists under 50 who’ve even heard of the latter. None could could spell it.
That said, our current IP-LC-MS/MS methods were not even dreams when I was starting out, and mass-based analysis is really hard to falsify.
That’s a great move by the authors. It takes some guts to retract your own paper and show everyone the correct way of approaching miRNA’s. I may have missed it but what is the best/alternative way to extract miRNA from brain and blood samples?