Aaron Brown says his new book, Wrong Number: How to Extract Truth from a Blizzard of Quantitative Disinformation, “isn’t an exposé of fraud—Retraction Watch covers that ground. It’s about legitimate-looking research that is absurd on its face.”
Published this month by Wiley, Brown uses dozens of case studies to show “why widely reported and influential studies in top journals are not just wrong, but obviously and egregiously illogical or contrary to simple fact. My focus is less on the policy and statistical errors than on why no one seems to care,” he says.
Brown is a risk manager working in hedge fund management. He also teaches statistics at New York University and the University of California San Diego and writes columns for Reason and Bloomberg, among other outlets. We asked him to tell us more about how he thinks about the nexus of science, journalism and the publish-or-perish system that also pushes researchers to engage with non-experts to promote their work.
RW: Your case studies often suggest that if journalists consulted statisticians more often, the public would be less likely to adopt bad information. Is that a fair characterization?
Brown: Closer to the opposite: Don’t outsource your skepticism.
There are times when journalists should defer to experts — including statisticians — but the message of Wrong Number is you don’t need statistics, you just need to treat a journal article’s claim with the same skepticism of someone trying to sell you a mutual fund.
Ernest Rutherford is reported to have said, “If your experiment needs statistics, you ought to have done a better experiment.” I would extend that to journalists. If you need a statistician to write about a study, you ought to report on a better study.
RW: You argue for the occasional use of what you call “limited honesty” about scientific findings. Tell us about that.
Brown: I understand the dilemma of researchers who come up with findings they know will cause harm if released. A study finding some negative outcomes from a vaccine could cause parents who distrust the medical establishment to withhold vaccines from a thousand children who need them for every thoughtful medical decision it informs. When the science behind the nuclear winter hypothesis — that even a limited nuclear war threatened all life on the planet — began to be challenged , I understand why prominent scientists preferred to conceal that fact as the hypothesis seemed to be driving progress on arms control and peace negotiations.
What matters more than honesty is truth. When Gregor Mendel probably fudged his work to make the results neater, he made it more likely its truth would be recognized. A narrow, accountant’s honesty can get in the way of brilliant scientists forging revolutionary advances. If the ideas are right, we can forgive some unjustified data cleaning. If the ideas are wrong, data honesty doesn’t make them any more useful.
RW: The book includes a lot of reanalysis of overly simplistic statistical claims, and in your reanalysis, you often rely on government statistics. How would you advise when to trust and when not to trust governmental numbers and claims?
Brown: I’m a universal skeptic. Treat government figures with the same skepticism you would the odometer of a used car. That said, many government statistics have some advantages. The methodologies are usually disclosed in detail, and the numbers compiled by career staffers. With documented exceptions they’re computed the same way every period and they’re subjected to rigorous criticism and frequent use. If nothing else, they provide a common starting point for researchers, which is useful even if they’re not particularly meaningful.
But I give plenty of examples in the book where people go wrong. A headline National Transportation Safety Board study that curbside bus services had seven times the fatal accident rate of traditional terminal carriers — used to shut down 26 “Chinatown” bus services with excellent safety records — turned out to have included 30 traditional carrier fatal accidents (24 by Greyhound) stuffed alongside seven curbside carrier fatal accidents. A subfield of economics, “kinked demand curve” theory, turned out to be based on a misunderstanding of how the government collected price information. We often see figures like the unemployment rate among Black workers in Wyoming based on a survey too small to have covered any Black workers in Wyoming.
RW: You come across as a big fan of rigorous peer review. But you also worry that peer review can quash novel ideas that ought to be given a hearing. How do you reconcile this?
Brown: I’m a fan of rigorous post-publication peer review — the messy, ongoing process of papers being replicated, contested, extended, or quietly ignored as the field moves past them. That’s where science self-corrects. Pre-publication review is a device for enforcing conformity, not filtering out error.
A finding becomes trustworthy when it gets woven into the broader web, when other researchers build on it, find related effects, fail to replicate it, modify it. A claim that lives only in its original paper, never built upon and never refuted, is not yet science regardless of where it was published. Letting competing researchers block entrants into the process causes more harm than good.
That said, I absolutely support rigorous audit by statisticians without stakes in the subject matter findings. Alongside pre-registration of hypotheses, full disclosure of data and code and holdout samples, this could filter out a lot of bad research. Other gatekeepers could check citations (a major issue is papers that cite studies for crucial assumptions, where the study’s abstract says precisely the opposite of the paper’s claim). But studies should be blocked for errors, not because anonymous colleagues don’t like the findings. And decisions about importance should be made by named editors who take responsibility for them.
I have an even more radical suggestion: In most fields researchers shouldn’t be doing the studies in the first place. Being a subject-matter expert in psychology, medicine, astronomy or some other field does not make you competent at the data collection and analysis to test your ideas. I would love to see disinterested, specialized testing institutions perform studies suggested by subject-matter experts.
RW: Speaking of peer review, your book is published by Wiley, which of course also publishes thousands of scholarly journals. What was the peer review and fact-checking process like?
Brown: I won’t pretend a trade book gets the kind of scrutiny a journal article does. It doesn’t. So readers should treat my book with the same skepticism I urge for everything else.
There were multiple rounds of editorial review focused on argument, structure, and accessibility, and I went through line-edits and copy-editing. The publisher sent the manuscript to outside readers whose comments shaped the revisions. I sent it out on my own, including to people whose views differ from mine. For a random example, Victor Haghani, a friend of mine who blurbed the book, caught a major error in my account of capture-recapture analysis.
RW: Do you think of yourself as a scientific sleuth?
A sleuth investigates wrongdoing — fraud, fabrication, paper mills, image manipulation. That’s important work and I’m glad people do it, but it isn’t mine. The studies I dissect were, for the most part, honestly produced by researchers who believed their results. My complaint isn’t that they cheated. It’s that their logic or statistics didn’t support what they claimed, and that the surrounding ecosystem distorted and amplified the claim.
I’m a critic, not a detective. I’m not trying to get papers retracted; I’m trying to refute them. Retraction removes a thread from what Xenophanes called the “woven web of conjecture.” Refutation leaves both sides of the issue accessible for future researchers to learn from and build on. That’s more useful for results that are honestly reported but reinterpreted by better analysis. Future researchers can take up the debate, or learn from prior mistakes..
RW: In many ways, this book tries to help average readers see past biases. You write, “I am no populist, but I am a libertarian.” Did you attempt to control for your own biases in the writing of this book, and if so, how?
Brown: Readers who embrace my philosophy of skepticism should be skeptical of my belief that biases played no part in my selections. But I don’t think you’ll find a specifically libertarian bias. The book does seem to have a populist bias, but only because populists are the ones who distrust official dogma, and I’m criticizing bad research that either has become official dogma, or aspires to it.
Libertarians have promoted plenty of wrong numbers. I know many who think Sam Peltzman (a former professor of mine and a major influence) established definitively that seatbelt requirements increase traffic fatalities, and that the Food and Drug Administration kills one hundred patients by delaying or rejecting good treatments for every life it saves by blocking bad ones. Sam did good work on both questions, but there’s a lot of other literature that casts considerable doubt on both claims.
I’ve written about these and other libertarian wrong numbers, but I don’t need to fight bias to do it. I’m a moral libertarian. I believe it’s wrong for the government to coerce a competent adult to wear a seatbelt or to prevent her from choosing her preferred medical treatment. I have no strong belief either way about whether that principle would result in more or fewer deaths.
RW: You write that “researchers and editors don’t even try to publish stuff that’s more likely true than false.” (p. 129) What do you mean by this, and how would you remedy the situation?
Brown: To publish more true results than false ones you have three main parameters to work with: the ratio of true to false hypotheses that you test, the fraction of false hypotheses you reject and the fraction of true hypotheses you accept. Most journals establish a standard only for the middle parameter and do not report even general information about the other two.
Journals prefer surprising results, creating an incentive to test a lot of hypotheses that are likely false. Low-power tests that fail to reject lots of false hypotheses and support relatively few true ones are cheaper and easier to run–sloppier methods, smaller sample sizes–and can generate more publications by failing to reject false hypotheses than they miss by failing to support true ones.
For example, suppose you test 1,000 hypotheses that would be surprising if true, so only 10% are true. You reject 95% of the 900 false hypotheses, but publish the 45 remaining false results. Meanwhile you reject 80% of the 100 true hypotheses. You end up publishing 45 false results to 20 true ones.
It is possible to do good research under this standard. Skilled researchers can identify hypotheses likely to be true, but surprising to others. They can apply stringent evidentiary standards — extraordinary claims require extraordinary evidence. They can do careful, large-sample, high-power tests. But few researchers have the talent and resources to do this at a rate to satisfy university and granting agency publication requirements. So even many good researchers do some low-quality work, and some researchers can do only low-quality work. The journal system is set up to accommodate academic careers, not to publish reliably true results.
And sadly, this is not the end of the story. In addition to the fraud and other issues highlighted at Retraction Watch, there are many well-known flaws even in the “p-value” significance calculations for the middle parameter.
It’s no surprise that, as John Ioannidis pointed out in 2005, most published research findings are false.
Journals that wanted a low ratio of false results would at least make some effort to disclose standards for prior probability and power, in addition to significance. If they did publish these and readers did the math there would be a lot more skepticism about published results.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
This (the interview, not the book) was an incredibly frustrating read, full of contradictions and double negatives. It’s hard to tell exactly what the author is trying to say, but the crux of it seems to be “sometimes journalists blind the public with stats, so we should beware of stats and go with what feels right or true, oh and scientists can’t be trusted to do science.”
Sorry what now? This is the antithesis of the scientific method. Robust statistitcs, conducted by domain experts, is the framework via which we determine what is right or true. Without that, there is no truth!
Sure, statistics (just like any other framework, e.g. religion) can be abused to tell a story, but that’s not a good argument to ditch the entire framework and chase some mystical underlying “truth”. Again, how do we know if it’s true? Aah yes, hypothesis testing and statistics, by those with just enough prior knowledge (aka experts) 🤦🏼. The author comes across as semi-religious, as if there is an underlying universal truth that scientists need to adhere to, begone all that evil statistical chicanery! But strip away rigorous data and stats and what’s left?
There are just so many inflammatory statements in this piece, is hard to know where to start. Here are just a few…
(1) “Being a subject-matter expert in psychology, medicine, astronomy or some other field does not make you competent at the data collection and analysis to test your ideas.”
OK then, what DOES qualify anyone to collect data? Please explain why it is somehow a bad idea to have people with intimate knowledge about the underlying structure of a data set (and thus more able to call BS early on) be the actual data collectors? Should we just hire random folks off the street to do data collection because it’s more objective, even if the resulting “data” is completely useless because the people collecting it don’t know how to organize it? Subject matter expertise is quite literally the bare minimum qualification for doing any kind of research!
By all means, let this be a call to action for more training in statistics for graduate students and other scientists, but the author seems to flip the whole argument on its head and say “subject matter experts don’t know stats, so they shouldn’t do experiments.” Clownish at best.
(2) “Gregor Mendel probably fudged his work to make the results neater, he made it more likely its truth would be recognized. A narrow, accountant’s honesty can get in the way of brilliant scientists forging revolutionary advances. If the ideas are right, we can forgive some unjustified data cleaning”
This is just a sorry rewriting of a journal erratum notice… other people have repeated it, so even if we faked it, no harm done. Wrong! Careers are made and destroyed on who gets credit. Mendel didn’t get tenure, but many copying his example did, and that’s morally and ethically reprehensible.
As long as science is a career, it carries a certain accountability – an understanding of a level playing field. If cheaters are eventually proven correct, they’re still cheaters, and it’s not fair when they get ahead, because this is a zero sum game – a grant / promotion / paper for a cheater means somebody else lost out. Mendel got lucky. We can do better now.
(3) “Pre-publication review is a device for enforcing conformity, not filtering out error.”
There are lots of faults with pre-pub peer review (a multi-$B industry with 40% profit margins built on unpaid labor) but “enforcing conformity” is not one of them. I am eager to hear the author’s suggestions as to what we should replace pre-pub peer review with? Do they honestly believe that eliminating it will lower error rates?
The loaded critique of “conformity” is telling of the libertarian motive that underpins the whole piece… Conformity is what has kept insanity out of mainstream science (so far). Be grateful that conformity is why the NSF doesn’t fund flat earth research. Conformity is why there isn’t a national institute of magnets at the NIH. Confirmity is why we have basic standards for peer review (is it reproducible, is it ethical, etc.)
Unless you’re an ivermectin-huffing, vaccine- denialist, conformity is not something to be scared of, it’s the shoulder of the giant you stand on to build a new discovery. Without conformity, our scientific journals would be (even more) awash with manure.
—–
The more I read the interview above, the more I pick up on a strong anti-science theme, wrapped up in a veil of “right to try” and “freedom to choose” libertarianism. Its deeply troubling that a trusted science journalism site such as this would give page space to such bunk.
Thank you. You saved me having to write all this myself.
while your concerns are valid, I sense a reverse bias in your overall thrust. You are expending his remarks beyond what he really says.
Personally, I felt that the piece highlights the necessity of a Bayesian approach: explicit statements of prior probabilities to plug into the equations.
The point is that you don’t waste your time constructing experiments to analyze hypotheses that have extremely low prior probabilities. If “The Earth is flat” is going to be your null hypothesis, then a few data points, casually obtained, are going to reject it every time.
Using basic analysis of your assumptions will allow you to plan experiments that have an obvious effect on the posterior probabilities— and avoid those that won’t move the needle from your prior analysis , even if the data appears to be highly significant at first glance.
For example, your null hypothesis is “the Earth is round.” Your hypothesis testing should be pretty cheap. A “round skeptic “ tells you he has five-sigma data “disproving” this, but it’s going to cost you. Do you buy it (perform the experiment on offer)?
No, because you’d still be unconvinced by that heavy data and ask for seven sigmas. How can data that supports a flat Earth possibly be accurate rather than fudged? Why spend big money to confirm a model that doesn’t help you accurately calculate the shortest distance from Berlin to San Francisco?
Seeing data that supports a flat Earth, you’d feel that it is more probable that the data is bad than that the Earth is flat— even if it pencils out to nine sigma significance.
So, he’s calling to respect prior probabilities and calculate how much you can change them with a reasonably priced experiment. Don’t start experiments that won’t change the posterior probabilities without breaking your budget.
That’s my impression of what he says— I could be biased in his favor.
@Seitz — what you say mostly makes sense, but “You are expending his remarks beyond what he really says.” To my reading, once Brown said “What matters more than honesty is truth…. If the ideas are right, we can forgive some unjustified data cleaning” above, we have a problem.
Whether it’s climate science or vaccine science, skeptics are looking for any flaws that they think justify their position. Unjustified data cleaning is enough to get people to reject the truth. We have to do our best to be honest, so that people can trust us enough to accept the “truth.” Advising anybody that if their ideas are important and true enough, they can do some “unjustified data cleaning,” is terrible advice.
I’m an experimental physicist, for what it’s worth.
I’m sorry you were frustrated. Double negatives are not bad and they are unavoidable when discussing null hypothesis testing. I assume you are referring to that discussion or “data honesty doesn’t make them any more useful” or “never built upon and never refuted, is not yet science.” How would you recast those to be clearer without double negatives.
I think the contradictions you found were intended as contrasts. “Pre-publication peer review is a device for enforcing conformity” while I endorse rigorous pre-publication review by data and methodology professionals with no stake in the paper claims, rigorous post-publication peer review, and rigorous paper selection by named editors.
The flat-earth paper should be rejected by a named editor. Editors should be responsible for only considering papers that are remotely plausible and important if true. To the extent the decisions are easy–like your example–peer review is a colossal waste of valuable people’s time. To the extent they’re debatable, such as a paper claiming to have demonstrated something considered unlikely but not impossible like cold fusion–that’s an editorial decision the journal should take responsibility for.
And, in fact, this is how it works at most good journals. The editor or staff turn away obvious junk. Everything else is assigned to a named co-editor who chooses whether or not to send it to peer reviewers. There is also a data and methodology review by professional statisticians. In my view, the peer reviewers seldom devote the time and energy necessary to catch errors or improve the paper, but they do shape what gets into print based on their personal views. Eliminating that step would let more good papers through to be exposed to broad discussion, validation or refutation, without significantly increasing the number of papers that are merely waste everyone’s time.
I don’t understand your hostility to more studies being run by bench and field scientists rather than theorists. This is productive in many fields and I think could be extended. A theorist who proposes an idea has a stake in the outcome, and is not necessarily the most qualified person to organize and run the test. What would be so terrible about having, say, a Behavioral Economics Testing Institute, staffed by behavioral economists trained and experienced in studies, statisticians, interviewers and other professionals? Theoretical proposals could be submitted for testing. I think this would remove a lot of the cloud from the field from non-reproducible results, save the time of theorists that could be better spent on theory, teaching and writing, and result in better tests.
I apologize that the Mendel point got overstated in the editing. I hope you read the relevant chapter in my book. I am a tell-the-full-truth-and-let-the-chips-fall-where-they-may, pure honesty guy in science (not so much in other areas of life). But I felt the book needed a steelman chapter for defensible amendments. The vaccine researcher who turns up a small risk that she knows will cause thousands of parents to irrationally withhold essential vaccines from their children. The physicist who comes up with something that could be, and likely would be, used for a weapon of mass destruction.
More fundamentally, no experiment is ever perfect and definitive. Much of the skill in science is in deciding when you have enough certainty to publish. That’s not something you can read off a p-value chart. And you can never disclose every possible limitation, assumption, outlier, uncontrolled influence. I don’t say that to defend dishonesty, only to note that there’s a continuum, and no one can achieve 100%. People who publish true results with somewhat cleaned-up data–necessary to get the work published–are not obviously worse people than people who refuse to clean up their work and never get anything published.
I believe in the woven web of conjecture–the consensus of a field open to all investigators and opinions, that validates and replicates work, and tests assumptions from multiple dimensions. I believe most published research findings are false, meaning most threads are weak, but a fabric does not require all threads to be strong. I urge people to be skeptical of threads that are rushed into reporters’ hands by university press offices–or into political speeches, legislation and court decisions–without ever having been woven into anything, merely blessed by anonymous peer reviewers (and sometimes not even that). And my book argues you don’t need deep subject-matter expertise nor a PhD in statistics to be skeptical, the flawed studies can be spotted easily without a lot of effort or expertise.
A couple years of reading PubPeer reports has convinced me that when a scientist decries anonymity, all too often it’s because anonymity is getting in the way of trying to silence their critics. I am deeply suspicious of any such proposal as a result.
Look at what happened to DataColada: while they successfully defended themselves from a lawsuit after criticizing someone, it was incredibly stressful and expensive. How many critiques simply won’t get made if every critique has to be signed?
There is a reason why every substantive whistleblower-protection law includes the right to do it anonymously.
I am also unconvinced that people randomly deciding to do post-publication peer review is a more effective approach than pre-publication peer review. I have personally caught two plagiarized papers and a dozen or so with gross issues in the statistics or data, as a peer reviewer. Even though I’m interested in post-pub review, I don’t have *anything* like that record there. Having an editor ask me to look at a specific paper is incredibly helpful.
It’s clear that your peer review is useful. My experience is that most peer review is not. And obvious plagiarism or data issues really should be caught earlier, by journal staff or the co-editor assigned to the paper. PhD biologists have better uses of their time, especially as we have AI tools today.
What a peer review could catch than a statistician or AI would not, are things that require subject-matter expertise. I have found these from time-to-time in my peer reviewing. But you don’t need peer review for this. Papers usually have multiple authors with subject-matter expertise, and are shown to colleagues, presented or pre-published. If all those subject-matter experts endorse the paper, why would a few more–often selected on the recommendation of the authors–improve the decision?
Balance that against the risk of peer reviewers suppressing work they dislike, perhaps out of genuine belief that it’s wrong, perhaps out of career reasons or personal squabbles.
It’s fascinating to see anyone who supposedly believes in data arguing for abolishing pre-publication peer review. The observational data on this are already available, because we already have lots of journals where pre-publication peer review is weak or nonexistent. And guess what? Almost nobody reads those journals, because they’re awash with spurious or trivial results.
My recent experience as a peer reviewer has included 1) a manuscript where the authors failed to correctly solve a simple differential equation, 2) a manuscript where spectacular results were derived by imposing a physically impossible boundary condition on the same equation, 3) a manuscript that violated dimensional analysis, and claimed to be performing “spectral decomposition” without actually estimating any spectra, 4) multiple manuscripts that cited other papers for things that those papers did not actually say, 5) multiple manuscripts that made grandiose claims that were unsupported by any actual evidence, and 6) an extra-special manuscript that violated dimensional analysis, conservation of mass, the second law of thermodynamics, and the fundamental theorem of calculus… all on the same page.
All of these manuscripts had multiple senior authors from major universities. And they were all submitted to respected journals (mostly PNAS, Nature, and Nature offspring), where they had already passed through the initial screening steps.
It is vexing enough to have to deal with this kind of stuff as a peer reviewer; inflicting it on the entire community would be scientific malpractice.
It’s clear that your peer review is valuable, but my experience is that most is not.
I don’t know which journals you have in mind that have eliminated pre-publication peer review, but I suspect they’re journals that do little or no pre-publication review of any kind.
Your specific account could be read as defending the value of peer review, or supporting my claims about the inadequacy of other pre-publication review. I admit that some–maybe all–of your six examples are things that a competent statistician carefully reviewing the methodology and replicating calculations from the provided data and code, and a competent journal editor reading the abstract to judge importance and plausibility, might both miss. So there is some value to some peer review. But how many hours of PhD environmental physicist time did it take for that value? And how many peer reviewers actually do that kind of a job? And what is the balance against good papers rejected by peer reviewers for bad reasons?
And note that all your papers were already not just reviewed but written by their multiple senior subject-matter experts–and presumably reviewed by many other colleagues in personal communications, pre-publication discussion or conferences. Assume a veil of ignorance. You know only that a paper was written and reviewed by multiple senior experts, and some other experts–often recommended by the authors–voted against it. If you had to make the same choice for all such papers would you suppress them all, or publish them all for exposure to post-publication validation, refutation or replication?
Very strange to see this on Retraction Watch. Aaron Brown is a climate “sceptic”:
https://reason.com/video/2026/02/05/these-climate-change-charts-are-scary-theyre-also-wrong/
TBF, he describes his position in a more nuanced fashion: “Warming is real, humans contribute substantially, it matters, and the responses we choose matter at least as much as the diagnosis.” For what that’s worth.
https://reason.com/2026/04/29/is-reasons-video-on-climate-change-alarmism-a-masterclass-in-manipulation/
I don’t like the idea that people who claim to be alarmed about climate change are deviously trying to manipulate you and arguing in bad faith.
I’m alarmed by the data and I haven’t seen anyone trying to claim that the worst case scenario is the most likely one. The middle scenario is bad enough.
So if he’s claiming manipulation, in my mind that makes him a “skeptic”—a minimizer, almost as bad as a denier. We need to build a fire under society, not justify complacency.
NB don’t stop pre-publication peer review. That’s just lazy. And don’t pierce anonymity— that’s doing the work of the intimidators by allowing doxxing.
I take back everything nice I said before.
I do not claim manipulation, I claim misleading presentation. The data are correct but the graph is drawn in a way that suggests a false conclusion. My alternative graphs make the valid point that global mean temperature is increasing at an increasing rate over the period in question, but in something like a quadratic curve, not an exponential hockey stick, and that the strong signature is declining record cold periods rather than increasing record hot periods.
I have been writing about climate since the 1970s, when I said that it was a 100-year problem. I now say 50-year, because 50 years have passed. I have devoted my energy and money to solutions to reduce the amount of energy necessary to produce a dollar of real GDP. I think this is the key to reducing the entire human environmental footprint while continuing to reduce poverty and support prosperity. That effort (not my personal efforts, everyone who thinks like me) has reduced that key cost by 60%. Another 60% in the next 50 years and a lot of environmental problems seem manageable.
And what has “lighting a fire” under people accomplished? A lot of talk and anxiety. A lot of government funds sucked up. Some legislation with a mixed record–sometimes merely pushing activity into less regulated jurisdictions, sometimes enriching cronies, sometimes mild benefits, a narrow focus on CO2 and warming that has unintended consequences on other environmental issues. I particularly object to subsidizing green energy–I think anything that encourages energy generation is net harmful, I support taxing or restricting dirty energy. I don’t say lighting fires is useless, but I’ll take the objective 60% of my approach over it.
While I’m glad there are a lot of people pursuing different approaches, I have grave reservations about fire-lighting. It can lead to rushed, panic solutions like geoengineering that I think carry more risks than benefits, especially if done quickly. It can lead to regulations that pick winners and losers, suppressing useful innovation.
Before you light a fire, make sure it’s under a boiler that’s hooked up to do useful work, and that won’t explode.
I assume the quotation marks around “skeptic” are meant to suggest something nefarious. I am an actual skeptic, and not just about climate claims. My book is an attempt to make people more skeptical of “studies prove” stories they may read.
However in the video you reference I am not expressing skepticism, I’m criticizing the presentation of data. The data in the chart is correct but the visual impression it gives is misleading.
My skeptometer was bobbling along in the green and yellow until it pegged in the red at, “What matters more than honesty is truth. When Gregor Mendel probably fudged his work to make the results neater, he made it more likely its truth would be recognized.”
This is a clear justification for committing scientific fraud. When fraudsters are asked why they did it, the most common motive I’ve heard of was that they believed in their hypothesis and were just trying to “clarify” the picture for the rest of us—exactly what Brown promotes.
The essence of science is that the data are *supposed* to alter your beliefs, not vice versa. If the data are at variance with your beliefs, that should arouse curiosity, not a cover-up.
I apologize that the Gregor Mendel piece lost some context in editing. I am a pure honesty in science supporter. But I felt my book required a steelman chapter considering counterarguments people have raised.
Defenders of Gregor Mendel point out that no experiments are ever perfect, and no one could possibly disclose every uncontrolled influence, assumption, outlier or other qualification. I might submit a 100 kilobyte paper to a journal, with a 10 gigabyte GitHub for data and code, and have 100 gigabytes on my hard drive produced while working on the paper. And there is a lot of stuff that never made it into the computer.
Much of the skill in science is deciding when the results give enough confidence to publish. That’s not something you can read from a p-value table. That gives us two dimensions to judge a scientist–whether or not the result was true, and how thoroughly she disclosed qualifications. No one is defending dishonesty, the question is in a world of necessarily limited honesty, how broad are the acceptable limits?
Mendel’s defenders put more weight on truth as a counterweight to perhaps more-limited honesty. I don’t subscribe to that belief myself, but I report it out of respect for honesty.
I suppose this shows that sometimes, very rarely, audience capture is actually a good thing. “Center for Scientific Integrity” needs to mean something.