If you are at all familiar with scientific sleuthing, you’re familiar with Elisabeth Bik. She is quoted so often in the mainstream media it is probably difficult to imagine a time before her supersense for spotting similarities in images wasn’t making headlines.
But it was 10 years ago, on April 19, 2016, when she made her debut, when we covered her work screening more than 20,000 biomedical research papers containing western blots. She and coauthors Ferric Fang – a member of the board of directors of our parent nonprofit organization, The Center for Scientific Integrity, and a professor at the University of Washington in Seattle – and Arturo Casadevall, of the Johns Hopkins School of Medicine in Baltimore, posted the work as a preprint on bioRxiv.org and it appeared two months later in mBio.
The preprint was a shot across the bow for journals and publishers, and in the decade since, Bik has advised and mentored others doing similar work. In 2024, she won the Einstein Foundation Award for “identifying misconduct and potential fraud in scientific publications, highlighting science’s problems policing itself.” She donated the proceeds to The Center for Scientific Integrity to create a fund to help other sleuths do their work.
Bik spoke with us earlier this month about the paper, sleuthing and more. The conversation has been edited for clarity and brevity.
Retraction Watch: We first interviewed you in 2016 just as you and your coauthors posted your review of western blots in 20,621 papers. Ten years later, do you know what has happened with those papers?
Elisabeth Bik: Well, not all 20,000, but the 800 or so papers that I found problems in, yes. Of the 782, 177 have been retracted, 42 have an expression of concern and 256 have been corrected. And if I count all three of them, that’s 475, so 60%.
RW: Do you think that should be 100%?
Bik: Yeah, I would have loved it to be a little bit closer to 100%. You can see papers are still being corrected. Like this paper, for example, was retracted in 2024, but I reported it in 2015. [On Zoom, Bik was pointing at the spreadsheet she uses to track papers.] Most of these were reported to the journals in 2015.
RW: What did people think of your paper?
Bik: It was rejected four or five times. In the end, we were like, we’ll just put it as a preprint and do an interview with Retraction Watch.
Nobody believed this paper. People didn’t believe I scanned 20,000 papers over a period of maybe roughly, I would say a year or two. I did a count on how much time I used to scan one paper. And it was about one minute per paper. Really, I’m not reading a paper, I’m just looking at the images. We took it out in the end because so many people were like, that’s impossible. I’m proud of it, but that’s apparently the point that breaks everybody.
People also wanted to know, ‘what is your false-positive and false-negative rates’? We weren’t quite sure. There’s no real gold standard for it. Like what is standard for image duplication? I was the first to raise this. So it’s hard to have to test it against another test. And I also don’t know how many papers I missed. I think we were more worried about claiming a positive where it wasn’t a positive. So that’s why my two coauthors were incredibly helpful. But I know I must have missed a lot of these problems.
RW: But 782 out of 20,000 is not nothing.
Bik: Yeah, it’s 4%, or 1 in 25.
RW: You’re known for finding duplications and manipulations in images, but you started out scrutinizing papers for plagiarism.
Bik: That is how it all started. I found that somebody had plagiarized my work. And I worked on plagiarism for nine months or so. And then I came across a Ph.D. thesis that had not only plagiarized text in the introduction, but also a duplicated image that my eye was drawn to. And that evening, I was thinking, wait, that happens? Maybe I should open a couple of PLOS One papers. And I found a couple already that evening. Otherwise, I would not have been talking to you today. Looking back, it’s one of those little moments that change your career.
RW: You had a recent correction to a paper you coauthored.
Bik: All my papers have been criticized, scrutinized. In a way, it’s fair. I criticize others, people can criticize me. In that paper there was a splicing where we left out a group, and you could see a remnant of a line. It wasn’t like we were trying to change the results or anything. But we corrected it. We found a lot of the original data and we worked with the journal to correct it.
All my papers have been torn apart for the weirdest reasons. You have to put so much work into addressing these things. In a way, it’s fair to be criticized, but I do feel sorry for my coauthors who are dragged into these long discussions.
RW: Do you still scan papers by eye or are you mostly using software?
Bik: Both. Sometimes I see the problem right away, and then I run it through Imagetwin and Proofig. Especially duplications between papers is something I’m not good at, because I cannot remember a million other papers, but the software can. Now you scan these papers and it finds, look, that blot has been used in that other paper, but it’s flipped and it’s representing a different protein. And so it’s the same photo, it’s just flipped and resized a bit. It’s very clear once you compare it, but I would never be able to remember all these blots and all these papers and see these patterns. So we’re finding more of these problems with these software tools that have these libraries of images.
RW: You, and many others – including Retraction Watch – have been accused of targeted attacks in post-publication peer review on social media. What effect does that have on your work?
Bik: It worries me a bit, especially when they tag my family. I’m always a bit worried about personal safety. Sometimes the critics will send emails to the host of an event I’m speaking at and say that I’m fraudulent. You have to say to the organizers, I’m very sorry you’re bothered by my enemies. And then, there’s talk about it. What should we do? Should we respond? Should we not respond? Emails have to be sent to all these dozens of people to not respond. It’s just a lot of work for everybody involved. And I feel so sorry that comes on top of organizing a conference, which already is a lot of work. On the other hand, I think it’s good that they see my work does result in personal criticism.
RW: Sleuths have become an essential part of the whole research integrity ecosystem. How has that changed in the last 10 years?
Bik: I think it’s wonderful to have this growing community because this work, at least the way I do it, is very by myself, which I like. I’m a super-introvert. I don’t really work well with other people. I like to be loosely connected to a community. We’re all sort of a bunch of misfits. I love to be independent. Then there’s other communities who are meta scientists. And people working at publishers doing this work are also wonderful people. And I think all the noses are sort of starting to point in the same direction, which is lovely. It’s becoming part of what science should be. But you have to start in a way that upsets a lot of people and makes people uncomfortable.
There’s still a lot of room to grow. I think we all agree on that. If you buy a car and the airbag is not good, there should be a recall, right? It should be better. Moving forward, all the cars should have better airbags or better wheels that don’t fall off. If we buy a product, we should be able to complain about it. There should be quality control and there should be customer service. And I think that was a bit lacking in the scientific publishing world. And both of these things are getting better. We are growing towards each other and learning from each other.
RW: One of the criticisms we’re seeing as a result of some of the big misconduct cases is the belief that they mean we can’t trust science. What do you say to that?
Bik: I end most of my talks with this exact point. I’m talking about that one rotten apple in the fruit basket. I love science and I do this to make science better. Maybe I’m considered a vigilante because I point out the bad stuff, but it doesn’t mean that we cannot trust science. We should just do a little bit better in screening before we publish things. We should be critical. And I feel we can all agree on that.
But it has been used, weaponized, in the misinformation era where people say, all science is fraudulent, that you cannot trust any science paper. I think that is the wrong attitude, but it’s the double-edged sword we’re working with.
It’s very easy to draw that conclusion, but that is the wrong conclusion. We need science.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Dr. Bik: Thank you for your pioneering and risky work!
The available data say we can’t trust the vast majority of what is published. Three surveys, run by different organizations, found out that 85% of the papers analyzed were irreproducible. Those surveys focused on papers published at top journals, which means the real irreproducibility rate would be much higher if we considered also the low-quality journals. The real rate could exceed 95%, I believe. Most of papers’ irreproducibility is due to fraud. In fact, it is unplausible to imagine scientists making mistakes 85+ percent of the time. What we find as sleuths is just the tip of the iceberg, just the detectable fraction of the faulty data.
I’m aware there are still good scientists out there, but those are likely a minority. I would rather change the metaphor into “one good apple in a basket of rotten fuits”. Unfortunately.
Is there empirical evidence that supports your belief that the rate of irreproducibility is higher in “low-quality journals” than in the “top journals”? Did the 3 surveys compare rates of irreproducibility among different levels of journal status? You say the real rate of irreproducibility “could” exceed 95 percent; where did that number, 95, come from?
So, the only two reasons for irreproducibility are either fraud or mistakes? Are there no other variables that you can think of that might contribute to this complex issue?
You take money from the public to design and build a bridge for traffic. You say it’s safe yet it collapses when a car tried to cross, and then it turns out the steel is counterfeit and it wasn’t designed with proper structural integrity for cars, it just looked like it was, yet the person took the paycheck from the public taxes and cashed it. What other options are there in deciding between incompetence and fraud?
You can say “well, the engineers got together and decided questionable engineering practices were OK because engineering is hard” and you’ll be laughed at. But not in social sciences, because it’s soooo haaaard
Provide a reference, please…
The three surveys, run by Bayer (2011), Amgen (2012) and Center for Open Science (2021) respectively, are here:
https://www.nature.com/articles/nrd3439-c1
https://www.nature.com/articles/483531a
https://www.cos.io/rpcb
Your first link is to a letter to a journal, not a survey run by Bayer.
Your second link is to another letter to a journal, not a survey by Amgen.
Your third link is to a replication project, not a survey, and it doesn’t say what you claim it says.
You know in the article where it talked about people weaponising the work of sleuths like Bik for the purposes of spreading misinformation and undermining science? They were talking about you.
The links I provided contain all the information on the three surveys I mentioned. You don’t like the word ‘survey’ and prefer ‘replication test’? Great, noted. Based on the tone of your response, it seems like actually reading the articles and websites I listed is too much effort for you. Those three suveys/replication tests are reknown to anyone has a glimpse of knowledge about the reprodicibility crisis in science. You have no clue what you are talking about.
I strongly disagree with the statement that all of this has to be fraud because it can’t be mistakes.
When I was a graduate student in the late 1980’s, my advisor told me that she ignored any report that gene X was associated with disease Y until it had been replicated. The rate of false positives was too high. This was not fraud; it was a failure to grapple with the multiple comparisons problem, combined with publication bias.
Similarly, large areas of psychology are rendered irreproducible by a reliance on too-small sample sizes, combined with publication bias. That’s not necessarily fraud either, though it is certainly a mistake.
There may well be subfields where most of the issues are fraud: anyone who browses PubPeer will notice a few of these. (Role of microRNAs in cancer leaps to mind; also certain forms of numerical optimization.) But I don’t think it’s general to all of science, by far.
I didn’t say it’s all fraud. I said it’s most fraud. I have no clue about psycology, I’m talking about biomedical sciences.
Sure microRNAs, but also oncology, Alzheimer, pharmacology, anesthesiology, toxicology, nanoparticles in medical applications, aging, Ayurveda, TCM, etc.
And outside biomedicine we have: perovskites, MOFs, bestiary-themed optimiser algorithms, fuzzy logic, etc., etc.
You say it isn’t fraud, but a lot of it is fraud we just don’t like to say it because we can’t necessarily prove it so we call it “questionable research practices.” When you look into a lot of why Psychology Research doesn’t replicate, much that didn’t was from actual lab studies specifically excluding self-report (also junky) and small samples. When you tweak knobs that somehow give you a result that falls apart under examination yet present it as a truth, what is it really except fraud? The fact that social sciences decided to get into a circle and nod their heads that it’s all OK doesn’t change the reality that much of what they’ve been selling isn’t actually science, it’s cosplay.
If my mechanic turns some knobs on the diagnostic software that will make it more likely the computer says I need to replace a part, it’s certainly fraud. When someone does it to get a paper published, what is it?
You might argue it’s gross incompetence, but since most of their salaries are derived from public funding in some form, I’d argue it’s most definitely fraud on the tax payer.
Yikes, no. Reproducibility does not imply fabrication. I have been around for decades, I am an old professor. Twenty years ago, my research rigor was the same as it is today. But the publishing landscape changed. There was little to no push for reproducibility in my early years. Jokes apart, good luck trying to reproduce what I did on a hot summer night in ’05. That does not mean I have cheated or any of my PhD students have cheated. We just did not have the impetus to put all our data on a clean spreadsheet or model because that was not a typical requirement.
Dr. Bik’s contributions to the scientific community, particularly in promoting research integrity, is huge. Her work has also revealed an uncomfortable reality: over the past decades, many published manuscripts, even peer-reviewed, are not be as reliable as they should be, no matter they from early-career researchers or so-called superstar professors. Even if I have left academia because of frustrations and some drama, I always believe her work encourages or warn more scientists to do science with right way and right attitude.
It is really remarkable how much is owed by the scientific community to the efforts of a small band of sleuths whose insights and doggedness have brought to light the undercurrent of bad science and steps to reduce it. And while I greatly admire and appreciate the pseudonymous sleuths working away in anonymity,* the courage of Elisabeth Bik and some others willing to openly call out bad science deserve more credit than they will ever get. Legal threats, risks to their own careers, and personal abuse seem to be the most likely rewards. Data Colada, Leonid Schneider, Fredrik Jutfelt, David Sanders, and Christopher Clack are some others that come to mind for courage in publicly taking on bad science. While there have been a few who hit cash jackpots for exposing fraud (e.g., Joseph Thomas, Sholto David), lets raise a glass to all the others.
* and have worked a couple of their pseudonymous scientific names into a published paper as a nod
There is another group of data sleuths. They are the researchers who reported their findings internally to their institutions. I am one of those. First, I was a witness to alleged wrongdoing, then I was a data sleuth, and then I reported the results of my sleuthing. In doing so, I inadvertently became a whistleblower. At least a third of my published account is the result of meticulous sleuthing (see for example, https://thehousewillwin.com/10-million-dollar-mentor). About another third is a step-by-step analysis – a blueprint – of how research wrongdoing is covered up, and yes, about a third pertains to retaliation. But my analysis reveals that retaliation is a fundamental component of the coverup, designed to discredit the original sleuthing work. And it is highly effective. Media outlets are reluctant to cover whistleblower accounts because the sleuthing part is dense and uninteresting to the general public; and all outlets seem to wrongly assume there is a personal agenda that increases their risk of defamation. The result is a category of data sleuths whose work is treated as not worth the risk, even by outlets dedicated to research integrity. Whistleblowers are data sleuths who have been excluded from the conversation.
Exactly! We need to still survive and maintain our jobs. We see “misconduct” all the time and report it to the authorities when possible. Our contributions are never recognised as we remain anonymous. Support us as well.
Bravo !
While great efforts by Dr. Bik and other sleuths have resulted in some corrections and retraction of bad-science papers (and exposing some charlatans), it reveals how insufficient and poor quality has been the peer-review process in journals.
On other hand, the focus has been mostly on Health & Medical Science related papers, leaving so many problematic papers in other sectors untouched, polluting the literature. As a researcher in materials science, I have seen so many flawed papers; I could only influence publication of those I was involved as peer reviewer.
Once, I reported to journals E-i-C a couple of papers deserved retraction due to cheating. Neglecting my legitimate concerns, they responded that, “These papers have already been reviewed, no need to another review”! Dr. Bik has been so defiance to interact with such irrational (and corrupt in some cases) people running scientific journals as well as some very powerful authors for many years with no legal protection and support.
I must say on a personal level, Dr. Bik’s work deserves much praise but she is also a kind, down-to-earth and accessible individual who was available via email and Zoom to meet with me, answer questions and provide guidance. Thank you, Dr. Bik, for being there 🙂
It’s incredible how controversial the statement “I want science to be accurate” is in the current climate. Dr. Bik does great work!
I wonder how many other scientist’s eyes were drawn to a suspicious looking image and then didn’t act on those suspicions. It’s a credit to Dr. Bik that she didn’t just pass over it, she investigated further, and she attempted to correct the record. That’s a true contribution.
I just want to say you have been a personal hero of mine. I don’t think people understand the undertaking you kicked off at great personal (and potentially professional) cost.
Somewhere along the way the body of science seems to have lost sight of what makes it beautiful and actually constructive to the world. It’s how we discover truth and grapple with it, not something to prop up an ideology or a career. Some seem to have an entitlement that we can take a percentage of the tax payers paycheck and we deserve it because we got a PHD, and it’s more like the taxpayer entrusted us and it’s been betrayed over and over while some want to keep looking the other way to “protect the institution.” Like what happened with the Catholic Church, Oennn State Football or other examples that simply won’t end well.