Replication probe finds ‘statistically improbable data’ tied to institute in Bangladesh

A Bangladesh-based organization focused on development economics and its founder have been churning out papers filled with misstatements, inconsistencies, ethical lapses and “statistically improbable data,” according to researchers involved in an ongoing effort to replicate the work.

One journal has already retracted a paper for falsely claiming to describe a randomized, controlled trial and data collection that failed to adhere to the journal’s ethical guidelines. The study, published in the European Economic Review, was retracted following a March 11 report from the Institute for Replication, or I4R. The group is conducting a broader probe into the Global Development & Research Initiative (GDRI), the organization that implemented the intervention described in the paper.

GDRI’s founder and the study’s sole author is Asad Islam, a developmental economist at Monash University in Australia. Since 2022, Islam has received over $2 million in funding from the Bill & Melinda Gates Foundation, the World Bank, the Asian Development Bank, and other organizations, according to a copy of his resume. Islam did not respond to multiple requests for comment on the retraction or the broader concerns about the work. But in a statement posted to his now-deleted account on X, he wrote: 

Given the volume of requests and the complexity of the issues raised, my team and I are diligently reviewing every point to ensure a comprehensive and accurate response. At this stage, these remain allegations, and I urge everyone to allow me the necessary time to address them fully.

Abel Brodeur, the chair of I4R and an economist at the University of Ottawa, said the probe into the GDRI began in November, following a tip from a whistleblower. 

Brodeur enlisted expert volunteers and assigned them to small teams to review five studies with ties to GDRI. Their job was to review the publicly available data and analysis code and look for any irregularities.

I4R’s review has since expanded to more than 30 studies and working papers listed on Islam’s personal website, resume, and other sources because of overlap in the data sets and concerns that have arisen along the way, according to Brodeur. I4R has requested data for many of these projects, but it only attempts to reproduce papers once they have been published. The group has completed 10 reports so far, releasing five of them along with a response from the original authors.

“The I4R has reproduced hundreds of studies over the past three years, and we’ve never seen anything of this scale,” said Brodeur.

Since the probe began, Islam and his coauthors have withdrawn two studies that had been accepted for publication at Economic Development and Cultural Change and Journal of the European Economic Association and another study that had recently been submitted, according to I4R’s correspondence with the journals.

Several of Islam’s coauthors have either withdrawn their names from published studies, citing the problems raised by I4R, or they have indicated that Islam is no longer involved in research initiated with him. For example, when I4R requested a replication package for a manuscript in review on gender norms in the workplace, development economist Emily Beam of the University of Vermont responded, “Asad [Islam]… is no longer involved with this project, and the three of us are completing it without him.” (Beam did not respond to a request for comment.) 

For a 2021 study published by PLOS One, I4R was unable to reproduce the study’s result regarding the correlation between food insecurity and mental health during the COVID-19 pandemic. After I4R shared its findings with the authors, all three authors, including Islam, withdrew their authorship, according to correspondence with I4R. 

Asked about the status of the paper, David Knutson, head of communications for PLOS, said that he was unable to comment on it because it was an “ongoing case,” but he noted that “withdrawal of authorship is not considered by PLOS or by COPE to be a suitable means of responding to concerns about the integrity, validity, or reproducibility of published results.”

The one paper retracted so far, published in 2019 European Economic Review, an Elsevier journal, has been cited 21 times. It claimed to show parent-teacher meetings were a “low-cost strategy” that could produce dramatic gains in students’ test scores.

When Brodeur’s team examined the Islam’s statistical analysis code published with the original paper, they noticed an annotation that read “try to balance the baseline.” 

“This remark prompted us to investigate the study closely for data irregularities, especially in the baseline data,” Brodeur and his colleagues wrote in their report, which the journal published April 3. Indeed, the baseline scores appeared to have been altered to generate significant results during the follow-up. As the team put it:

Our analysis uncovered several data irregularities that severely question the validity and, thus, the credibility of the original study’s findings. First and foremost, we find that the school-level treatment allocation has a clear non-random spatial pattern. We demonstrate using permutation tests that the observed allocation is extremely unlikely to have occurred by the school-level randomization described in Islam (2019). Second, baseline test scores for the same students differ systematically across datasets, with discrepancies varying between the treatment and control groups.

In his response to the replication report, Islam said he stood by the study and its results. He claimed the contractor on the project had failed to follow his instructions to randomize the schools and stated the comment about balancing the baseline was written by an undergraduate student and was “never acted upon.”

When Brodeur’s team asked Islam to provide the underlying data and code from another study, Islam pointed them to the GDRI, which he described as “an NGO that helped us collect the data, but it does not store the data for an extended period.” He added, “we might not have alternative options available at this time.”

Islam is the founder of GDRI and its predecessor, the Initiative for Development Research and Implementation, according to archived versions of the organization’s website, his resume and his personal homepage from 2018 and 2019.

“I grew up in a rural village in Bangladesh,”  he wrote on his personal homepage. “I believe my childhood in a remote rural village and experience working with NGOs and development research organization [sic] helped me to understand better of what I do [sic], and what works in development!”

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].