Do researchers need a new “Craigslist?” We were recently alerted to a new online platform called StudySwap by one of its creators, who said it was partially inspired by one of our posts. The platform creates an “online marketplace” that previous researchers have called for, connecting scientists with willing partners – such as a team looking for someone to replicate its results, and vice versa. As co-creators Christopher Chartier at Ashland University and Randy McCarthy at Northern Illinois University tell us, having a place where researchers can find each other more efficiently “is in everyone’s best interest.”
Retraction Watch: What inspired you to create StudySwap?
Christopher Chartier and Randy McCarthy: StudySwap was inspired by a few ideas. Recently there has been a lot of discussion about how scientists can improve the information value and reproducibility of their research. In our home field of psychological science, this typically means things like collecting larger samples, testing whether effects generalize to different contexts, getting more precise estimates of effect sizes, etc. One of the ways that researchers have worked towards achieving these goals is to “crowdsource” their research efforts. Essentially, you have two or more researchers pool their modest resources into larger collaborative projects.
Meanwhile, at the 2016 meeting of the Society for the Improvement of Psychological Science we extensively discussed how to make replication easier and more prevalent. One idea that seemed quite promising was an online exchange platform to facilitate pre-publication independent replications (PPIRs). Others have called for similar online spaces (even here on RW, no less!), but no one had created a centralized place for researchers to find each other for these replication attempts.
RW: Why do researchers conducting replication experiments need something like StudySwap?
CC and RM: Despite the potential benefits of collaborative projects, there currently is not a good way for researchers to find one another. Thus, the primary function of StudySwap is to help researchers connect. Many current replication attempts are viewed as adversarial and can become personal. While the importance of reproducibility is a central tenet of scientific pursuit, authors of original findings are not always excited that another researcher has decided their particular finding needs another look. Replicators can feel like their efforts are unwelcome, and some researchers (rightly or wrongly) can feel unfairly targeted by the “replication police.”
On StudySwap, those attempting the replication find a pro-active and willing original author. This allows both parties to agree beforehand on issues of authorship, methodological details, open data and materials policies, and anything else they find important and mutually acceptable. This process is also a great benefit for those posting their studies on StudySwap for others to replicate. They are involved in the planning process and can ensure that critical details are passed along to the replication team. There is also an important signaling function for authors of original findings. By posting on StudySwap that they are seeking an independent lab to corroborate their findings, they can announce to the scientific community that they take reproducibility seriously and are ready to engage. This type of dedication to getting things right, instead of simply seeking the quickest possible publication, can come with very positive reputational outcomes.
RW: How many users are there, at the moment? What are your goal numbers?
CC and RM: StudySwap is in its early stages. We launched in March and in our first month we’ve had over 300 researchers follow us on twitter to stay up-to-date on StudySwap posts, four researcher posts have gone “live”, one PPIR has been initiated, and we have well over a hundred downloads of StudySwap posts and instructional materials. Further, we have been absolutely overwhelmed with the enthusiastic response we’ve gotten from researchers in many subfields of psychology, as well as a diverse set of other disciplines of science such as microbiology, genetics, chemistry, and economics to name a few. The challenge we now face is turning this mountain of interest and support into actually changing normative practice in science and getting researchers to use StudySwap to help in this process. We would like to see StudySwap become a commonly-used tool in the scientific kit. Our aim is to facilitate 100 collaborations in our first year. If we build sufficient early momentum, we believe StudySwap will thrive on its own as researchers experience or observe the benefits of these lab-to-lab collaborations.
RW: Do you envision this as becoming a resource for more than just replication?
CC and RM: Absolutely! PPIR facilitation is just one of many uses we envision for StudySwap. It can be used for any project that could benefit from collaboration. There are many ways that finding willing collaborators can be turned into meaningful scientific contributions. For example, many researchers have resources that are underutilized. Perhaps an expensive piece of experimental software lies dormant 23 hours per day, or a difficult-to-recruit sample of human subjects (children, CEOs, members of underrepresented groups, etc.) comes to the lab for only a 5-minute study. Other researchers would love to collaborate and gain access to those valuable resources or subjects. It is in everyone’s best interest—researchers, the public, and even the participants—for those resources to be used more efficiently. StudySwap allows these parties to find each other quickly and easily to initiate a collaboration. We’ve put together a set of example uses here. The collective intelligence of scientific researchers is mind-bogglingly huge. We are excited to see how these smart people use tools like StudySwap to achieve their research goals.
RW: What are some of the potential pitfalls of the system, and how do you plan to try to address them?
CC and RM: The potential pitfalls are the same as in any collaboration. Because exchanges made via StudySwap will require a great deal of mutual trust and respect, researchers will need to vet possible collaborators to ensure they are someone they trust. Further, collaborators will need to come to agreements on open data policies, have a system in place for full ethics approval at all institutions involved, and decide if they will follow other open science procedures such as pre-registration. We hope that the use of open and transparent exchange agreements, which are essentially publicly posted collaboration contracts, will be effective in ensuring that collaborations via StudySwap are positive and productive experiences for all involved. Perhaps the biggest barrier is the culture of the research community. We strongly believe in the potential of multi-site collaborations and we hope that StudySwap is a tool to help facilitate them. However, only time will tell whether this tool is ultimately seen as useful to researchers or not.
We continue to actively improve the site and associated support activities. We would be thrilled to hear from researchers or any consumers of science with their feedback, ideas, and questions. Keep an eye on our twitter account for updates. We have a lot of exciting plans for this summer and will share developments as they occur!
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.