We’ve all heard horror stories of lab disputes that can quickly spin out of control. (Such as a graduate student obtaining a restraining order against his supervisor, which we covered earlier this year for Science.) Naturally, prospective students want to do their homework before committing to a particular laboratory or supervisor. A new website, QCist, is trying to make that process easier, by letting students rate labs. It’s still new – only several dozen lab heads have been rated so far, mostly from the U.S. – but founder and Executive Director Qian-Chen Yong has plans for it to grow much bigger. We spoke with Yong, currently a research fellow at the Cancer Research Institute, Baylor Scott & White Health in Texas — who completed a postdoc at Texas A&M Health Science Center and a PhD at the National University of Singapore — about the plan to keep the site from becoming a place to smear a tough boss’s reputation.
Retraction Watch: What inspired you to create this site?
Qian-Chen Yong: We want to help create great teams of scientists with similar interests and goals, while filtering out those researchers who do not maintain standards of lab practices and personal characteristics, and are actively hindering the next generation of scientists. Creating productive, efficient, positive environments in which scientific progress can thrive is our main goal.
The most recent specific event that motivated me to create QCist.com was when I was searching for my second postdoc. It was a tedious task to search for my ideal mentor. Some labs have up-to-date websites, some departments provide descriptions of their faculty, but some mentors are only found through networking at conferences or “cold-calling” to see if they have any openings in their lab. What I really needed was a database that I could search and shortlist the mentors whom I may share common goals, and then contact them to talk more about the possibility of joining their lab.
Another issue is that, quite a few of trainees I know were physically or verbally abused by their mentors. Some of these abusive mentors are well-funded and regularly publish in high impact journals, and so they are usually “protected” by the department/institute. This made me realize that if there was a website that allows trainees/researchers to comment about those abusive mentors without direct repercussions, other prospective trainees may think twice before making a decision to go to a lab that looks good from the outside (good grants and publications), but may have trade-offs in terms of personal development.
The other important factor is that some trainees have been receiving incorrect teachings regarding experimental methods and study design. We already have issues with reproducibility in science, and if these misleading practices continue, the integrity of the next generation of scientists will be polluted, which will likewise be passed down to the next generation, and so on. Therefore, ratings and reviews of scientists on QCist.com will facilitate trainees to identify a good scientist with great scientific integrity to be their mentor.
RW: How is the site different from ratemyprofessor.com?
QY: RateMyProfessor focuses on college professors who teach undergraduate classes; they are generally scored based on their classroom teaching and how difficult their exam questions were. QCist.com focuses on scientists who are also mentors for undergraduate and postgraduate trainees, including Ph.D. students, master students, postdoctoral fellows and MD/PhD trainees. QCist.com scores scientists based on qualities important for laboratory settings (mentorship, communication, data transparency). Reviewers (usually past/present trainees, co-workers, research assistants) can assess the qualities of scientists by answering a series of multiple choice questions, thereby providing information to prospective trainees regarding those qualities that are important for career development. For example, a medical student is interested in a summer research project; a good mentor for this individual might also hold an M.D. or have an active clinical/translational research program, but up-to-date comments from trainees will allow an assessment of what this mentor will actually allow a trainee to do. Our hope is that the more trainees review a mentor, the more the mentors’ strengths and weaknesses can be used to match up with a trainee that has a specific talent or need.
RW: How many mentors have been added to the site so far? What countries are they based in?
QY: Since we launched the website in April, nearly 70 scientists have been added to our database. Most of them are in the United States, and a few are from Europe. Right now QCist.com will focus on US-based scientists, because of the large number of top institutes that attract thousands of graduate students, postdoctoral fellows, and exchange scholars from around the world.
RW: How do you plan to moderate the site?
QY: Moderation is the most important aspect for the success and trustworthiness of the website. Right now, we heavily rely on users to flag any inappropriate or potentially fake ratings. When a user contacts us about a potential problem, we will investigate the user who left the comments if the comments indeed include serious allegations. We will ask the user to either have the evidence ready or amend the comments. If no response is received from the user, we will remove the entire rating. In addition, we also implemented IP restriction to prevent spam ratings. So each IP address can only register for one account. Therefore, each IP address can only rate one scientist one time. We believe, for now, these measures should be good enough to moderate the reviews and ratings of scientists in QCist.com.
RW: How can you prevent people from using the site to just bad-mouth professors they don’t like?
QY: Defaming a professor without any evidence, or for personal reasons, is not something we will accept. However, we want to promote honesty so that, over time, we are improving relationships between mentors and trainees. Therefore, everyone can criticize mentors who they don’t like, as long as they are being truthful in their report of what was seen or experienced. Ideally, a popular (and truly good) professor should receive multiple reviews. So one or two “bad-mouth” reviews will be drowned out by other good reviews. In addition, we also encourage the reviewer to leave words in the “additional comments” section to make the review more authentic. This way, a prospective trainee will be able to tell a bogus review from a truly authentic one. Therefore, we don’t think that one single “just bad-mouth” review is going to affect the overall feeling and rating of a professor.
RW: Anything else you’d like to add?
QY: QCist.com is in its infancy, so getting the word out about what we are trying to accomplish is key! The most important growth we currently need is to get more users to add and review scientists they know in order to generate a database that is large enough to be used as a reference for prospective trainees. To do that, we have started to volunteer at conferences around us, actively join seminars, and speak to people who may be interested in promoting the concept and goals of QCist.com.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
“Another issue is that, quite a few of trainees I know were physically or verbally abused by their mentors.”
I’m sure that a blog/website is NOT the best place to report physical abuse by a mentor.
No university would tolerate this type of activity and EVERY university is going to have a formal process to deal with physical abuse of a student.
I agree that the website should not have been, and should not be, the place to report physical abuse. However, to my experience, there are institutions choose to resolve the abuse issue internally as much as they can until it is exposed by media. So at least this website can be used to alert others regarding this issue.
The noose around the neck of academic medical research tightens. But who or what will topple the chair? I had no idea that recruitment of foreign students could be a sign of problems instead of opening up opportunities on a global scale.
Until such a site comes into play, there are a few alternate ways; I tell graduate students in my program to search both Retraction Watch and PubPeer for the prospective mentor’s name. If the name is present, the student can lookup the comments and make an informed decision on suitability of the mentor. Comments on PubPeer or RW do not necessarily mean bad press – thus the student needs to make an informed decision. Of course, it is difficult to identify whether the prospective mentor is a foul-mouthed-verbal-abuser, a slave-driver, or someone who pits multiple postdocs and graduate students against each other on the same project to “gain” maximum productivity.
For this to be even remotely informative, the response rates would have to be astronomically high, which seems unrealistic. Let’s say 10% of all trainees participate (probably a much higher rate than what a well-established review site like ratemyprofessor.com is able to achieve). For a lab that trains 10 students over 5 years, this would correspond to one rating, which is essentially just random noise. Moreover, lab dynamics constantly change, and training environments that are great for one person might be a poor match for someone else. I don’t see an alternative to carefully reading papers from a lab of interest to estimate productivity and rigor, checking funding levels (with e.g. NIH reporter), contacting present and former lab members (or graduate program administrators, who often keep track of the career paths of former students), and most importantly, scheduling an interview and asking focused questions about research, training, and career development.
I agree that for a small lab, it’ll be hard to gather many reviews. But just one or two will be enough to say something about the lab/mentor. Moreover, we agree that some important steps when searching a mentor are not replaceable, including carefully reading the paper from the lab, checking funding levels (very tedious task), contacting present and former lab members (it’s not always easy to find their contacts), and scheduling an interview and asking focused questions. But QCist.com, if the database is large enough, will give you a lot more information about the mentor and lab atmosphere when you are making a decision.
What methods are in place to verify that trainee X was indeed in the lab of Dr Y? What’s to stop me from posting anonymous and mostly negative comments about someone?
“In addition, we also implemented IP restriction to prevent spam ratings. So each IP address can only register for one account. Therefore, each IP address can only rate one scientist one time.”
Can’t I just spoof my IP address (perhaps I want to be Canadian today) and create multiple accounts using different email addresses? Of course, an unusually high number of postings about a particular scientist within a short span of time would look suspicious, but if one remained cognizant regarding frequency and duration between postings, they could likely add two or three reviews undetected, for better or worse.
I agree- It does sound simple for an anonymous poster to leave negative comments, whether warranted or unwarranted. However, when verifying identities, will you have a problem similar to PubPeer? If a trainee is still working in a lab and needs to finish, they do not want to risk retaliation by using their real name. On the other hand, if the site verifies their identities, but allows them to post their review without a name, what’s to stop a scientist from suing the site claiming that the negative reviews caused them to lose a highly valued lab recruit? That’s a less likely scenario, but I thought I’d play devil’s advocate for a moment. What’s the optimal solution? Allow everyone to leave reviews but add a mark to identify the reviewers that have chosen to be verified as their reviews may hold more weight? As with other websites that allow reviews, sometimes the best option is just to read the reviews, consider the overall evidence, and follow-up any suspicions with current and past lab members, if possible, in order to make an informed decision about joining a lab.
Finally I share a similar concern to a poster above- the number of reviews, especially for smaller labs, will be small, possibly too small to draw too many conclusions. If the site becomes more popular, this may change…but it’s an interesting idea if it can be executed properly.
The idea will not work. There was a similar website for high-energy physics postdocs to rate their bosses but (I believe) the site went under after just a short amount of time. The reason RateMyProfessor works is that undergraduate classes are big such that it’s possible to collect a large number of samples. Also, students can post a rating/comment without worrying about being identified. This is not the case for PhD students and postdocs working in a lab.
The best way to prevent abuse in the lab is to reform the funding system: Give funding to graduate students and postdocs in the form of fellowships. They bring their own money to the lab and are free to “jump ship”. Under this system, a good mentor will have his/her lab well staffed while a bad mentor will be phased out.
Miguel,
Yes, your solution makes the most sense in terms of likely cost-effectiveness. Put the consumer/student in control of decisions. The existing hierarchical authority-based system is not cost-effective because there are too many “middle men” taking a cut of the resources before the ultimate consumer gets his/her share. So, in my view, eliminating the middle men is the optimal way to meet consumer demand, something akin to buying “wholesale.”
The policy, in my view, should be one that enables the ultimate consumer to bypass as many as possible of the go-betweens in the purchase of the desired educational product. BTW, autonomous consumers are less likely to be exploited or abused by educational providers. Autocratic and abusive lab directors need to be brought to heel, and your policy prescription appears to be the most likely the way to make it happen.
That said, the politics of making the change will be difficult. I think those now in control will resist mightily!
I agree that your idea about reforming the funding system is much better than simply relying on QCist.com, but before any of us, who really concern about the scientific system, become the President (or at least Senator), I don’t think this is going to change easily. Moreover, even if your idea is happening, it’s easier to “jump” if there is a database to show who is a good mentor and who is a bad one. You can’t just keep jumping ship until you find a good match or good lab because you’ll be wasting too much time before the fellowship ends.
I am just doing what I can do (by setting up this website), and hope that we can change the system a little by a little. So your support is very much appreciated!
I’ve just been reading “PhD programmes: Doctorate deluge” in Nature (https://t.co/WCTv7ckUsK). As long as there are many more newly coined PhDs than open jobs, a bad lab will look great to the young, hungry scientist.
Depend on what kind of bad lab. If it’s a lab that has been involved in scientific misconduct, you’re risking the lab being brought down by Retraction Watch/Pubpeer, or other whistleblower. In my opinion, it does not worth it. I am sure that there are still many high quality lab with great scientific integrity. If it is called “bad” lab because of super micromanaging mentor or super busy mentor. Then it’s up to the trainee whether she/he wants to join those kind of lab (because if you work hard, you may publish well. Therefore QCist.com will help the trainee to make better decision.
The website in Chinese (https://www.mysupervisor.org) where students rate and comment their supervisors anonymously could be a better choice at current stage.
In China, some supervisors severely abuse the graduate students by forcing them to work in name of academic(no matter that’s valid or not), giving no guidance while pretending to be professional, delaying student’s graduation, or even humiliating the students. There are system flaws in the Chinese university system that the students can’t protect themselves if the supervisor abused them. The supervisor is like an emperor in a lab who solely decide whether the student’s academic achievement is enough for graduation. They refuse to debate but only give orders.
Essentially, the supervisors are taking advantage of their superiority as a professor to force the student to work for them without any employment contract signed between both sides. They can do this because it is up the the professor’s decision if the student can submit his/her thesis to the school.
In European countries it is clearly defined that PHD is a job. As fas as I know, they have salary, they are protected by the labor law, they could quit anytime they want, and prolonging the candidature without salary is strictly prohibited. Whereas in China, or in other asian countries like Singapore etc., PHD is not well regulated by law.