We were struck recently by a paper in Scientometrics that proposed a unique way to fund scientists: Distribute money equally, but require that each scientist donate a portion to others – turning the federal funding system into a crowd-sourcing venture that funds people instead of projects. The proposal could save the inordinate amount of time scientists currently spend writing (and re-writing) grants, but would it actually work? First author Johan Bollen, of Indiana University, explains.
Retraction Watch: You propose something quite unique: Fund everyone equally, but ask them to give a fraction of their funding to someone else. Is the idea that scientists most respected by their peers will “earn” a higher percentage of funding, and everyone is just acting as reviewers?
Johan Bollen: To some degree, yes. Every scientist in this system becomes a small funding agency themselves. The size of the fund at your disposition, however, depends on how much you receive from others. Everybody receives the same base amount, but also funding from other scientists. Since you distribute a percentage of everything that you receive, some scientists will have more funding to distribute than others. So if you are a well-respected, trusted scientist, people will be more likely to give you funding, and you will become a more important “funder” in the system.
This makes sense because scientists are supposed to know who is doing the most interesting or promising work. The system aggregates this information by directing the flow of funding directly and indirectly towards those scientists who then become more important funders in the system.
Retraction Watch: You actually simulated how this might work. Can you explain your procedure, and why it serves as a proxy for this funding system? What are its limitations?
JB: We simulated the system to see whether it produces sensible results and under which conditions. Since we don’t know beforehand who a particular scientist would direct funding to, we relied on citation data from the Web of Science. Our simple assumption was that a scientist would be more likely to direct funding at someone whose work they had cited frequently, because a citation indicates that you found someone’s work influential. We counted the number of citations that pointed from a given scientist’s publications to those of others. Using that data we tried to estimate how much funding they would direct to other scientists. For example, let’s say in 2004 I cited two people. One person I cited 5 times, the other 15 times. The assumption would then be that I would distribute 5/20 of my funding to the first, and 15/20 to the second. We ran our funding system in this simulation for 20 years. The result indicated that under certain reasonable conditions (donation fraction of 50%) the distribution of funding would not be very dissimilar to that of the present system. We know this because we also had funding data for the U.S. National Science Foundation and National Institutes of Health for the same period that we ran the simulation for.
The limitations of this simulation are, of course, that we don’t know whether citation data is a good proxy for funding preferences. I might have cited scientist A frequently because they happen to have done a lot of research in my area, but I might not think it is very interesting. I could instead direct my funding to scientist B whose work is very interesting but only just getting started so there is little to cite. That’s the beauty of this system. It’s easy to reward innovation.
We therefore caution in the paper that our use of citation data is not a recommendation; we do not propose funding be distributed on the basis of citation data. We are simply using it to estimate in a simulation that which is essentially the discretion of each individual scientist.
RW: Since this is a fairly radical change from the current system, I think readers would raise many questions about its practicality. For instance, how could we ensure that early career researchers, who aren’t well known by many outside researchers, will get enough money to get a robust research program going?
JB: The present system certainly also presents challenges to early career researchers. When you are just starting out it is difficult to compete for funding. The writing and submitting of proposals takes a lot of effort away from building a robust research program, and success rates are quite low in certain programs.
In our system everybody receives the same base amount. This assures every beginning researcher a yearly unconditional source of funding to work with. They don’t have to compete for this funding, they don’t have to write and submit proposals, and their work does not have to meet with the approval of anybody to receive this base amount. That’s a significant advantage for an early career scientist.
It is however true that this base amount might not be sufficient for everyone. So early career scientists should be encouraged to get out and make their community aware of their work, vision and plans, so that they can receive additional funding. Special programs might still be needed, but one of the advantages of our system is that biases against early career scientists and underrepresented groups can be detected and mitigated, e.g. by changing donation percentages, base amounts, and encouraging the community to give more to those groups — e.g., there can be a specific donation option for “women in science” or “early career scientists”, almost like straight-ticket voting.
RW: How can we ensure that scientists don’t just fund their colleagues and friends, and distribute the funds based on merit alone?
JB: First, donations should be completely anonymous to prevent people from colluding. Second, the system would also need strict Conflict of Interest rules and restrictions such as not being allowed to donate to colleagues in the same group or institution, etc. These rules can be automatically enforced. For example, when you log in to the donation system and start entering names, the system can automatically detect that you are giving to a colleague, someone that you have co-authored a paper with, donated a lot to before, etc. and “gray out” those names. In addition, the system generates data on who donates to whom allowing the management of the system to detect collusion and infractions. We can’t completely rule out any problems, but I think they are certainly manageable, and perhaps to a greater degree than the present system where small review boards make sometimes inscrutable decisions on who either wins or loses in a given competition.
RW: You note that scientists would have to somewhat “pitch” their research to other scientists, through communications and conferences, etc. Are you at all concerned that the time and effort spend campaigning for peer funding could start to approach what scientists currently spend in the current system?
JB: The numbers seem to suggest otherwise. We cite an analysis in our paper showing that Australian researchers spend five (!) centuries worth of research time on proposal preparation. Assuming the U.S. population is 14x larger, you can imagine the enormous costs of the present system. Imagine what could be done with that amount of research time and effort instead.
Since scientists already attend conferences, participate in workshops, visit with colleagues, post to blogs, etc. I don’t think they need to do any extra work to make their peers fully aware of their vision, plans, and present work. What might change however is how scientists approach their participation in conferences and workshops, and how the role of publications might change. That will be difficult to anticipate, but it is very unlikely all of this will require the amount of time and effort presently going into proposal writing, submission, reviewing, project admin, reporting, etc.
RW: In lieu of overhauling the entire federal funding system, is there a more practical way to put this proposal into practice, perhaps on a smaller scale?
JB: We’re certainly not proposing to overhaul the entire federal funding system. A lot of really good work is being done by our funding agencies, and we should be grateful for the hard work that their many scientists and administrators are putting into supporting excellent science.
The system can very easily be implemented on a small scale, either for a specific population or group of scientists, or with a smaller overall budget to distribute. If results are good and we learn more about making these systems better, we can gradually extend them to greater populations and/or increase the overall budget. It’s actually very flexible that way. One part of the funding system could remain competition, project proposal-based, and the other could function according to our proposal.
I should mention that the Dutch parliament recently approved a motion directing the Dutch science foundation to start experiments with this system. The idea is that you can gradually scale things up over time as you gain experience with what works and what doesn’t.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
If everyone receives the same base amount, wouldn’t that require scientists working in fields with higher costs (where expensive lab equipment or field work is necessary) to always be short, and rely on getting funded by their colleagues in inherently cheaper disciplines?
Hi Eva, good point. I think it might be possible to adjust the base amounts for scientists in certain fields, but scientists in fields with higher costs could indeed be more reliant on “donations” from other scientists if the equal base amount were to be insufficient. On the other hand, these same scientists are now dependent on actually getting funded via the traditional grant peer review system, and that system is all-or-nothing. You either get funded or you don’t. In the proposed system everyone at least receives the base amount.
All nice in theory, will not work in reality due to an unaccounted for factor – human nature.
How does human nature _not_ factor in the present grant proposal review system? In fact, if through some historical accident I were proposing this “new” grant proposal review system, your reaction could be exactly the same. “All nice in theory, but it will not work due to human nature. Review panels will simply only award grants to the researchers they like the most and their friends and colleagues. “
“Our simple assumption was that a scientist would be more likely to direct funding at someone whose work they had cited frequently”
This is naive. Run another simulation in which individuals form coalitions to maximize their own interest and see what happens. This is more realistic. It would not be pretty.
your comment hits the nail on the head. citations gangs will just turn into funding gangs. game theory may help to model such situation.
Might I add that 1) we only used citation data in our simulation to approximate decisions that scientists *might* make, but are not suggesting this system be publication- or citation-based at all. Scientists should make their own decisions, and 2) donations should be anonymous and subject to strict COI rules, such as not being allowed to donate to present and former colleagues, co-authors, etc. These rules are also in effect for grant peer review for the exact same reason. Whether that is enough to prevent collusion is not a 100% certainty, but if it isn’t, it is almost certainly not enough to prevent collusion in our present grant peer review system.
Desperate times call for desperate measures.
Stanish, thanks for the insightful comment. Our methods and data are fully described in the paper, so please free to conduct your own simulations and prove your assertions. Just curious: how would you realistically and not-naively simulate people forming coalitions to maximize their own interest? Also, I think you should simulate what would happen to grant peer review under the same conditions.
Johan Bollen, what’s stopping someone from funding, or receiving funding from an ethically suspect researcher? I see multiple issues with this proposal at the practical level. What selective criteria would scientist’s use if not a citation or related field? Surely this proposal is fraught with bias, because inherently a scientist would neither seek to fund or seek funding from a competitor, or from an academic fraud, even though some of them are very well funded, so selection is heavily skewed from the word go. Can external scientists boycott a funding partner? Would funding agencies that already fund projects be happy with this socialist model? Would funding be enough to satisfy the hungry APCs in OA publishing? Would funding be allocated to the PI, senior author, last author, first author, or corresponding author? What about papers with 6 co-first authors, who gets a slice of the cake? What about papers with 1000 or 2000 authors, who would fund who?
Hi Jaime, our simulation was based on citation data, but we stressed that it is NOT intended to be a publication- or citation-based model, so author order, citations, etc don’t factor in. Each scientist individually choses whom to donate the mandated part of the funding they receive to. At that level it is personally biased, but funding circulates and converges on a distribution that is supported by the entire community. When a small review panel decides to fund a proposal and not fund others, that’s also decision coming from their personal and professional expertise and opinion, but it’s an all or nothing situation where if you are unlucky with that particular group of reviewers you can spend months writing proposals and not receive a dime of funding. I guess sometimes “ethically challenged researchers” might receive funding, but that’s also perfectly possible in the present peer review system, because those researchers don’t exactly carry a card or hat that says “ethically challenged”. This is something the scientific community must detect and correct by making different decisions. The other issues you mention about funding not being enough to cover specific costs, how is this any better in the present system where if your proposal(s) do(es) not funded you get $0? One last thing: the system is not socialist. Everybody does receive an unconditional base amount that is equal, but the system can generate any level of inequality depending on the “donation percentage” mandated. Please have a look at our paper.
How do you define “everyone” within this system? I mean, the list of all the people who should receive the base funding? Like, all people with PhD inside the country? All people who published in the past 3 years? (cf. publish or perish) Or all people hired by a university for a research position? (and I can see early-career researchers badly racing for these positions, probably badlier than now. Also, the amount of time they’ll spend on sending applications for these positions. And what about the industry researchers?)
I would be afraid that the system gets a priori too exclusive.
this indeed requires some thought. Funding agencies have restrictions as well. We can broaden or restrict the requirements to best fit the community.
I think this is a very important point. Defining who gets to play the game cannot be as simple as “they have a position at an accredited institution”. Such a low bar would simply result in Universities hiring thousands of new faculty, to maximize their skin in the game. If every new hire came with a guaranteed base level of funding, faculty ranks would balloon, and it would be a race to the bottom wherein [starting salary = guaranteed base funding – indirect costs].
In the current system, the difficulty of playing the game is a major deterrent to entering it. If everyone who wants to play is guaranteed a base level of funding, all those post-docs who left for industry or teaching or “alternative careers” would decide to stay in academia and claim their guaranteed share. The classic “too many mouths at the trough” problem would get infinitely worse.
The only way this could work is to impliment a pre-qualification stage, to determine which researchers are eligible to participate. Again though, Universities would game this to maximize their exposure – for example if only full professors (who have achieved a certain level of credibility, YMMV) were eligible, Universities would just promote all their faculty to full professor rank.
Maybe we could build a wall (it’ll be huge, let me tell you!) The graduate students WILL pay for it, and only scientists demonstrating True American ValuesTM would be allowed in to claim their base level funding.
This sounds like a fascinating idea. (I’m the child of two research mathematicians.)
I totally agree!
Are human beings capable of doing like Jesus when he did the “Feeding of the 4,000”, with seven loaves of bread and fish miracle?
My guess? We’d continue to see sexism/racism playing out. This already plays out in terms of mentoring, sharing data, etc. I’ve had other thoughts gift culture with data and it’s effects on diversity over at my blog last year with a number of links to articles demonstrating the potential ill effects. http://hedgehoglibrarian.com/2015/01/08/effects-of-academic-gift-culture-surrounding-data-on-diversity-a-question/
Hi Abigail, this is definitely a serious concern. First, the system can generate data that allows these biases to be quantified (e.g. 50% women receiving 20% of donations?) and, second, they could be corrected by adjusting the system’s parameters, e.g by applying a multiplier. One could even imagine a donation option in the online system that would allow a scientist to distribute one’s donation to underrepresented groups or specific groups of scientists that merit consideration. In the present system these types of biases are very difficult to detect and correct, but I know it is of great concern to our funding agencies who are working very hard to mitigate these biases.
While this is a great idea, I think this would bias funding away from primary caregivers (which tends to be enriched in URM populations). Here, you propose researchers attend conferences, workshops and visits, but researchers who have family responsibilities will attend fewer conferences overall. Additionally, current URM faculty are already taxed by expected over-participation at institutional diversity/outreach committees, which can eat into conference times.
With regards to funding consistency, how would a plan like this allow for multi-year grants especially considering the concept of a “grey out” model of not funding the same group too many times? I do like the idea of not allowing funding within the same institution, though. Maybe that approach would incentivize higher profile professors to migrate to more underserved universities.
Thanks. I am sure your parents could appreciate the notion of determining funding through power iteration.
Ah, a popularity contest. So instead of spending time writing grant applications, scientists will spend time schmoozing each other. And the valuable money will be diffused over a whole bunch of stuff, with no oversight guidance regarding direction. Academia is “gimme-gimme” badly enough already.
Everything that’s wrong with peer review would be continued, and everything that’s right with it would be discontinued.
Lord of the flies re-enacted.
Very naive idea. Far too many will support their friends/relatives/etc instead of funding by merit. This would exacerbate the known problems of peer review to the extreme.
There is some merit to this. The main issue would be new people, who know few folks, would have trouble getting funded. Others would tend to fund known quantities. The money would gravitate to well-known people.
I have been responding to most comments, but it’s getting to be a lot of work. If you have any additional questions about the paper (I recommend you read it before you comment), please send me an email.
“I recommend you read it before you comment”
So, new to the internet, eh?
Most of the arguments commenters are making against this (rise of cartels and self-dealing) are huge problems with the existing system. Without data it’s basically impossible to argue if these problems would be better or worse in the proposed new system. The eligible group would be faculty at Ph.D. granting departments. (that’s an existing limitation in practical effect anyways). It’s also clear that allocations to a field would need to be proportional to costs (biology is more cost-intensive etc.)
One thing is for sure though, more money would be given to individuals and money would be separated from “Teams”. Many prominent scientists (for example Axel Becke, who is amongst the most highly cited chemists in history) believe this would be a valuable change.
This proposal will never happen unless it can seem appealing to the powers that be. It certainly would not in its current form.
Something similar has been tried at my institution. Why would I spend the (large) amount of time to become an expert in someone else’s field so that I can review their work? Who pays me to do that? Additionally,since people won’t do this (do the basic research necessary to make an informed decision), money will get directed at people who hype their work, rather than to people who are doing things that are both fundamentally important and likely to get results.
David, can you tell me which institution and how they (or you) arrived at the conclusion that the results were bad or worse than what you would expect with regular grant proposal peer review?
Interesting idea but I don’t think it would work. You’d see an aggregation of money with a few groups.
I’d be more for doing away with the R01 bs, doing career style awards only and increasing the funding attached with those. If you are smart and doing good work, you shouldn’t have to constantly apply for grants, just go do good work! We could keep some kind of supplemental funding system on top of that (like an R01) but it should be a bonus, not a necessity for existence. My two cents…
I do see one potential problem. Suppose in 2015 I receive $100,000 in government funding and donations totalling $400,000 from other scientists. That would give me $250,000 to spend on my own research, and an obligation to give away $250,000 in 2016. What happens if I only get government funding in 2016? I end up in debt. What then?
Perhaps if we focused on more practical, more realistic ideas…
What do you have in mind?
Sorry, but anonymity requires all parties maintaining their anonymity. Regardless of the protections put in, “I’ll fund you if you fund me,” would be very easy to accomplish (my research is determining the a, b, c properties of x, y, z. You fund that project and I’ll fund yours. Just send me a description of your project). If you put a safeguard in for that scenario, then I would collude with two other researchers – I fund researcher b, b funds c, and c funds me. If everyone funds the research of others, that would likely happen ‘honestly’ occasionally, making collusion difficult to determine.
you seem to have amazing confidence in the ability of large groups of individuals to keep to agreements when free loading is rewarded and collaboration can never return more to you than you have given. All it takes is for one of those a, b, c, d, x, y, z free loaders to accept your donation and not return the favor (anonymously of course).
To disrupt this type of collusion one could simply add a rule that you are not allowed to donate repeatedly the same amount to the same people, and must distribute your donations over more than x (e.g. 5) people.