The Hard Decisions
29/09/10 02:54 Filed in: GenomeWeb Daily Scan
Submitted by S. Pelech - Kinexus on Wed, 09/29/2010 - 02:54.
As I write this, I am actually sitting on an airplane flying back home after participating in a yet another grant panel that I attended for the National Research Council of Canada in Ottawa for the last 2 days. Over the last 25 years that I have reviewed grant proposals for more than two dozen different agencies, I have observed a steady improvement in the quality of submitted grant applications. The amount of peripheral information that is required for these grant applications also markedly escalated during this period. This has created an extra burden for both the applicants and reviewers alike. It has now reached a point where I would not be surprised if a quarter of a typical academic scientist's time is spent either writing or reviewing grant proposals. This is extremely counterproductive.
I believe that most people do actually get colleagues to critique their grants and in some faculties, for example in the Faculty of Medicine at the University of British Columbia, this is a requirement for junior faculty for university approval for submission. It is also clear from recent data released by the U.S. National Institutes of Health that new grant proposals from established investigators have about the same chance of getting funding as applications from brand new faculty. Apparently, experience does not seem to carry much weight in the decision of whether an investigator receives funding, nor supposedly does their ability to write a fundable grant. After all, an established investigator must have a previous track record of success, otherwise he/she would not have lasted very long.
Typically only about 1/7 of submitted grant applications receive funding in North America. Most proposals are rated in the very good to excellent category, so it is really a crap shoot as to which particular applications are funded. Over the years, I have noticed two particularly disturbing trends that have exacerbated the problem.
The first relates to a general decline in the quality of the grant reviews if my personal experience is indicative of the current state of affairs. It seems that with the many more grant applications that have to be reviewed, grant fatigue is now pretty rampant. I understand from my conversations with granting agencies that it has become extremely difficult to find external reviewers for most grant proposals. Some agencies don’t bother anymore. This probably also explains why I am now solicited regularly by agencies from foreign countries. The lack of external reviews means that the fate of a grant application is largely at the mercy of two or three internal reviewers on a grant panel who have actually read most or all the proposal. Generally the rest of the grant panel just doesn’t have the time to read all of the submitted applications. These assigned internal reviewers may well be relatively new and inexperienced investigators, people who are not necessary experts in the subject of a particular proposal, and most certainly individuals that have to plough through a dozen or more other applications in short order and write reports. In the general panel discussion of the merits of a grant application that survives the triage process, a great deal of time is spent going over the specifics of the proposed work. (The odds are that by the time an investigator actually gets funded and towards the end of the first year of funding, the research will probably have taken quite a different course.) By contrast, the previous track record of an applicant gets pretty short shrift in the deliberations that I have been privy to on grant panels. Shouldn’t we be rewarding past success more?
The second relevant trend relates to the funding of many more and larger project grants over the last decade. Genome Canada is one example of agency in my country that is government mandated to award only large scale program grants. The typical budgets for these proposals can exceed $10 million dollars, but the applications are only about 2- to 3-times larger in content than typical proposals requesting 10- to 20-fold less operating funds. While external reviews are usually solicited and received for such large program proposals, the size of internal review panels for program grants are typically only about 50% larger than a regular project grant panel. The net effect is actually much less scrutiny of the spending of significantly more research dollars.
While it is difficult to obtain hard numbers from granting agencies, based on my own reviews of scores of program grants and their renewals, it appears to me that the number of publications (also factoring in impact) per dollar is much poorer from program grants than individual project grants. I wish that a careful analysis of the net productivity and other impacts of program verses small project grants over the last decade was available so that research funding in the future could be spent in the most-effective manner. If such results were applied, I suspect that the success rates of individual grant applications would improve markedly as would the rate of scientific progress.
Link to the original blog post.
As I write this, I am actually sitting on an airplane flying back home after participating in a yet another grant panel that I attended for the National Research Council of Canada in Ottawa for the last 2 days. Over the last 25 years that I have reviewed grant proposals for more than two dozen different agencies, I have observed a steady improvement in the quality of submitted grant applications. The amount of peripheral information that is required for these grant applications also markedly escalated during this period. This has created an extra burden for both the applicants and reviewers alike. It has now reached a point where I would not be surprised if a quarter of a typical academic scientist's time is spent either writing or reviewing grant proposals. This is extremely counterproductive.
I believe that most people do actually get colleagues to critique their grants and in some faculties, for example in the Faculty of Medicine at the University of British Columbia, this is a requirement for junior faculty for university approval for submission. It is also clear from recent data released by the U.S. National Institutes of Health that new grant proposals from established investigators have about the same chance of getting funding as applications from brand new faculty. Apparently, experience does not seem to carry much weight in the decision of whether an investigator receives funding, nor supposedly does their ability to write a fundable grant. After all, an established investigator must have a previous track record of success, otherwise he/she would not have lasted very long.
Typically only about 1/7 of submitted grant applications receive funding in North America. Most proposals are rated in the very good to excellent category, so it is really a crap shoot as to which particular applications are funded. Over the years, I have noticed two particularly disturbing trends that have exacerbated the problem.
The first relates to a general decline in the quality of the grant reviews if my personal experience is indicative of the current state of affairs. It seems that with the many more grant applications that have to be reviewed, grant fatigue is now pretty rampant. I understand from my conversations with granting agencies that it has become extremely difficult to find external reviewers for most grant proposals. Some agencies don’t bother anymore. This probably also explains why I am now solicited regularly by agencies from foreign countries. The lack of external reviews means that the fate of a grant application is largely at the mercy of two or three internal reviewers on a grant panel who have actually read most or all the proposal. Generally the rest of the grant panel just doesn’t have the time to read all of the submitted applications. These assigned internal reviewers may well be relatively new and inexperienced investigators, people who are not necessary experts in the subject of a particular proposal, and most certainly individuals that have to plough through a dozen or more other applications in short order and write reports. In the general panel discussion of the merits of a grant application that survives the triage process, a great deal of time is spent going over the specifics of the proposed work. (The odds are that by the time an investigator actually gets funded and towards the end of the first year of funding, the research will probably have taken quite a different course.) By contrast, the previous track record of an applicant gets pretty short shrift in the deliberations that I have been privy to on grant panels. Shouldn’t we be rewarding past success more?
The second relevant trend relates to the funding of many more and larger project grants over the last decade. Genome Canada is one example of agency in my country that is government mandated to award only large scale program grants. The typical budgets for these proposals can exceed $10 million dollars, but the applications are only about 2- to 3-times larger in content than typical proposals requesting 10- to 20-fold less operating funds. While external reviews are usually solicited and received for such large program proposals, the size of internal review panels for program grants are typically only about 50% larger than a regular project grant panel. The net effect is actually much less scrutiny of the spending of significantly more research dollars.
While it is difficult to obtain hard numbers from granting agencies, based on my own reviews of scores of program grants and their renewals, it appears to me that the number of publications (also factoring in impact) per dollar is much poorer from program grants than individual project grants. I wish that a careful analysis of the net productivity and other impacts of program verses small project grants over the last decade was available so that research funding in the future could be spent in the most-effective manner. If such results were applied, I suspect that the success rates of individual grant applications would improve markedly as would the rate of scientific progress.
Link to the original blog post.