Scores and Output
06/06/11 14:08 Filed in: GenomeWeb Daily Scan
Submitted by S. Pelech - Kinexus on Mon, 06/06/2011 - 14:08.
This NIGMS peer-review study actually demonstrated a relatively poor correlation between peer review scores and various measures of scientific output, especially within the top 20 percentile of peer review scores. Presently, NIH grant applications with scores above the top 12-14 percentile are missing the cut off for funding. As acknowledged by Dr. Berg in his post, the findings are based entirely on funded NIH grants and there is no comparison with data for grants that were not deemed to have sufficiently high scores for NIH funding. Of course, with no funding, it is a lot harder to conducted the proposed research and ultimately publish the findings.
Peer review committees commonly look for strong preliminary data in support of the hypotheses being advanced in grant applications, so it is not surprising even if a higher rate of publication was observed for funded projects. Much of the proposed work may have already been completed by the time the applicants actually start to receive the award funding.
For established investigators that have had sufficient time to build their research programs, their funding should really be dependent on their demonstrated productivity over the previous 5 year period unless there is extenuating circumstances such as maternity leave or illness. The amount of funding provided in a single grant award should be scaled back and more individual grants awarded.
Amongst the many problems associated with the present peer-review process for grant funding, I am concerned about the time constraints and burden placed on the reviewers in grant panels. On the one hand, they often don't have the requisite knowledge and their dedication to perform a good job can negatively impact the time that they have for their own research. On the other hand, for those with the appropriate background and experience, serving on grant review committees provides very nice insights into what other competing laboratories are doing that would be very difficult to ignore in their own research programs. In any event, most researchers are increasingly spending more of their time looking for funding than actually performing innovative and ground breaking research.
Link to the original blog post.
This NIGMS peer-review study actually demonstrated a relatively poor correlation between peer review scores and various measures of scientific output, especially within the top 20 percentile of peer review scores. Presently, NIH grant applications with scores above the top 12-14 percentile are missing the cut off for funding. As acknowledged by Dr. Berg in his post, the findings are based entirely on funded NIH grants and there is no comparison with data for grants that were not deemed to have sufficiently high scores for NIH funding. Of course, with no funding, it is a lot harder to conducted the proposed research and ultimately publish the findings.
Peer review committees commonly look for strong preliminary data in support of the hypotheses being advanced in grant applications, so it is not surprising even if a higher rate of publication was observed for funded projects. Much of the proposed work may have already been completed by the time the applicants actually start to receive the award funding.
For established investigators that have had sufficient time to build their research programs, their funding should really be dependent on their demonstrated productivity over the previous 5 year period unless there is extenuating circumstances such as maternity leave or illness. The amount of funding provided in a single grant award should be scaled back and more individual grants awarded.
Amongst the many problems associated with the present peer-review process for grant funding, I am concerned about the time constraints and burden placed on the reviewers in grant panels. On the one hand, they often don't have the requisite knowledge and their dedication to perform a good job can negatively impact the time that they have for their own research. On the other hand, for those with the appropriate background and experience, serving on grant review committees provides very nice insights into what other competing laboratories are doing that would be very difficult to ignore in their own research programs. In any event, most researchers are increasingly spending more of their time looking for funding than actually performing innovative and ground breaking research.
Link to the original blog post.