Two researchers in lab coats look through separate lenses on the same microscope.

Former NIH staffer reveals grant review process

A stylized representation of two people, one of whom has a speech bubble with the word YES! and the other of whom has a speech bubble that says NO?

 

“No amount of grantsmanship will turn a bad idea into a fundable one . . . but there are many outstanding ideas that are camouflaged by poor grantsmanship.”

-William Raub, Past Deputy Director of the NIH

The mission of the National Institutes of Health (NIH) is to seek fundamental knowledge that will enhance health, lengthen lifespan, and reduce illness and disability. Yearly, Congress provides funding so that NIH can help meet this mission and its goals of supporting creative discoveries and innovative research. While innovation and significance are core to the NIH review process, researchers must instill a high degree of confidence that they have the training, experience, methodology, and supportive institutional environment to be successful in their research endeavors.

On Thursday, November 3, Dr. Norman Braveman, Miami alumnus, former senior member of the NIH extramural program, and current current President of Braveman BioMed Consultants, shared with faculty in Miami University’s Department of Psychology the importance of understanding the NIH review process for successful grantsmanship. Not only should a grant proposer understand the review process, they should also understand the mission of the NIH and most importantly of the institute or center (IC) to which they are applying. Braveman suggested going to the IC website to get a snapshot of the IC strategic plan prior to formulating project objectives so as to focus your research on areas of current importance to the mission of the IC.

Braveman began by providing a 50,000-foot overview of the NIH grant review process. Most applications are assigned to the Center for Scientific Review (CSR), where they undergo initial scientific merit review. Some applications may undergo initial peer review in a specific IC, as specified in the funding opportunity announcement (FOA). Understanding where your proposal will be reviewed (CSR or IC) and by whom (e.g., scientific review group) can help you target your application accordingly.

A flow chart describing NIH review process. The chart starts with Researcher idea and Institution. An solid arrow then leads to Grant Application (R01, R03, R21, K01, K08, etc.). A solid line leads to NIH/CSR Referral and Review. From there, there are two solid arrows. The first leads to IC/Program, which is the end of that chain. The second arrow leads to Initial Peer Review CSR or IC. A solid arrow leads from Initial Peer Review CSR or IC to Review Summary Statement. There are two solid arrows leading from Review Summary Statement. The first leads to PO/Applicant, and there is a dotted line that leads from there to Secondary Review National Advisory Council. The second solid arrow leads from Review Summary Statement to IC Decision Process. From IC Decision Process, there are two options: 1 - funded (represented by a handful of cash) and 2 - unfunded (represented by a thumbs-down). From unfunded, a solid arrow labeled revision leads back to the start - Researcher idea and institution.
This flow chart from Dr. Braveman’s presentation shows how the NIH review process works.

While NIH review criteria include significance, investigator(s), innovation, approach, and environment, Braveman suggested that the un-scored “Specific Aims” section is the most important part of any NIH grant. It is in this section that researchers tell the reviewers about the purpose and importance of their project. The specific aims may determine whether a reviewer wants to read more or set the proposal aside — which equates to getting triaged or not discussed and/or scored.

If the specific aims are clear, concise, and attention-grabbing, then the proposal may be discussed by the study section. It is important to note that any proposal that has been triaged can be pulled into the discussion by any of the reviewers serving on the study section. Proposals that are discussed receive a score from 1- 9, with 1 being “exceptional” and 9 being “poor.” Average scores are multiplied by 10, then scores from all proposals from that review round are percentile-ranked.

Table describing the NIH review scoring system. The first column is Score, the second Descriptor and the third Additional Guidance on Strengths/Weaknesses. Data in the rows are as follows: 1/Exceptional/Exceptionally strong with essentially no weaknesses. 2/Outstanding/Extremely strong with negligible weaknesses. 3/Excellent/Very strong with only some minor weaknesses. 4/Very Good/Strong but with numerous minor weaknesses. 5/Good/Strong but with at least one moderate weakness. 6/Satisfactory/Some strengths but also some moderate weaknesses. 7/Fair/Some strengths but with at least one major weakness. 8/Marginal/A few strengths and numerous major weaknesses. 9/Poor/Very few strengths and numerous major weaknesses.
This table from Dr. Braveman’s presentation describes the NIH review scoring system.

Once proposals are scored and ranked, they go to the National Advisory Council for second-level review. This council reviews the nature of the proposals submitted to see how they fit into the objectives of the NIH and into the objectives of the current IC. Considerations such as how project ideas may work synergistically to answer larger questions are taken into account in making funding decisions. Therefore, it may be the case that one proposal that doesn’t score as well as another may get funded because it addresses a question that no other proposal addresses and is currently important to the mission of the IC.

Another important component in the NIH decision making process is the program officer (PO). The PO interacts with the National Advisory Council and should be able to speak on behalf of any given proposal. It is important to contact the PO early in the proposal development phase to ensure:

  • Your proposal objectives match the current program or IC objectives
  • The PO understands your objectives in case he or she needs to advocate to the National Advisory Council

In addition to establishing a working relationship with the PO, other things to keep in mind when writing the NIH proposal include:

  • NIH and IC objectives
  • Reviewers read 20-25 applications three times a year, so yours needs to be meaningful and well-written
  • You should write to the NIH review criteria: significance, investigator(s), innovation, approach, and environment
  • Use the biosketch to help establish capabilities of the investigator(s)
  • Use the “Resources” section to elaborate on the appropriateness of the environment

Remember that when writing an NIH — or any — grant proposal, as Francis Bacon said, “knowledge is power.” Be certain to do your homework and understand the funding agency, as well as the review criteria and review process to increase your chances of funding success.


Written by Tricia Callahan, Director, Office for the Advancement of Research and Scholarship, Miami University.

Microscope image by the National Cancer Institute, via Wikimedia Commons, used under public domain. YES!/NO? illustration by I and Materia via Wikimedia Commons, used under Creative Commons license. Flow chart and table illustrations by Dr. Norm Braveman, used with permission.

Former NSF program officer demystifies review and resubmission processes

An illustrated figure of a featureless person holds a giant red pencil, checking off boxes on the piece of paper on which he stands.

In the spring of 2010, the National Science Foundation (NSF) created a task force on the merit review process in order to align the review criteria with NSF’s new strategic plan. The final recommendations from that task force included two review criteria and five review elements, all of which were outlined during a presentation by Miami University professor of biology and former NSF program officer Joyce Fernandes on November 11. Fernandes’ talk,, “NSF resubmission: How to decipher the panel summary,” was part of OARS’ fall NSF workshop series.

Fernandes says that when evaluating proposals, all reviewers are asked to evaluate against two criteria:

  • Intellectual merit (potential to advance knowledge)
  • Broader impacts (potential to benefit society and contribute to the achievement of specific, desired societal outcomes)

Five key elements are considered when evaluating both of the review criteria. These elements include:

  • The potential for the proposed activity to advance knowledge and understanding within the field (or across different fields) and benefit to society.
  • The extent that proposed activities explores creative, original, or potentially transformative concepts.
  • A well-reasoned, well-organized plan for carrying out proposed activities.
  • Qualifications of the individual, team or institution to conduct proposed activities.
  • Adequate resources to carry out proposed activities.

“During a typical NSF review process,” Fernandes explains, “reviewers are selected based on their knowledge of the specific content area. Each proposal is typically assigned to three reviewers, with one acting as the primary reviewer.”

Fernandes goes on to explain that after discussing strengths and weaknesses of the proposal in regard to the Intellectual merit and broader Impacts, the panel makes a recommendation to place the proposal in one of 3-4 categories that indicate priority for funding (high, medium, low), or how competitive the proposal was (outstanding, highly meritorious, meritorious, non-competitive) . The panel summary, which is provided to the Principal Investigator (PI) along with the reviews, is a record of the discussion of a proposal and is intended to provide a rationale for the category in which the proposal was placed. PIs usually receive a context statement that tells how many proposals were submitted and reviewed, along with the ranking system (categories), and percentage of proposals that were placed into the categories.

“Bottom line: funding of proposals is based on dollars available,” Fernandes says. While program officers try to balance their portfolios based on things like emerging areas, broadening representation/participation, unique approaches to research questions, and transformational advances in the field, the number of proposals that are recommended for funding is based on the availability of program funds.

“At times, a lower ranked proposal might get funded over a more competitive proposal,” Fernandes says, because of an unusual merit of the proposal. She says it’s important for PIs to remember this as they review their summary statements.

“Almost all proposals submitted are meritorious, so it doesn’t take much to get your proposal kicked out of competition,” Fernandes says.

It is important that investigators who are responding to panel feedback take it seriously. Most investigators (after they have had time to simmer) find the panel feedback to be valuable as they rework their next submission.

Suggestions Fernandes offers for responding to the panel summary include reading the summary, taking time away before responding, re-reading and digesting the summary, discussing next-steps with the program officer, and if appropriate, resubmitting the proposal. (It should be noted that technically there is no category for “resubmissions.” All proposals to the NSF are considered new submissions).

Fernandes advises PIs who are still unclear about the feedback after reading and re-reading the panel summary to contact their program officer for clarification.

Finding the right program and understanding and responding to reviewer feedback are essential for increasing your chances for a successful submission. However, it is important to note that panel compositions change every year, and a new group of proposals is submitted for each deadline; therefore PIs should not expect that simply responding to reviewer feedback will move their proposal into a higher category.

Finally, understanding the NSF review process is one key to writing a successful proposal. You can learn more about the NSF proposal process here or by volunteering to serve on a NSF review panel.

Written by Tricia Callahan, Director of Proposal Development, Office for the Advancement of Research & Scholarship, Miami University.

Illustrations by AJ Cann via Flickr, used under Creative Commons license.