The scientific community considers peer review as the gold standard, employing this method to determine which research projects receive funding which serves a gatekeeping function, shaping the direction of future science. However, low reliability and the potential for bias are sometimes associated with peer review; yet this process remains largely understudied. In recent years, low funding success rates for research proposals may exacerbate biases, as reviewers are more often tasked to discriminate between great and excellent applications. Recent studies have identified empirical examples of reviewer biases against novelty, as well as gender and racial biases. In addition, reviewer characteristics have been explored as sources of bias. While most review systems have procedures to mitigate the risk of explicit bias, few have effective systems to prevent implicit bias. While reviewer training is sometimes employed, and even some behavioral strategies are recommended (e.g. stereotype replacement), little research has shown the effectiveness of these interventions. More exploration in this area is needed, as well as in the early detection of biases. Moreover, a better understanding of the mechanisms of decision-making in peer review can likely be achieved by applying existing decision-making models from other research fields to peer review data sets. Finally, better definitions of scientific success are needed, as the use of bibliometric measurements in evaluation can easily lead to bias. Science policy should be scientific; therefore, it is incumbent on the scientific community to better understand peer review to ensure research is funded in the most equitable and impactful way.
- Stephen Gallo, Ph.D., Chief Scientist, American Institute of Biological Sciences