Crimbrary

Reviewers are Blinkered by Bibliometrics
"Science panels still rely on poor proxies to judge quality and impact. That results in risk-averse research, say Paula Stephan, Reinhilde Veugelers and Jian Wang.

There is a disconnect between the research that reviewers purport to admire and the research that they actually support. As participants on multiple review panels and scientific councils, we have heard many lament researchers' reluctance to take risks. Yet we've seen the same panels eschew risk and rely on bibliometric indicators for assessments, despite widespread agreement that they are imperfect measures.

Although journal impact factors (JIFs) were developed to assess journals and say little about any individual paper, reviewers routinely justify their evaluations on the basis of where candidates have published. Panel members judge applicants by Google Scholar results and use citation counts to score proposals for new research. This practice prevails even at agencies such as the European Research Council (ERC), which instructs reviewers not to look up bibliometric measures."