Evaluating scientific impact: A comparison study using the NSF CAREER Award

Measuring scientific impact is at the heart of the question, 'Do people who receive grant funding have more scientific impact than their equal-potential counterparts who do not get funded?' In partnership with the Gordon and Betty Moore Foundation Experimental Physics Investigator (EPI) Initiative, we present a method and analysis for comparing the scientific impact of grant awardees and their unfunded counterparts. The method draws on the network normalized citation index, Ĉ, (Ke, Gates, Barabasi, 2023). This citation index addresses concerns common to other indices, particularly that different fields have different publication practices and that citation counts increase over time. Ĉ compares papers to others by creating a citation network and then using the average yearly citations of nearest neighbors to normalize. Because Ĉ uses average yearly citations, it also accounts for the cumulative nature of citations. We present a comparison of the scientific impact of NSF CAREER awardees for Early Faculty Development in physics to that of a randomized sample of associate physics professors using this newly developed metric. This analysis is based on the set of papers from the 262 NSF CAREER awardees from 1995-2015 since their respective award date (N = 10346) and the publications from 250 associate professors (N = 9096). The results from this analysis will inform an upcoming study on the EPI investigators, which will apply the same techniques to a sample of awardees and a comparison group.