Why Impact Factor Rankings Favor Small Journals

Can a comparison of two population averages be misleading? We show that, because of the natural differences within a population, an unweighted average metric tends to favor smaller entities over larger ones. When the degree of inequality within a population is small, this scale-dependence effect is of little practical importance. But for citations of scientific papers in journals, which can span up to four orders of magnitude, the effect is strong enough to interfere with impact factor rankings. By analyzing 166,498 journals in the 1997–2016 Journal Citation Reports of Clarivate Analytics (formerly Thomson Reuters), we have identified a boundary curve for the Impact Factor as a function of journal size. We confirm the functional form of this boundary curve by analyzing the top-cited portion of 2,700,000 papers published in 2015, and the citation distributions of several journals. We propose a scale-independent average citation index, as a potential remedy against the problem.