Critical level of epistasis separates trajectories toward evolution of mutational or drift robustness in small populations

High mutation rates select for the evolution of mutational robustness, where populations inhabit flat fitness peaks with little epistasis [1]. Recent evidence shows that a different effect shields small populations from fitness declines. In drift robustness [2], populations occupy peaks with steep “flanks”, and positive epistasis. But what happens when mutation rates are high and population sizes are small at the same time? Using a fitness model with both variable epistasis and mutational effect size, we show that the equilibrium fitness has a minimum as a function of the parameter that tunes epistasis, implying that this critical point is an unstable fixed point for evolutionary trajectories. In agent-based simulations of evolution at finite mutation rate, we demonstrate that when mutations can change epistasis, trajectories with a subcritical value of epistasis evolve to decrease epistasis, while those with supercritical initial points evolve towards higher epistasis. These two fixed points can be identified with mutational and drift robustness, respectively.

[1] C.O Wilke, J.L Wang, C. Ofria, R.E. Lenski, C. Adami. Nature 412 (6844), 331-333
[2] T. LaBar & C. Adami. Nature Comm. 8 (2017) 1012