Survival probabilities of mutants with antagonistic interactions

Strains within a population interact not only via competition due to varying growth rates, but also via secretion of toxins, predation, parasitism, and other adaptations which result in frequency-dependent selection. These dynamics often play out in a spatial context, such as at the frontier of a growing spherical cell mass, or on the surface of a Petri dish. We study how a mutant, interacting antagonistically with a wild-type strain (i.e., the mutant grows more slowly in the presence of the wild-type), survives in two-dimensional populations with flat and curved geometries, representing, for example, the Petri dish surface and cell mass frontier, respectively. The antagonistic interactions significantly diminish the survival probability of the mutant even when, in isolation, the mutant grows much faster than the wild-type strain. We show that the survival probability can be thought of as a kind of "nucleation rate" of the mutant strain. The predictions of classical nucleation theory agree with our simulation results and provide an important modification, due to the antagonistic interactions, of the classic Kimura formula for the survival probability. We comment on both the effects of small-number fluctuations and the curved population geometry.