Network architecture can shape in-context learning

Language models trained on biological sequences can be used to estimate how natural a given sequence looks. The degree to which a sequence seems natural to the model, as measured by its likelihood score, often correlates with experimentally measured fitness. We demonstrate that in-context learning can distort this relationship between fitness and likelihood scores, particularly in transformer-based architectures. This phenomenon manifests as anomalously high likelihood scores for sequences that contain repeated motifs. We use protein language models trained on the masked language modeling objective to show that this behavior is mediated by a look-up operation: the model seeks the identity of a masked position by using the other copy of the repeated motif as a reference. We systematically investigate how perturbations in a model's architecture influence its capability to learn in context, as measured by the length of repeats that it can successfully copy in a given prompt. Repeats can serve as an excellent probe to study the internal mechanics of large language model behavior.