Emergent topological complexity and dimensional reduction

Emergent topological complexity has been observed in many different complex systems, from the weight matrix in machine learning to physical neural networks in human and mouse brain organoids. Are such emergent features a function of dimensional reduction and sparse sampling, or are they inherent features even in the original high dimensional embedding space? To address this question, we study topological complexity in small language models (SLMs) under dimensional reduction, in particular Betti number in the weight matrix taken as a network.