Optimal Control and Reinforcement Learning of Regulation and Enzyme Activities

Experimental measurement or computational inference/prediction of the enzyme regulation needed in a metabolic pathway is hard problem. Consequently, regulation is known only for well-studied reactions of central metabolism in various model organisms. In this study, we use statistical thermodynamics and metabolic control theory as a theoretical framework to calculate enzyme regulation policies for controlling metabolite concentrations to be consistent with experimental values. A reinforcement learning approach is utilized to learn optimal regulation policies that match physiological levels of metabolites while maximizing the entropy production rate and minimizing the heat loss. The learning takes a minimal amount of time, and efficient regulation schemes were learned that either agree with theoretical calculations or result in a higher cell fitness using heat loss as a metric. We demonstrate the process on four pathways in the central metabolism of Neurospora crassa (gluconeogenesis, glycolysis-TCA, Pentose Phosphate-TCA, and cell wall synthesis) that each require different regulation schemes.