Information and Optimization in Markov Decision Process Models of Drug Protocols

A major medical concern is the evolution of drug resistance. Yet mutated resistance to one drug can be accompanied by particular susceptibility to a second drug - a phenomenon known as "collateral sensitivity". This suggests that cycling between different drugs in a specific sequence can be more effective than a single drug.

Markov Decision Processes (MDPs) describe controlled random systems, providing a natural framework for modeling drug protocols: the genetic makeup of the disease evolves stochastically in the presence of drugs chosen at each time step to steer the evolution. Given perfect knowledge of the genetic state of the disease, an optimal protocol can be found to maximize treatment efficacy (minimizing disease fitness).

But perfect knowledge is impossible - biopsies and drug updates occur at finite intervals; we can quantify the incomplete knowledge with information theory. Specifically, we formulate an MDP model of drug protocols controlling evolving disease with constrained mutual information between the genetic state of the disease and the drug administered for that genetic state, inducing a tradeoff between information and optimality. Analytical results can be obtained in simple cases.

In physics terms, this is feedback control, leading to a non-equilibrium steady-state driven by a kind of Maxwell Demon. Medically, this kind of model could potentially inform real clinical decisions regarding frequency of biopsying a disease, by quantifying the benefit of that increased information.