Origins and mitigation of double descent in sparse sensing

Many modern high-dimensional datasets have latent low rank and thus can be represented as a linear combination of just a few modes, and reconstructed from just a few localized sensor measurements. However, when the number of sensors exactly matches the number of modes, the reconstruction process suddenly becomes unstable, echoing the double descent phenomenon recently investigated in a variety of linear and nonlinear supervised learning scenarios. In this work we locate the origin of the instability in the inevitable running out of orthogonal sensors. This orthogonality crisis catastrophically amplifies any remaining spectral content or measurement noise. We compute the error response curve without restrictive assumptions on the data matrix in order to explore different scenarios. We show that the reconstruction instability is best mitigated by an appropriate choice of state prior or regularization, but can also be avoided by strategic undersampling or intelligent sensor placement.