Simmulated Annealing for Global Search

Global search over black box functions describes much of computational physics, typically for determining critical points on energy surfaces. As surrogate (machine learned) models have come of age and taken center stage, global search of optimal parameters (or "hyper parameters") for these models have become concomitantly more important. For most systems (including neural networks) finding optimal hyper parameters is far from easy. Most implementations focus on using first order search methods (e.g. batch stochastic gradient methods). We demonstrate the applicability of simulated annealing (via a new, high performance anneal python package) to the problem of finding minima in both hyper parameter surfaces and standard energy systems. Our approach is unique in that it demonstrates the coupling (at a computational implementation level) of the related concept of Metropolis Hasting sampling and its connections to simulated annealing. We will introduce several standard paradigms and demonstrate how these can be "lifted" into a unified framework using object-oriented programming in Python. We demonstrate how clean, inter operable, reproducible programming libraries can be used to access and rapidly iterate on variants of Simulated Annealing in a manner which can be extended to serve as a best practices blueprint or design pattern for a data-driven optimization library.