Conditioning generative models without retraining: How to make a horse into a car

Generative models excel at creating new samples from a learned distribution. However, guiding the generation process toward a specific target typically requires conditional sampling, which involves either retraining the unconditional model or employing specialized architectures. Both approaches can be computationally expensive and difficult to apply to new datasets. In this work, we introduce a flexible method that leverages a pre-trained unconditional generative model and a separate classifier to perform conditional sampling without retraining. Our approach utilizes Bayes’ rule, where feedback from the classifier informs the conditional probability, p(class|sample), and the generative model provides a proxy for p(sample). We demonstrate our method using conditional image generation and de novo protein design.