Reward-Driven Optimization Framework for Physics-Informed and Explainable Image Analysis
ORAL
Abstract
Automated scientific imaging increasingly requires robust and explainable workflows capable of adapting to dynamic experimental conditions. We introduce a reward-driven optimization framework that formulates image analysis as a decision-making process guided by physics-informed reward functions. These rewards quantify alignment between analysis outcomes and domain objectives such as structural continuity, symmetry, or physical plausibility, enabling unsupervised, interpretable optimization of complex workflows. The framework couples classical operations with machine-learning models through Bayesian optimization to tune hyperparameters and descriptors based on reward feedback, ensuring that extracted features represent genuine physical phenomena rather than artifacts. Applied to scanning transmission electron microscopy (STEM), the approach achieves real-time segmentation and feature extraction in ion-irradiated and thin films and enhances foundational models such as SAM for streaming analysis. Beyond microscopy, this methodology provides a general strategy for autonomous, physics-guided analysis across imaging and hyperspectral data, bridging explainable AI with experimental decision-making.
*This work was supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences as part of the Energy Frontier Research Centers program: CSSAS-The Center for the Science of Synthesis Across Scales under award number DE-SC0019288.
–
Publication:1. K. Barakati, H. Yuan, A. Goyal and S. V. Kalinin, Physics-based reward driven image analysis in microscopy, Digital Discovery 3 (10), 2061-2069 (2024). 2. K. Barakati, Y. Liu, C. Nelson, M. Ziatdinov, X. Zhang, I. Takeuchi and S. V. Kalinin, Reward driven workflows for unsupervised explainable analysis of phases and ferroic variants from atomically resolved imaging data, Advanced Materials 37 (35), 2418927 (2025). 3. K. Barakati, U. Pratiush, A. C. Houston, G. Duscher and S. V. Kalinin, Unsupervised Reward-Driven Image Segmentation in Automated Scanning Transmission Electron Microscopy Experiments, arXiv preprint arXiv:2409.12462 (2024). 4. K. Barakati, Y. Liu, U. Pratiush, B. N. Slautin and S. V. Kalinin, Rewards-based image analysis in microscopy, arXiv preprint arXiv:2502.18522 (2025). 5. K. Barakati, U. Pratiush, S. L. Sanchez, A. Raghavan, D. J. Milliron, M. Ahmadi, P. D. Rack and S. V. Kalinin, SAM $^{*} $: Task-Adaptive SAM with Physics-Guided Rewards, arXiv preprint arXiv:2509.07047 (2025).
Presenters
Sergei V Kalinin
University of Tennessee
Authors
Kamyar Barakati
University Tennessee-Knoxville
Sergei V Kalinin
University of Tennessee
Utkarsh Pratiush
University of Tennessee
Department of Materials Science and Engineering, University of Tennessee, Knoxville