Towards foundation models for large-scale scientific facilities: Characterizing utility, scaling and transfer learning

Large-scale scientific user facilities such as particle accelerators, tokamaks, and observatories are complex cyber-physical systems composed of thousands of heterogeneous instruments that must operate in coherence for diagnostics, control, and safe, reliable operations. These instruments generate massive multimodal data streams that are often noisy, incomplete, and distributed across subsystems. Operational tasks such as anomaly detection, tuning, and control decisions depend critically on these measurements. Large particle accelerators are user facilities which supports multiple scientific applications including fundamental research, engineering, and critical applications such as medical isotope production. The LANSCE particle accelerator is instrumented with a large number of sensors for diagnostics and controls. We introduce foundation models designed to learn generalizable representations from historical measurements, control signals, logs, and beam diagnostics. We evaluate its utility across key tasks including missing-data imputation, anomaly detection and accelerator tuning assistance. We characterize scaling with model capacity and pretraining data volume, quantifying capability gains as scale increases. Finally, we study transfer learning for zero-shot performance on unseen data and efficient fine-tuning with limited out-of-distribution data. Together, the findings indicate that a single self-supervised model unifies disparate accelerator tasks and provides rich, generalizable representations for building task-specific models. This work charts a path toward robust and efficient data-driven operations across large-scale scientific facilities.