NXRefine: An automated workflow for analyzing single crystal diffuse x-ray scattering

Recent advances in synchrotron instrumentation have enabled the rapid acquisition of x-ray diffraction data from single crystals, allowing large contiguous volumes of scattering in reciprocal space to be collected in a matter of minutes. This allows the rapid collection of many data sets as a function of a parametric variable such as temperature, amounting to several TB a day. With such large data volumes, it is imperative to automate data reduction so that the results can be assessed in real time during an experiment. The Python package, NXRefine, implements a complete data reduction workflow, from ingesting the data, orienting the single crystals, transforming the data into reciprocal space coordinates, and generating 3D-ΔPDF maps, i.e., maps of real space interatomic vector probabilities using the punch-and-fill method [1]. The data are stored in NeXus files for immediate visualization using the NeXpy package [2].  It has been used to process several hundred terabytes of data at both the APS and CHESS. I will discuss recent developments, such as optimizing the workflow for GPU-intensive supercomputing facilities and integrating unsupervised machine learning to identify significant features in the data as they are collected.

[1] https://nexpy.github.io/nxrefine/

[2] https://nexpy.github.io/nexpy/