Transformer networks for continuous gravitational-wave searches

Continuous gravitational waves (CWs) are emitted by rapidly spinning neutron stars with a non-axisymmetric deformation and lie in the sensitive band of the LIGO detectors. Wide parameter-space searches for these waves using the semi-coherent matched-filter methods require enormous computing power, which limits their achievable sensitivity. We explore an alternative search method based on training neural networks as classifiers on detector strain data with minimal pre-processing. Contrary to our previous studies using convolutional neural networks (CNNs), we investigate the suitability of the transformer architecture, specifically the Vision Transformer (ViT). We train ViTs to perform different types of CW benchmark searches: targeted search using 10 days of data, directed and all-sky searches using 1 day of data, and compare their sensitivity with the coherent matched filter method. The trained ViTs achieve essentially matched-filter sensitivity on the targeted benchmarks, and close to matched filter sensitivity on the directed and all-sky benchmarks. We find that unlike the CNNs in our previous studies, which required extensive manual design and hyperparameter tuning, the ViT achieves better performance with a standard architecture and minimal tuning.