Benchmarking in the early fault-tolerant era

Historically, single-number benchmarks like randomized benchmarking (RB) and quantum volume (QV) have played a special role in the quantum computing community. These benchmarks, which are designed to summarize a device’s entire performance, allow the community to compare different platforms and track the field’s progress. To track progress, a benchmark must meaningfully measure the performance of state-of-the-art quantum computers. While RB initially served as the de facto standard for measuring the performance of one- and two-qubit devices, it was ultimately supplanted by QV as the standard summary benchmark for noisy intermediate-scale quantum computers (NISQ) because QV better captured a NISQ device’s overall computational power. In this talk, we will argue that new single-metric summary benchmarks --- based on measuring a quantum computer’s capability --- are needed for the early fault-tolerant era. We will then introduce a new capability-based benchmark that meaningfully tracks progress toward achieving quantum utility.