Theory of overparametrization in quantum neural networks

Martín Larocca; Nathan Ju; Diego García-Martín; Patrick J Coles; Marco Cerezo

doi:10.1038/s43588-023-00467-6

Theory of overparametrization in quantum neural networks

Nat Comput Sci. 2023 Jun;3(6):542-551. doi: 10.1038/s43588-023-00467-6. Epub 2023 Jun 26.

Authors

Martín Larocca^#^{1

2}, Nathan Ju^#³, Diego García-Martín^{3

4

5}, Patrick J Coles³, Marco Cerezo^{3

6

7}

Affiliations

¹ Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM, USA. larocca@lanl.gov.
² Departamento de Física 'J. J. Giambiagi' and IFIBA, FCEyN, Universidad de Buenos Aires, Buenos Aires, Argentina. larocca@lanl.gov.
³ Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM, USA.
⁴ Barcelona Supercomputing Center, Barcelona, Spain.
⁵ Instituto de Física Teórica, UAM-CSIC, Madrid, Spain.
⁶ Information Sciences, Los Alamos National Laboratory, Los Alamos, NM, USA.
⁷ Center for Nonlinear Studies, Los Alamos National Laboratory, Los Alamos, NM, USA.

^# Contributed equally.

PMID: 38177434
DOI: 10.1038/s43588-023-00467-6

Abstract

The prospect of achieving quantum advantage with quantum neural networks (QNNs) is exciting. Understanding how QNN properties (for example, the number of parameters M) affect the loss landscape is crucial to designing scalable QNN architectures. Here we rigorously analyze the overparametrization phenomenon in QNNs, defining overparametrization as the regime where the QNN has more than a critical number of parameters M_c allowing it to explore all relevant directions in state space. Our main results show that the dimension of the Lie algebra obtained from the generators of the QNN is an upper bound for M_c, and for the maximal rank that the quantum Fisher information and Hessian matrices can reach. Underparametrized QNNs have spurious local minima in the loss landscape that start disappearing when M ≥ M_c. Thus, the overparametrization onset corresponds to a computational phase transition where the QNN trainability is greatly improved. We then connect the notion of overparametrization to the QNN capacity, so that when a QNN is overparametrized, its capacity achieves its maximum possible value.