As of the writing of this paper, lower bounds are not a staple of quantum chemistry computations and for good reason. All previous attempts at applying lower bound theory to Coulombic systems led to lower bounds whose quality was inferior to the Ritz upper bounds so that their added value was minimal. Even our recent improvements upon Temple's lower bound theory were limited to Lanczos basis sets and these are not available to atoms and molecules due to the Coulomb singularity. In the present paper, we overcome these problems by deriving a rather simple eigenvalue equation whose roots, under appropriate conditions, give lower bounds which are competitive with the Ritz upper bounds. The input for the theory is the Ritz eigenvalues and their variances; there is no need to compute the full matrix of the squared Hamiltonian. Along the way, we present a Cauchy-Schwartz inequality which underlies many aspects of lower bound theory. We also show that within the matrix Hamiltonian theory used here, the methods of Lehmann and our recent self-consistent lower bound theory (J. Chem. Phys. 2020, 115, 244110) are identical. Examples include implementation to the hydrogen and helium atoms.