Zinc deficiency is linked to poor prognosis in COVID-19 patients while clinical trials with zinc demonstrate better clinical outcomes. The molecular targets and mechanistic details of the anti-coronaviral activity of zinc remain obscure. We show that zinc not only inhibits the SARS-CoV-2 main protease (Mpro) with nanomolar affinity, but also viral replication. We present the first crystal structure of the Mpro-Zn2+ complex at 1.9 Å and provide the structural basis of viral replication inhibition. We show that Zn2+ coordinates with the catalytic dyad at the enzyme active site along with two previously unknown water molecules in a tetrahedral geometry to form a stable inhibited Mpro-Zn2+ complex. Further, the natural ionophore quercetin increases the anti-viral potency of Zn2+. As the catalytic dyad is highly conserved across SARS-CoV, MERS-CoV and all variants of SARS-CoV-2, Zn2+ mediated inhibition of Mpro may have wider implications.