B cell response plays a critical role against SARS-CoV-2 infection. However, little is known about the diversity and frequency of the paired SARS-CoV-2 antigen-specific BCR repertoire after SARS-CoV-2 infection. Here, we performed single-cell RNA sequencing and VDJ sequencing using the memory and plasma B cells isolated from five convalescent COVID-19 patients, and analyzed the spectrum and transcriptional heterogeneity of antibody immune responses. Via linking BCR to antigen specificity through sequencing (LIBRA-seq), we identified a distinct activated memory B cell subgroup (CD11chigh CD95high) had a higher proportion of SARS-CoV-2 antigen-labeled cells compared with memory B cells. Our results revealed the diversity of paired BCR repertoire and the non-stochastic pairing of SARS-CoV-2 antigen-specific immunoglobulin heavy and light chains after SARS-CoV-2 infection. The public antibody clonotypes were shared by distinct convalescent individuals. Moreover, several antibodies isolated by LIBRA-seq showed high binding affinity against SARS-CoV-2 receptor-binding domain (RBD) or nucleoprotein (NP) via ELISA assay. Two RBD-reactive antibodies C14646P3S and C2767P3S isolated by LIBRA-seq exhibited high neutralizing activities against both pseudotyped and authentic SARS-CoV-2 viruses in vitro. Our study provides fundamental insights into B cell response following SARS-CoV-2 infection at the single-cell level.