The emergence of new variants of the novel coronavirus SARS-CoV-2 with increased infectivity, superior virulence, high transmissibility, and unmatched immune escape has demonstrated the adaptability and evolutionary fitness of the virus. The subject of relative order of the binding affinity of SARS-CoV-2 variants with the human ACE2 (hACE2) receptor is hotly debated and its resolution has implications for drug design and development. In this work, we have investigated the energetics of the binding of receptor binding domain (RBD) of SARS-CoV-2 variants of concern (VOCs): Beta (B.1.351), Delta (B.1.617.2), Omicron (B.1.1.529), variant of interest (VOI): Kappa (B.1.617.1), and Delta Plus (B.1.617.2.1) variant with the human ACE2 receptor by using the umbrella sampling (US) method. Our work indicates that Delta and Delta Plus variants have greater values of the US binding free energy than Wild-type (WT), whereas Beta, Kappa, and Omicron variants have lower values. Further analysis of hydrogen bonding, salt bridges, non-bonded interaction energy, and contact surface area at the RBD-hACE2 interface establish Delta as the variant with the highest binding affinity among these variants.