Excellent "CHON" compatible ligands based on a heterocyclic skeleton for the separation of trivalent actinides [An(III)] from lanthanides [Ln(III)] have been widely explored, the aim being spent nuclear fuel reprocessing. The combination mode of a soft/hard (N/O) donor upon the coordination chemistry of An(III) and Ln(III) should play a vital role with respect to the performance of ligands. As such, in this work, two typical experimentally available phenanthroline-derived tetradentate ligands, CyMe4-BTPhen (L1) and Et-Tol-DAPhen (L4), and two theoretically designed asymmetric tetradentate heterocyclic ligands, L2 and L3, with various N/O donors were investigated using scalar relativistic density functional theory. We have evaluated the electronic structures of L1-L4 and their coordination modes, bonding properties, and extraction reactions with Am(III) and Eu(III). We found that the Am/Eu-N interactions play a more important role in the orbital interactions between the ligand and Am(III)/Eu(III) ions. Compared with those of L1, the coordinated O atoms of L2 and L4 weaken the metal-N bonds. The Am(III)/Eu(III) selectivity follows the order L1 > L2 > L4 based on the change in Gibbs free energy, reflecting the fact that the Am(III)/Eu(III) selectivity of the ligand is affected by the number of coordinated N atoms. In addition, L3 displays the strongest binding ability for Am(III)/Eu(III) ions and the smallest Am(III)/Eu(III) selectivity among the four ligands, due to its structural preorganization. This work clarifies the influence of the number of coordinated N and O atoms of ligands on Am(III)/Eu(III) selectivity, which provides valuable fundamental information for the design of efficient ligands with N and O donors for An(III)/Ln(III) separation.