Optimization of compound 11L led to the identification of novel HIV capsid modulators, quinazolin-4-one-bearing phenylalanine derivatives, displaying potent antiviral activities against both HIV-1 and HIV-2. Notably, derivatives 12a2 and 21a2 showed significant improvements, with 2.5-fold over 11L and 7.3-fold over PF74 for HIV-1, and approximately 40-fold over PF74 for HIV-2. The X-ray co-crystal structures confirmed the multiple pocket occupation of 12a2 and 21a2 in the binding site. Mechanistic studies revealed a dual-stage inhibition profile, where the compounds disrupted capsid-host factor interactions at the early stage and promoted capsid misassembly at the late stage. Remarkably, 12a2 and 21a2 significantly promoted capsid misassembly, outperforming 11L, PF74, and LEN. The substitution of easily metabolized amide bond with quinolin-4-one marginally enhanced the stability of 12a2 in human liver microsomes compared to controls. Overall, 12a2 and 21a2 highlight their potential as potent HIV capsid modulators, paving the way for future advancements in anti-HIV drug design.