Prostate-specific membrane antigen (PSMA) overexpressed on prostate cancer (PCa) cells is a satisfactory theranostic target in PCa. To seek novel non-glutamate-urea-based PSMA inhibitors by the strategy of bioisosterism, 10 ligands were designed, synthesized, and characterized. Among them, ligands 17, 18, and 21-24 bearing the squaramic acid moiety proved to be potent PSMA inhibitors, with Ki values ranging from 0.40 to 2.49 nM, which are comparable or higher in inhibitory potency compared to previously reported glutamate-urea-based inhibitors. Docking studies of 15, 17, and 19 were carried out to explore their binding mode in the active site of PSMA. Two near-infrared (NIR) probes, 23 (λEM = 650 nm) and 24 (λEM = 1088 nm), displayed favorable in vivo NIR imaging and successful NIR-II image-guided tumor resection surgery in PSMA-positive tumor-bearing mice, which demonstrated the effectiveness of these new squaramic acid-based inhibitors.