Factor XIa (FXIa) in the intrinsic pathway of the coagulation process has been proven to be an effective and safe target for anticoagulant discovery with limited or no bleeding. Numerous small-molecule FXIa inhibitors (SMFIs) with various scaffolds have been identified in the early stages of drug discovery. They have served as the foundation for the recent discovery of additional promising SMFIs with improved potency, selectivity, and pharmacokinetic profiles, some of which have entered clinical trials for the treatment of thrombosis. After reviewing the coagulation process and structure of FXIa, this perspective discusses the rational or structure-based design, discovery, structure-activity relationships, and development of SMFIs disclosed in recent years. Strategies for identifying more selective and druggable SMFIs are provided, paving the way for the design and discovery of more useful SMFIs for anticoagulation therapy.