Alzheimer's disease (AD) is an incurable, progressive neurodegenerative disease whose pathogenesis cannot be defined by one single element but consists of various factors; thus, there is a call for alternative approaches to tackle the multifaceted aspects of AD. Among the potential alternative targets, we aim to focus on glutaminyl cyclase (QC), which reduces the toxic pyroform of β-amyloid in the brains of AD patients. On the basis of a putative active conformation of the prototype inhibitor 1, a series of N-substituted thiourea, urea, and α-substituted amide derivatives were developed. The structure-activity relationship analyses indicated that conformationally restrained inhibitors demonstrated much improved QC inhibition in vitro compared to nonrestricted analogues, and several selected compounds demonstrated desirable therapeutic activity in an AD mouse model. The conformational analysis of a representative inhibitor indicated that the inhibitor appeared to maintain the Z-E conformation at the active site, as it is critical for its potent activity.