Collagen is the dominant protein of the extracellular matrix. Its distinguishing feature is a three-stranded helix of great tensile strength. (2 S,4 R)-4-Hydroxyproline residues are essential for the stability of this triple helix. These residues arise from the post-translational modification of (2 S)-proline residues by collagen prolyl 4-hydroxylases (CP4Hs), which are members of the Fe(II)- and α-ketoglutarate (AKG)-dependent dioxygenase family. Here, we provide a framework for the inhibition of CP4Hs as the basis for treating fibrotic diseases and cancer metastasis. We begin with a summary of the structure and enzymatic reaction mechanism of CP4Hs. Then, we review the metal ions, metal chelators, mimetics of AKG and collagen strands, and natural products that are known to inhibit CP4Hs. Our focus is on inhibitors with potential utility in the clinic. We conclude with a prospectus for more effective inhibitors.