Among numerous posttranslational regulation patterns, phosphorylation is reversibly controlled by the balance of kinases and phosphatases. The major form of cellular signaling involves the reversible phosphorylation of proteins on tyrosine, serine, or threonine residues. However, altered phosphorylation levels are found in diverse diseases, including cancer, making kinases and phosphatases ideal drug targets. In contrast to the success of prosperous kinase inhibitors, design of small molecules targeting phosphatase is struggling due to past bias and difficulty. This is especially true for serine/threonine phosphatases, one of the largest phosphatase families. From this perspective, we aim to provide insights into serine/threonine phosphatases and the small molecules targeting these proteins for drug development, especially in cancer. Through highlighting the modulation strategies, we aim to provide basic principles for the design of small molecules and future perspectives for the application of drugs targeting serine/threonine phosphatases.