Ligand-directed reactions allow chemical transformations at very low reactant concentrations and can thus provide access to efficient approaches for the post-translational modification of proteins. The development of these proximity-induced reactions is hampered by the number of appropriate ligands and the lack of design principles. Addressing these limitations, we report a proximity-induced labeling system which applies a moderate affinity peptide ligand. The design process was structure-guided and supported by molecular dynamics simulations. We show that selective protein labeling can be performed inside living cells enabling the subcellular translocation of a protein via ligand-directed chemistry for the first time.