The kinetochore is the central molecular machine that drives chromosome segregation in all eukaryotes. Genetic studies have suggested that protein sumoylation plays a role in regulating the inner kinetochore; however, the mechanism remains elusive. Here, we show that Saccharomyces cerevisiae Ulp2, an evolutionarily conserved SUMO specific protease, contains a previously uncharacterized kinetochore-targeting motif that recruits Ulp2 to the kinetochore via the Ctf3CENP-I-Mcm16CENP-H-Mcm22CENP-K complex (CMM). Once recruited, Ulp2 selectively targets multiple subunits of the kinetochore, specifically the Constitutive Centromere-Associated Network (CCAN), via its SUMO-interacting motif (SIM). Mutations that impair the kinetochore recruitment of Ulp2 or its binding to SUMO result in an elevated rate of chromosome loss, while mutations that affect both result in a synergistic increase of chromosome loss rate, hyper-sensitivity to DNA replication stress, along with a dramatic accumulation of hyper-sumoylated CCAN. Notably, sumoylation of CCAN occurs at the kinetochore and is perturbed by DNA replication stress. These results indicate that Ulp2 utilizes its dual substrate recognition to prevent hyper-sumoylation of CCAN, ensuring accurate chromosome segregation during cell division.