Sex chromosomes are a pair of specialized chromosomes containing a sex determination region that is suppressed for recombination. Without recombination, Y chromosomes are thought to accumulate repetitive DNA sequences which contribute to their degeneration. A pair of primitive sex chromosomes controls sex type in papaya with male and hermaphrodite determined by the slightly different male-specific region of the Y (MSY) and hermaphrodite-specific region of Y(h) (HSY) chromosomes, respectively. Here, we show that the papaya HSY and MSY in the absence of recombination have accumulated nearly 12 times the amount of chloroplast-derived DNA than the corresponding region of the X chromosome and 4 times the papaya genome-wide average. Furthermore, a chloroplast genome fragment containing the rsp15 gene has been amplified 23 times in the HSY, evidence of retrotransposon-mediated duplication. Surprisingly, mitochondria-derived sequences are less abundant in the X and HSY compared to the whole genome. Shared organelle integrations are sparse between X and HSY, with only 11 % of chloroplast and 12 % of mitochondria fragments conserved, respectively, suggesting that the accelerated accumulation of organelle DNA occurred after the HSY was suppressed for recombination. Most of the organelle-derived sequences have divergence times of <7 MYA, reinforcing this notion. The accumulated chloroplast DNA is evidence of the slow degeneration of the HSY.