Chromosomal fragile sites are genomic loci sensitive to replication stress which accumulate high levels of DNA damage, and are frequently mutated in cancers. Fragile site damage is thought to arise from the aberrant repair of spontaneous replication stress, however successful fragile site repair cannot be calculated using existing techniques. Here, we report a new assay measuring recombination-mediated repair at endogenous genomic loci by combining a sister chromatid exchange (SCE) assay with fluorescent in situ hybridization (SCE-FISH). Using SCE-FISH, we find that endogenous and exogenous replication stress generated unrepaired breaks and SCEs at fragile sites. We also find that distinct sources of replication stress induce distinct patterns of breakage: ATR inhibition induces more breaks at early replicating fragile sites (ERFS), while ERFS and late-replicating common fragile sites (CFS) are equally fragile in response to aphidicolin. Furthermore, SCEs were suppressed at fragile sites near centromeres in response to replication stress, suggesting that genomic location influences DNA repair pathway choice. SCE-FISH also measured successful recombination in human primary lymphocytes, and identificed the proto-oncogene BCL2 as a replication stress-induced fragile site. These findings demonstrate that SCE-FISH frequency at fragile sites is a sensitive indicator of replication stress, and that large-scale genome organization influences DNA repair pathway choice.