Fractures are a critical process in how materials wear, weaken, and fail, whose unpredictable behavior can have dire consequences. While the behavior of smooth cracks in ideal materials is well understood, it is assumed that for real, heterogeneous systems, fracture propagation is complex, generating rough fracture surfaces that are highly sensitive to specific details of the medium. Here we show how fracture roughness and material heterogeneity are inextricably connected via a simple framework. Studying hydraulic fractures in brittle hydrogels that have been supplemented with microbeads or glycerol to create controlled material heterogeneity, we show that the morphology of the crack surface depends solely on one parameter: the probability to perturb the front above a critical size to produce a steplike instability. This probability scales linearly with the number density, and with heterogeneity size to the 5/2 power. The ensuing behavior is universal and is captured by the 1D ballistic propagation and annihilation of steps along the singular fracture front.