Many pathogens continuously change their protein structure in response to immune-driven selection, resulting in weakened protection even in previously exposed individuals. In addition, for some pathogens, such as dengue virus, poorly targeted immunity is associated with increased risk of severe disease through a mechanism known as antibody-dependent enhancement. However, it remains unclear whether the antigenic distances between an individual's first infection and subsequent exposures dictate disease risk, explaining the observed large-scale differences in dengue hospitalizations across years. Here, we develop a framework that combines detailed antigenic and genetic characterization of viruses with details on hospitalized cases from 21 years of dengue surveillance in Bangkok, Thailand, to identify the role of the antigenic profile of circulating viruses in determining disease risk. We found that the risk of hospitalization depended on both the specific order of infecting serotypes and the antigenic distance between an individual's primary and secondary infections, with risk maximized at intermediate antigenic distances. These findings suggest that immune imprinting helps determine dengue disease risk and provide a pathway to monitor the changing risk profile of populations and to quantifying risk profiles of candidate vaccines.