Ventral attention network (VAN), associated with cognitive performance, is one of the functional networks that are most vulnerable in white matter hyperintensity (WMH). Considering the global interaction of networks for cognitive performance, we hypothesized that VAN-related between-network connectivity might play a role in maintaining cognition in patients with WMH. We included 139 participants for both cross-sectional and longitudinal analysis from CIRCLE study (ClinicalTrials.gov ID: NCT03542734) between January 2014 and January 2021. Differences of VAN-related between-network connectivity were compared between normal-cognition (NC) and cognitive-impairment (CI) groups cross-sectionally, and between cognitive-decline (CD) and cognitive non-decline (CND) groups longitudinally by using t-test. False Discovery Rate was used for multiple comparison correction. The relationship between the network connectivity and WMH was tested on linear and quadratic models. Subgroup analysis of different WMH burdens were performed to test the difference of network connectivity between NC and CI groups. Among VAN-related between-network connectivity, only VAN-Visual Network (VN) connectivity was higher both in NC (n = 106) and CND (n = 113) groups versus CI (n = 33) and CD groups (n = 26), respectively. There was an inverted U-shaped relation between periventricular WMH (PWMH) burden and VAN-VN connectivity. Normal-cognition participants had higher VAN-VN connectivity among high, but not low PWMH burden subgroups. These findings suggest that the VAN-VN connectivity plays an important role in maintaining cognitive performance in WMH patients. It may serve as a unique marker for cognitive prediction and a potential target for intervention to prevent cognitive decline in WMH patients.