Baffin Bay hosts the largest and most productive of the Arctic polynyas: the North Water (NOW). Despite its significance and active role in water mass formation, the history of the NOW beyond the observational era remains poorly known. We reconcile the previously unassessed relationship between long-term NOW dynamics and ocean conditions by applying a multiproxy approach to two marine sediment cores from the region that, together, span the Holocene. Declining influence of Atlantic Water in the NOW is coeval with regional records that indicate the inception of a strong and recurrent polynya from ~ 4400 yrs BP, in line with Neoglacial cooling. During warmer Holocene intervals such as the Roman Warm Period, a weaker NOW is evident, and its reduced capacity to influence bottom ocean conditions facilitated northward penetration of Atlantic Water. Future warming in the Arctic may have negative consequences for this vital biological oasis, with the potential knock-on effect of warm water penetration further north and intensified melt of the marine-terminating glaciers that flank the coast of northwest Greenland.