We assessed the added value and limitations of generating directly estimated ZIP Code-level estimates by aggregating 5 years of data from an annual cross-sectional survey, the New York City Community Health Survey (n = 44,886) from 2009 to 2013, that were designed to provide reliable estimates only of larger geographies. Survey weights generated directly-observed ZIP Code (n = 128) level estimates. We assessed the heterogeneity of ZIP Code-level estimates within coarser United Hospital Fund (UHF) neighborhood areas (n = 34) by using the Rao-Scott Chi-Square test and one-way ANOVA. Orthogonal linear contrasts assessed whether there were linear trends at the UHF level from 2009 to 2013. 22 of 37 health indicators were reliable in over 50% of ZIP Codes. 14 of the 22 variables showed heterogeneity in ≥4 UHFs. Variables for drinking, nutrition, and HIV testing showed heterogeneity in the most UHFs (9-24 UHFs). In half of the 32 UHFs, >20% variables had within-UHF heterogeneity. Flu vaccination and sugary beverage consumption showed significant time trends in the largest number of UHFs (12 or more UHFs). Overall, heterogeneity of ZIP Code-level estimates suggests that there is value in aggregating 5 years of data to make direct small area estimates.
Keywords: Population health; Survey; ZIP Code.