The COVID-19 pandemic has yielded disproportionate impacts on communities of color in New York City (NYC). Researchers have noted that social disadvantage may result in limited capacity to socially distance, and consequent disparities. We investigate the association between neighborhood social disadvantage and the ability to socially distance, infections, and mortality in Spring 2020. We combine Census Bureau and NYC open data with SARS-CoV-2 testing data using supervised dimensionality-reduction with Bayesian Weighted Quantile Sums regression. The result is a ZIP code-level index with weighted social factors associated with infection risk. We find a positive association between neighborhood social disadvantage and infections, adjusting for the number of tests administered. Neighborhood disadvantage is also associated with a proxy of the capacity to socially isolate, NYC subway usage data. Finally, our index is associated with COVID-19-related mortality.