We developed a new flexible approach for a co-analysis of multi-modal brain imaging data using a non-parametric framework. In this approach, results from separate analyses on different modalities are combined using a combining function and assessed with a permutation test. This approach identifies several cross-modality relationships, such as concordance and dissociation, without explicitly modeling the correlation between modalities. We applied our approach to structural and perfusion MRI data from an Alzheimer's disease (AD) study. Our approach identified areas of concordance, where both gray matter (GM) density and perfusion decreased together, and areas of dissociation, where GM density and perfusion did not decrease together. In conclusion, these results demonstrate the utility of this new non-parametric method to quantitatively assess the relationships between multiple modalities.