The correlation map of neurons emerges as an important mathematical framework for a spectrum of applications including neural circuit modeling, neurologic disease bio-marking and neuroimaging. However, constructing a correlation map is computationally expensive, especially when the number of neurons is large. This paper proposes a hardware design using hierarchical systolic arrays to calculate pairwise correlations between neurons. Through mapping a computationally efficient algorithm for cross-correlation onto a massively parallel structure, the hardware is able to construct the correlation maps in a much shorter time. The proposed architecture was evaluated using a field programmable gate array. The results show that the computational delay of the hardware for constructing correlation maps increases linearly with the number of neurons, whereas the growth of delay is quadratic for a software-based serial approach. Also, the efficiency of our method for detecting abnormal behaviors of neural circuits is demonstrated by analyzing correlation maps of retinal neurons.