In the environment of unknown mutual coupling, many works on direction-of-arrival (DOA) estimation with sensor array are prone to performance degradation or even failure. Moreover, there are few literatures on off-grid direction finding using regularized sparse recovery technology. Therefore, the scenario of off-grid DOA estimation in sensor array with unknown mutual coupling is investigated, and then a reweighted off-grid Sparse Spectrum Fitting (Re-OGSpSF) approach is developed in this article. Inspired by the selection matrix, an undisturbed array output is formed to remove the unknown mutual coupling effect. Subsequently, a refined off-grid SpSF (OGSpSF) recovery model is structured by integrating the off-grid error term obtained from the first-order Taylor approximation of the higher-order term into the underlying on-grid sparse representation model. After that, a novel Re-OGSpSF framework is formulated to recover the sparse vectors, where a weighted matrix is developed by the MUSIC-like spectrum function to enhance the solution's sparsity. Ultimately, off-grid DOA estimation can be realized with the help of the recovered sparse vectors. Thanks to the off-grid representation and reweighted strategy, the proposed method can effectively and efficiently achieve high-precision continuous DOA estimation, making it favorable for real-time direction finding. The simulation results validate the superiority of the proposed method.
Keywords: DOA estimation; Sparse Spectrum Fitting; off-grid error; reweighted sparse recovery; sensor array; unknown mutual coupling.