In this paper, a WiFi and visual fingerprint localization model based on low-rank fusion (LRF-WiVi) is proposed, which makes full use of the complementarity of heterogeneous signals by modeling both the signal-specific actions and interaction of location information in the two signals end-to-end. Firstly, two feature extraction subnetworks are designed to extract the feature vectors containing location information of WiFi channel state information (CSI) and multi-directional visual images respectively. Then, the low-rank fusion module efficiently aggregates the specific actions and interactions of the two feature vectors while maintaining low computational complexity. The fusion features obtained are used for position estimation; In addition, for the CSI feature extraction subnetwork, we designed a novel construction method of CSI time-frequency characteristic map and a double-branch CNN structure to extract features. LRF-WiVi jointly learns the parameters of each module under the guidance of the same loss function, making the whole model more consistent with the goal of fusion localization. Extensive experiments are conducted in a complex laboratory and an open hall to verify the superior performance of LRF-WiVi in utilizing WiFi and visual signal complementarity. The results show that our method achieves more advanced positioning performance than other methods in both scenarios.
Keywords: WiFi channel state information; end-to-end; fingerprint localization; low rank fusion; visual images.