A Streaming model for Generalized Rayleigh with extension to Minimum Noise Fraction

Proc IEEE Int Conf Big Data. 2019 Dec:2019:74-83. doi: 10.1109/BigData47090.2019.9006512. Epub 2020 Feb 24.

Abstract

The Rayleigh quotient optimization is the maximization of a rational function, or a max-min problem, with simultaneous maximization of the numerator function and minimization of the denominator function. Here, we describe a low-rank, streaming solution for Rayleigh quotient optimization applicable for big-data scenarios where the data matrix is too large to be fully loaded into main memory. We apply this for a maximization of the Signal to Noise ratio of big-data, of very large static and dynamic data. Our implementation is shown to achieve faster processing time compared to a standard data read into memory. We demonstrate the trade-offs with synthetic and real data, on different scales to validate the approach in terms of accuracy, speed and storage.

Keywords: Generalized Rayleigh Quotient; Hyperspectral Image (HSI); Low-Rank Projection; Minimum Noise Fraction (MNF); Streaming.