Limit Theorems as Blessing of Dimensionality: Neural-Oriented Overview

Entropy (Basel). 2021 Apr 22;23(5):501. doi: 10.3390/e23050501.

Abstract

As a system becomes more complex, at first, its description and analysis becomes more complicated. However, a further increase in the system's complexity often makes this analysis simpler. A classical example is Central Limit Theorem: when we have a few independent sources of uncertainty, the resulting uncertainty is very difficult to describe, but as the number of such sources increases, the resulting distribution gets close to an easy-to-analyze normal one-and indeed, normal distributions are ubiquitous. We show that such limit theorems often make analysis of complex systems easier-i.e., lead to blessing of dimensionality phenomenon-for all the aspects of these systems: the corresponding transformation, the system's uncertainty, and the desired result of the system's analysis.

Keywords: curse and blessing of dimensionality; limit theorems; neural networks.

Publication types

  • Review