Two-hidden-layer feed-forward networks are universal approximators: A constructive approach

Abstract

It is well-known that artificial neural networks are universal approximators. The classical existence result proves that, given a continuous function on a compact set embedded in an n-dimensional space, there exists a one-hidden-layer feed-forward network that approximates the function. In this paper, a constructive approach to this problem is given for the case of a continuous function on triangulated spaces. Once a triangulation of the space is given, a two-hidden-layer feed-forward network with a concrete set of weights is computed. The level of the approximation depends on the refinement of the triangulation.

Keywords: Multi-layer feed-forward network; Simplicial Approximation Theorem; Triangulations; Universal Approximation Theorem.