Star algorithm for neural network ensembling

Abstract

Neural network ensembling is a common and robust way to increase model efficiency. In this paper, we propose a new neural network ensemble algorithm based on Audibert's empirical star algorithm. We provide optimal theoretical minimax bound on the excess squared risk. Additionally, we empirically study this algorithm on regression and classification tasks and compare it to most popular ensembling methods.

Keywords: Deep neural networks; Ensemble methods; Excess risk bounds; Offset Rademacher complexity.