Extraction of Important Factors in a High-Dimensional Data Space: An Application for High-Growth Firms

Entropy (Basel). 2023 Mar 10;25(3):488. doi: 10.3390/e25030488.

Abstract

We introduce a new non-black-box method of extracting multiple areas in a high-dimensional big data space where data points that satisfy specific conditions are highly concentrated. First, we extract one-dimensional areas where the data that satisfy specific conditions are mostly gathered by using the Bayesian method. Second, we construct higher-dimensional areas where the densities of focused data points are higher than the simple combination of the results for one dimension, and then we verify the results through data validation. Third, we apply this method to estimate the set of significant factors shared in successful firms with growth rates in sales at the top 1% level using 156-dimensional data of corporate financial reports for 12 years containing about 320,000 firms. We also categorize high-growth firms into 15 groups of different sets of factors.

Keywords: Bayesian method; big data; feature selection; high-growth firms; variable selection.

Grants and funding

This research was funded by the Center for TDB Advanced Data Analysis and Modeling, Tokyo Institute of Technology for academic research purposes. TEIKOKU DATABANK, Ltd. supported our research by providing the data related to Japanese business firms.