On a Generalization of the Jensen-Shannon Divergence and the Jensen-Shannon Centroid

Entropy (Basel). 2020 Feb 16;22(2):221. doi: 10.3390/e22020221.

Abstract

The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar α -Jensen-Bregman divergences and derive thereof the vector-skew α -Jensen-Shannon divergences. We prove that the vector-skew α -Jensen-Shannon divergences are f-divergences and study the properties of these novel divergences. Finally, we report an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probability densities belonging to a mixture family: This includes the case of the Jensen-Shannon centroid of a set of categorical distributions or normalized histograms.

Keywords: Bregman divergence; Jensen diversity; Jensen–Bregman divergence; Jensen–Shannon centroid; Jensen–Shannon divergence; capacitory discrimination; difference of convex (DC) programming; f-divergence; information geometry; mixture family.