A class of estimators of the Rényi and Tsallis entropies of an unknown distribution f in ℝm is presented. These estimators are based on the kth nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon’s entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.
Publié le : 2008-10-15
Classification:
Entropy estimation,
estimation of statistical distance,
estimation of divergence,
nearest-neighbor distances,
Rényi entropy,
Havrda–Charvát entropy,
Tsallis entropy,
94A15,
62G20
@article{1223908088,
author = {Leonenko, Nikolai and Pronzato, Luc and Savani, Vippal},
title = {A class of R\'enyi information estimators for multidimensional densities},
journal = {Ann. Statist.},
volume = {36},
number = {1},
year = {2008},
pages = { 2153-2182},
language = {en},
url = {http://dml.mathdoc.fr/item/1223908088}
}
Leonenko, Nikolai; Pronzato, Luc; Savani, Vippal. A class of Rényi information estimators for multidimensional densities. Ann. Statist., Tome 36 (2008) no. 1, pp. 2153-2182. http://gdmltest.u-ga.fr/item/1223908088/