Stowell-Plumbley estimator

Back to Shannon differential entropy

The Stowell-Plumbley differential entropy estimator is a non-parametric estimator based on approximating the underlying probability density function by a cover of disjoint axis-aligned boxes with uniform densities. The implementation of this estimator in TIM has an average runtime of ''O(m log(m) + m n)'', where ''m'' is the number of samples and ''n'' is the dimension. This is very fast compared to the other estimators. Unfortunately, it is not very accurate in higher dimensions, and thus we do not recommend using it in dimensions higher than 3. For higher dimensions you should use the estimators based on nearest neighbors, such as Kozachenko-Leonenko or Nilsson-Kleijn.

Assumptions

References

Fast Multidimensional Entropy Estimation by k-d Partitioning,
Dan Stowell, Mark D. Plumbley,
IEEE Signal Processing Letters, Vol. 16, No. 6, June 2009.

Files

Differential entropy estimation

Stowell-Plumbley recursive partition estimator

differential_entropy_sp.h

differential_entropy_sp.hpp

Differential entropy estimation

Stowell-Plumbley recursive partition estimator

differential_entropy_sp.m

differential_entropy_sp

matlab_differential_entropy_sp.cpp