An *entropy combination* of a random variable ''X'' is defined by

''C(X) = sum_{k = 1}^m c_k H(X_{L_k})''

where

- ''X = (x_1, ..., x_d) in RR^d'',
- ''H'' is the differential entropy,
- ''forall k in [1, p]: L_k sub [1, d] sub ZZ'',
- ''X_L = [x_{l_1}, ..., x_{l_{|L|}}]'',
- ''c in RR^m'', and
- ''sum_{k = 1}^m c_k |L_k| = 0''.

It can be shown that the last condition is a necessary and sufficient condition for a discrete entropy combination to converge to a continuous entropy combination under a shrinking tiling of ''RR^d''.

The implementation of the entropy combination estimator requires additionally that each ''L_k'' is an interval. This allows to save memory by allowing the memory for the joint signal to be shared with the marginal signals. It is easily seen that mutual information, partial mutual information, transfer entropy, and partial transfer entropy can all be arranged to have this property.

matlab_entropy_combination.cpp