Maximum Entropy Modelling of Mixture Distributions Available to Purchase

Shannon’s entropy is usually defined separately for discrete, and for (absolutely) continuous random variables. However, many random variables encountered in practice have mixed (discrete‐continuous) distributions. A simple expression for the entropy of random variables with mixed (discrete‐continuous) distributions is derived in terms of the usual entropy definitions. In addition, the maximum entropy problem in the general setting of mixture distributions is discussed.