AXIOMATIC DERIVATION OF THE MUTUAL INFORMATION PRINCIPLE AS A METHOD OF INDUCTIVE INFERENCE

The Mutual Information Princip le (MIP) has already been used in various areas, as a generalization of the Maximum Entropy Principle (MEP), in the very common situation where our measurements of a random variable contain errors having some known average value. An axiomatic derivation of the MIP is given below, in order to place it in a rigorous mathematical framework with the least possible intuitive arguments. The procedure followed is similar to the one proposed by Shore and Johnson for the Minimum Cross‐entropy Principle, and some relationships between the two methods of inductive inference are pointed out.