THE RATE DISTORTION THEORY AND PROBABILITY MODELS

The Rate Distortion Theory is a branch of the Information Theory applicable to the case when the entropy of the source exceeds the capacity of the Channel. A rate distortion function R(D) is defined between the input and output alphabets X, Y of a channel. It can be shown that it is possible to design a communication system which achieves a fidelity D when the capacity of the channel C is greater than R(D). In this paper, the formulation of the Rate Distortion Theory is used for the problem of derived probability models. The variables X, Y and the Channel are given new interpretations, and the result is an ability to pick a derived probability model for Y when X is of a known probability structure. The fidelity criterion assumes the rle of an error function in this terminology. Two specific cases are discussed.