Table 1 Categories of information theory... | Emerald Publishing

Table 1

Categories of information theory based methods.

Refs.	Information method	Description
[12]	Mutual information maximization (or information gain)	Mutual information maximization (also known as information gain) feature importance level by its correlation with a class label. The assumption of this method is that in the event of a feature having strong correlations with a class label, it can be used to accomplish good classification performance.
[13]	Mutual information feature selection (MIFS)	MIFS was introduced to resolve the limitation of mutual information maximization. It can take into consideration feature relevance and feature redundancy at the same time during feature selection phase.
[8]	Minimum redundancy maximum relevance (mRMR)	To reduce the effect of feature redundancy, mRMR is used to select features that have a high correlation with the class (output) and low correlations among themselves.
[14]	Conditional infomax feature extraction	Conditional infomax feature extraction was introduced to resolve the gaps in both MIFS and mRMR, which both consider feature relevance and feature redundancy at the same time.This method assumes that given the class labels if feature redundancy is stronger than intra-feature redundancy then there is a negative effect on feature selection.
[15]	Joint mutual information	Since MIFS and mRMR are useful in lowering feature redundancy during the process of feature selection, this alternative method known as joint mutual information was recommended to increase the sharing of complementary information between a new unselected feature and the selected feature when the class labels are given.
[16]	Conditional mutual information maximization (CMIM)	In CMIM, features are iteratively selected to enhance the sharing of mutual information with class labels when the selected features are given. In other words, CMIM does not select the feature that is most similar to the previously selected ones, even though the predictive power of that feature for the class labels would be strong.
[17]	Informative fragments	The intuition behind informative fragments is that adding a new feature should maximize the value of conditional information that the new feature and the existing features share rather than the information that the features and the class share.
[18]	Interaction capping	Interaction capping is similar to CMIM, but instead of restricting the formula, interaction capping is non-negative.
[19]	Double input symmetrical relevance	Another type of information theory based method known as double input symmetrical relevance takes advantage of normalization approaches to normalize mutually exclusive information.
[9]	Fast correlation based filtering (Yu and Liu, 2003)	This filtering method takes advantage of feature-feature and feature-class correlations at the same time, using feature selection methods that cannot be turned into a unified conditional likelihood maximization framework easily.

Refs.	Information method	Description
[12]	Mutual information maximization (or information gain)	Mutual information maximization (also known as information gain) feature importance level by its correlation with a class label. The assumption of this method is that in the event of a feature having strong correlations with a class label, it can be used to accomplish good classification performance.
[13]	Mutual information feature selection (MIFS)	MIFS was introduced to resolve the limitation of mutual information maximization. It can take into consideration feature relevance and feature redundancy at the same time during feature selection phase.
[8]	Minimum redundancy maximum relevance (mRMR)	To reduce the effect of feature redundancy, mRMR is used to select features that have a high correlation with the class (output) and low correlations among themselves.
[14]	Conditional infomax feature extraction	Conditional infomax feature extraction was introduced to resolve the gaps in both MIFS and mRMR, which both consider feature relevance and feature redundancy at the same time.This method assumes that given the class labels if feature redundancy is stronger than intra-feature redundancy then there is a negative effect on feature selection.
[15]	Joint mutual information	Since MIFS and mRMR are useful in lowering feature redundancy during the process of feature selection, this alternative method known as joint mutual information was recommended to increase the sharing of complementary information between a new unselected feature and the selected feature when the class labels are given.
[16]	Conditional mutual information maximization (CMIM)	In CMIM, features are iteratively selected to enhance the sharing of mutual information with class labels when the selected features are given. In other words, CMIM does not select the feature that is most similar to the previously selected ones, even though the predictive power of that feature for the class labels would be strong.
[17]	Informative fragments	The intuition behind informative fragments is that adding a new feature should maximize the value of conditional information that the new feature and the existing features share rather than the information that the features and the class share.
[18]	Interaction capping	Interaction capping is similar to CMIM, but instead of restricting the formula, interaction capping is non-negative.
[19]	Double input symmetrical relevance	Another type of information theory based method known as double input symmetrical relevance takes advantage of normalization approaches to normalize mutually exclusive information.
[9]	Fast correlation based filtering (Yu and Liu, 2003)	This filtering method takes advantage of feature-feature and feature-class correlations at the same time, using feature selection methods that cannot be turned into a unified conditional likelihood maximization framework easily.