One widely adopted approach for effectively handling class-imbalanced datasets is data over-sampling, which involves generating synthetic samples for the minority class. Among these methods, the synthetic minority oversampling technique (SMOTE) is one of the most commonly used algorithms across various domain-specific imbalanced datasets. Numerous SMOTE variants have been proposed in the literature to enhance performance. However, no single algorithm consistently outperforms others across all types of domain-specific datasets.
This paper introduces several ensemble over-sampling methods, which combine the results of multiple over-sampling algorithms applied in both parallel and sequential manners. In the experiments, 58 binary-class datasets are used in Study One, while Study Two involves 10 datasets from various medical domains, encompassing both binary and multi-class classification tasks. Additionally, four baseline over-sampling algorithms, i.e. SMOTE, Poly-Fit-SMOTE, ProWSyn and SMOTE-IPF, are evaluated alongside three classifiers: C4.5, SVM and XGBoost.
The experimental results demonstrate that the parallel-based ensemble method, which combines ProWSyn and SMOTE-IPF and selects the k nearest synthetic samples around the corresponding minority class centers, yields the best performance. Using this ensemble approach, the XGBoost classifier achieves superior AUC and G-mean results compared to those obtained with the four baseline over-sampling algorithms and most other ensemble methods.
This paper presents parallel and serial ensemble methods that integrate multiple over-sampling results to enhance the performance of individual over-sampling algorithms. The most effective ensemble method and its associated combined algorithms can serve as a representative baseline for future research in class imbalance learning.
