Mediating between filter and wrapper via probabilistic models, A hybrid feature selection framework for multi-label data

Traditional hybrid feature selection methods rely on the collaboration between filter and wrapper techniques but often lack proper interaction mechanisms, limiting their ability to capture complex feature dependencies. Filter methods evaluate features individually and may overlook important combinations, whereas wrapper methods account for feature interactions but are computationally intensive and prone to overfitting. Without an Artificial Intelligence-powered probabilistic cohesive mechanism to align filter and wrapper components, these methods suffer from inconsistent decision-making, leading to local convergence and reduced effectiveness. In this paper, a novel three-component framework of filter-interface-wrapper is introduced, incorporating an interface layer between filter and wrapper. This layer leverages the inherent differences between objectives of filter and wrapper to enhance exploration capabilities and facilitate escape from local optima. It manages the transition procedure during collaboration between filter and wrapper by initially focusing on filter insights and gradually converging to the wrapper as it matures. The interface employs learnable Importance Probability Models (IPMs) that start with filter information and iteratively refine feature significance through population generation and mutation in the wrapper. These IPMs are updated after each iteration based on wrapper outputs to stay informed of its performance. By combining multiple IPMs with an evolutionary wrapper, the framework improves the exploration–exploitation balance in the solution space. Experiments on 15 multi-label datasets demonstrate significant improvements in feature selection, balancing efficiency and predictive power in complex scenarios.