Dynamic Feature Subspaces Selection for Decision in a Nonstationary Environment

Abstract : The presence of noise, loss of information or feature nonstationarity in data is the limiting factor for many machine learning decision systems. Previous research has shown that relevant feature selection may be helpful to alleviate the impact of these possible perturbations. This paper presents a dynamical feature subspaces selection method based on ensembles of one-class Support Vector Machine (SVM), with the objective to optimize the performance of a decision system in such a nonstationary environment. Our method is predicated on the assumption that only the performance of the classifiers using perturbed features is degraded. We propose a mechanism for constructing an ensemble of classifiers based on a large number of feature subspaces generated from the initial full-dimensional space. In the phase of classification, the ensemble system is capable to select adaptively feature subspaces which are supposed to be immune to the nonstationary disturbance and to make the final decision by combining the individual decisions of classifiers built in these subspaces. One characteristic of this method is that we use the one-class SVM ensemble to accomplish simultaneously the tasks of feature subspace selection and classification. The effectiveness of the proposed method has been demonstrated through the experiments conducted in the context of textured image classification.