Bonev, Boyan, Escolano, Francisco, Cazorla, Miguel
A novel information theory method for filter feature selection
BONEV, Boyan; ESCOLANO, Francisco; CAZORLA, Miguel Ángel. "A novel information theory method for filter feature selection". En: MICAI 2007: Advances in Artificial Intelligence 6th Mexican International Conference on Artificial Intelligence, Aguascalientes, Mexico, November 4-10, 2007, Proceedings / Alexander Gelbukh, Ángel Fernando Kuri Morales (Eds.). Berlin : Springer, 2007. (Lecture Notes in Computer Science; 4827). ISBN 978-3-540-76630-8, pp. 431-440
URI: http://hdl.handle.net/10045/23400
DOI: 10.1007/978-3-540-76631-5_41
ISSN: 0302-9743 (Print)
ISBN: 978-3-540-76630-8
Abstract: 
In this paper, we propose a novel filter for feature selection. Such filter relies on the estimation of the mutual information between features and classes. We bypass the estimation of the probability density function with the aid of the entropic-graphs approximation of Rényi entropy, and the subsequent approximation of the Shannon one. The complexity of such bypassing process does not depend on the number of dimensions but on the number of patterns/samples, and thus the curse of dimensionality is circumvented. We show that it is then possible to outperform a greedy algorithm based on the maximal relevance and minimal redundancy criterion. We successfully test our method both in the contexts of image classification and microarray data classification.
Keywords:Filter feature selection, Information theory method, Mutual information estimation, Entropy estimation
Springer Berlin / Heidelberg
info:eu-repo/semantics/conferenceObject