A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector maching
Received date: 2007-09-03
Revised date: 2008-02-17
Online published: 2008-05-25
Supported by
国家自然科学基金(40601010); 中国博士后基金(20060390208); "985工程"GIS与遥感的地学应用科技创新平台资助(105203200400006); 国家杰出青年科学基金资助项目(40525002)
The high-dimensional feature vectors of hyper spectral data often impose a high computational cost as well as the risk of "over fitting" when classification is performed. Therefore it is necessary to reduce the dimensionality through ways like feature selection. Currently, there are two kinds of feature selection methods: filter methods and wrapper methods. The former kind requires no feedback from classifiers and estimates the classification performance indirectly. The latter kind evaluates the "goodness" of selected feature subset directly based on the classification accuracy. Many experimental results have proved that the wrapper methods can yield better performance, although they have the disadvantage of high computational cost. In this paper, we present a Genetic Algorithm (GA) based wrapper method for classification of hyper spectral data using Support Vector Machine (SVM), a state-of-art classifier that has found to be success in a variety of areas. The genetic algorithm (GA), which seeks to solve optimization problems using the methods of evolution, specifically survival of the fittest, was used to optimize both the feature subset, i.e. band subset, of hyper spectral data and SVM kernel parameters simultaneously. A special strategy was adopted to reduce computation cost caused by the high-dimensional feature vectors of hyper spectral data when the feature subset part of chromosome was designed. The GA-SVM method was realized using the ENVI/IDL language, and was then tested by applying a HYPERION hyper spectral image. Comparison of the optimized results and the un-optimized results showed that the GA-SVM method could significantly reduce the computation cost while improving the classification accuracy. The number of bands used for classification was reduced from 198 to 13, while the classification accuracy increased from 88.81% to 92.51%. The optimized values of the two SVM kernel parameters were 95.0297 and 0.2021, respectively, which were different from the default values as used in the ENVI software. In conclusion, the proposed wrapper feature selection method GA-SVM can optimize feature subsets and SVM kernel parameters at the same time, therefore can be applied in feature selection of the hyper spectral data.
ZHUO Li, ZHENG Jing, WANG Fang, LI Xia, AI Bin, QIAN Jun-ping . A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector maching[J]. GEOGRAPHICAL RESEARCH, 2008 , 27(3) : 493 -501 . DOI: 10.11821/yj2008030002
[1] 陈蜜,易尧华,刘志刚, 等. 基于分块特性的高光谱影像波段选取方法的研究. 测绘通报,2006(3): 10~13.
[2] 杨哲海, 张雅争, 宫大鹏,等. 基于Tabu搜索的高光谱影像特征选择. 海洋测绘, 2006, 26(4): 11~14.
[3] 王长耀, 刘正军, 颜春燕. 成像光谱数据特征选择及小麦品种识别实验研究. 遥感学报, 2006,10(2):249~255.
[4] Liu Y, Zheng Y F. FS_SFS:A novel feature selection method for support vector machines. Pattern Recognition, 2006, 39: 1333~1345.
[5] Kohavi R, John G H. Wrappers for feature subset selection. Artif. Intell. ,1997, 97: 273~324.
[6] Huang Ch L, Wang Ch J.A GA-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 2006, 31: 231~240.
[7] Mao K Z.Feature subset selection for support vector machines through discriminative function pruning analysis. IEEE Transactions on Systems, Man, and Cybernetics, 2004, 34(1): 60~67.
[8] Yu E, Cho S. Ensemble based on GA wrapper feature selection. Computers & Industrial Engineering, 2006, 51(1): 111~116.
[9] Bi J, Bennett K P, Embrechts M, et al. Dimensionality reduction via sparse support vector machines. J. Mach. Learn. Res. 2003, 3: 1229~1243.
[10] Sikora R, Piramuthu S. Framework for efficient feature selection in genetic algorithm based data mining. European Journal of Operational Research, 2007, 180(2): 723~737.
[11] Cover T M. The best two independent measurements are not the two best. IEEE Transactions on Systems, Man, and Cybernetics, 1974, 1: 116~117.
[12] Elashoff J D, Elashoff R M, Goldman G E.On the choice of variables in classification problems with dichotomous variables. Biometrika, 1967, 54: 668~670.
[13] Toussaint G T.Note on optimal selection of independent binary-valued features for pattern recognition. IEEE Transactions on Information Theory IT, 1971,17: 618.
[14] Meiri R, Zahavi J. Using simulated annealing to optimize the feature selection problem in marketing applications. European Journal of Operational Research, 2006, 171: 842~858.
[15] Liu H, Motoda H. Feature Extraction, Construction, and Selection: A Data Mining Perspective.Dordrecht: Kluwer Academic Publishers,1998.
[16] Marill T, Green D M. On the effectiveness of receptors in recognition systems. IEEE Trans. Inf. Theory, 1963, 9: 11~17.
[17] Raymer M L, Punch W F, Goodman E D, et al. Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. , 2000, 4(2): 164~171.
[18] 杨青生, 黎夏. 基层遗传算法自动获取CA模型的参数——以东莞市城市发展模拟为例.地理研究,2007,26(2): 226~231.
[19] Corts C, Vapnik V. Support vector networks. Machine Learning, 1995, 20: 273~ 297.
[20] Vapnik V N.The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
[21] Cortes C, Vapnik V N. Support vector networks. Mach. Learn. , 1995, 20(3): 273~297.
[22] 邓乃扬, 田英杰. 数据挖掘中的新方法——支持向量机. 北京: 科学出版社, 2004.
[23] Pontil M, Verri A.Support vector machines for 3D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(6), 637~646.
[24] Bradley P S, Mangasarian O L, Street W N.Feature selection via mathematical programming. INFORMS Journal on Computing, 1998, 10: 209~217.
[25] Guyon I, Weston J, Barnhill S, et al.Gene selection for cancer classification using support vector machines. Machine Learning, 2002, 46(1~3): 389~422.
[26] Yu E, Cho S.GA-SVM wrapper approach for feature selection in keystroke dynamics identity verification. Proceedings of 2003 INNS-IEEE International Joint Conference on Neural Networks, 2253~2257.
[27] Weston J, Mukherjee S, Chapelle O, et al. Feature selection for SVMs. Advances in Neural Information Processing Systems, 13.Cambridge, MA: MIT Press, 2001.
[28] Chapelle O, Vapnik V, Bousquet O, et al. Choosing multiple parameters for support vector machines. Machine Learning, 2002, 46 (123): 131~159.
[29] Lee J H, L in C J.Automatic Model Selection for Support Vector Machines. Taipei: Taiwan University, 2000.
[30] Bengio Y.Gradient-based optimization of hyperparameters. Neural Computation, 2000, 12(8): 1889~1900.
[31] Keerthi S S.Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transon Neural Networks, 2002, 13(5):1225~1229.
[32] Hsu C W, Chang C C, Lin C J.A practical guide to support vector classification, 2003. Available at:http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.
[33] Jarecke P. EO-1/Hyperion Science Data User's Guide. TRW Space, Defense & Information System, 2001.
/
| 〈 |
|
〉 |