基于GA-SVM封装算法的高光谱数据特征选择

doi:10.11821/yj2008030002

地理研究 ›› 2008, Vol. 27 ›› Issue (3): 493-501.doi: 10.11821/yj2008030002

基于GA-SVM封装算法的高光谱数据特征选择

卓莉¹, 郑璟², 王芳^1,3, 黎夏¹, 艾彬¹, 钱峻屏¹

1. 中山大学地理科学与规划学院, 广州 510275;
2. 广东省气候中心, 广州 510080;
3. 广州大学地理科学学院,广州510006

收稿日期:2007-09-03 修回日期:2008-02-17 出版日期:2008-05-25 发布日期:2008-05-25
作者简介:卓莉| 女, 博士,讲师。主要从事资源环境遥感与地理信息系统研究。E-mail:zhuoli@mail.sysu.edu.cn *通讯作者 :郑璟。E-mail:jingzheng2005@gmail.com
基金资助:
国家自然科学基金(40601010); 中国博士后基金(20060390208); "985工程"GIS与遥感的地学应用科技创新平台资助(105203200400006); 国家杰出青年科学基金资助项目(40525002)

A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector maching

ZHUO Li¹, ZHENG Jing², WANG Fang^1,3, LI Xia¹, AI Bin¹, QIAN Jun-ping¹

1. School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China;
2. Guang Dong Climate Center, Guangzhou 510080, China;
3. School of Geographical Sciences, Guangzhou University, Guangzhou 510006, China

Received:2007-09-03 Revised:2008-02-17 Online:2008-05-25 Published:2008-05-25
Supported by:
国家自然科学基金(40601010); 中国博士后基金(20060390208); "985工程"GIS与遥感的地学应用科技创新平台资助(105203200400006); 国家杰出青年科学基金资助项目(40525002)

摘要/Abstract

摘要：

封装型的特征选择算法相对于过滤算法而言更有助于提高分类精度,因此在当前计算技术及效率快速发展的背景下必将成为未来之趋势。本文以支持向量机(SVM)为分类器,遗传算法(GA)为特征子集的搜索算法,构建了封装型的特征选择算法GA-SVM,并用ENVI/IDL语言编程实现,最后以HYPERION高光谱数据为例对算法予以应用。结果表明,GA-SVM算法可从196个波段中选择出13个波段,同时分类精度较不做特征选择时提高了约4%。由此可见,GA-SVM封装型特征选择算法具有较好的同时优化特征子集和SVM核函数的性能,可为当前高光谱数据的特征选择提供一个较好的算法。

关键词: 特征选择, 高光谱, 遗传算法(GA), 支持向量机(SVM)

Abstract:

The high-dimensional feature vectors of hyper spectral data often impose a high computational cost as well as the risk of "over fitting" when classification is performed. Therefore it is necessary to reduce the dimensionality through ways like feature selection. Currently, there are two kinds of feature selection methods: filter methods and wrapper methods. The former kind requires no feedback from classifiers and estimates the classification performance indirectly. The latter kind evaluates the "goodness" of selected feature subset directly based on the classification accuracy. Many experimental results have proved that the wrapper methods can yield better performance, although they have the disadvantage of high computational cost. In this paper, we present a Genetic Algorithm (GA) based wrapper method for classification of hyper spectral data using Support Vector Machine (SVM), a state-of-art classifier that has found to be success in a variety of areas. The genetic algorithm (GA), which seeks to solve optimization problems using the methods of evolution, specifically survival of the fittest, was used to optimize both the feature subset, i.e. band subset, of hyper spectral data and SVM kernel parameters simultaneously. A special strategy was adopted to reduce computation cost caused by the high-dimensional feature vectors of hyper spectral data when the feature subset part of chromosome was designed. The GA-SVM method was realized using the ENVI/IDL language, and was then tested by applying a HYPERION hyper spectral image. Comparison of the optimized results and the un-optimized results showed that the GA-SVM method could significantly reduce the computation cost while improving the classification accuracy. The number of bands used for classification was reduced from 198 to 13, while the classification accuracy increased from 88.81% to 92.51%. The optimized values of the two SVM kernel parameters were 95.0297 and 0.2021, respectively, which were different from the default values as used in the ENVI software. In conclusion, the proposed wrapper feature selection method GA-SVM can optimize feature subsets and SVM kernel parameters at the same time, therefore can be applied in feature selection of the hyper spectral data.

Key words: feature selection, hyperspectral, genetic algorithm, supported vector machine

卓莉, 郑璟, 王芳, 黎夏, 艾彬, 钱峻屏. 基于GA-SVM封装算法的高光谱数据特征选择[J]. 地理研究, 2008, 27(3): 493-501.

ZHUO Li, ZHENG Jing, WANG Fang, LI Xia, AI Bin, QIAN Jun-ping. A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector maching[J]. GEOGRAPHICAL RESEARCH, 2008, 27(3): 493-501.

参考文献

[1] 陈蜜,易尧华,刘志刚, 等. 基于分块特性的高光谱影像波段选取方法的研究. 测绘通报,2006(3): 10~13.

[2] 杨哲海, 张雅争, 宫大鹏,等. 基于Tabu搜索的高光谱影像特征选择. 海洋测绘, 2006, 26(4): 11~14.

[3] 王长耀, 刘正军, 颜春燕. 成像光谱数据特征选择及小麦品种识别实验研究. 遥感学报, 2006,10(2):249~255.

[4] Liu Y, Zheng Y F. FS_SFS:A novel feature selection method for support vector machines. Pattern Recognition, 2006, 39: 1333~1345.

[5] Kohavi R, John G H. Wrappers for feature subset selection. Artif. Intell. ,1997, 97: 273~324.

[6] Huang Ch L, Wang Ch J.A GA-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 2006, 31: 231~240.

[7] Mao K Z.Feature subset selection for support vector machines through discriminative function pruning analysis. IEEE Transactions on Systems, Man, and Cybernetics, 2004, 34(1): 60~67.

[8] Yu E, Cho S. Ensemble based on GA wrapper feature selection. Computers & Industrial Engineering, 2006, 51(1): 111~116.

[9] Bi J, Bennett K P, Embrechts M, et al. Dimensionality reduction via sparse support vector machines. J. Mach. Learn. Res. 2003, 3: 1229~1243.

[10] Sikora R, Piramuthu S. Framework for efficient feature selection in genetic algorithm based data mining. European Journal of Operational Research, 2007, 180(2): 723~737.

[11] Cover T M. The best two independent measurements are not the two best. IEEE Transactions on Systems, Man, and Cybernetics, 1974, 1: 116~117.

[12] Elashoff J D, Elashoff R M, Goldman G E.On the choice of variables in classification problems with dichotomous variables. Biometrika, 1967, 54: 668~670.

[13] Toussaint G T.Note on optimal selection of independent binary-valued features for pattern recognition. IEEE Transactions on Information Theory IT, 1971,17: 618.

[14] Meiri R, Zahavi J. Using simulated annealing to optimize the feature selection problem in marketing applications. European Journal of Operational Research, 2006, 171: 842~858.

[15] Liu H, Motoda H. Feature Extraction, Construction, and Selection: A Data Mining Perspective.Dordrecht: Kluwer Academic Publishers,1998.

[16] Marill T, Green D M. On the effectiveness of receptors in recognition systems. IEEE Trans. Inf. Theory, 1963, 9: 11~17.

[17] Raymer M L, Punch W F, Goodman E D, et al. Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. , 2000, 4(2): 164~171.

[18] 杨青生, 黎夏. 基层遗传算法自动获取CA模型的参数——以东莞市城市发展模拟为例.地理研究,2007,26(2): 226~231.

[19] Corts C, Vapnik V. Support vector networks. Machine Learning, 1995, 20: 273~ 297.

[20] Vapnik V N.The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.

[21] Cortes C, Vapnik V N. Support vector networks. Mach. Learn. , 1995, 20(3): 273~297.

[22] 邓乃扬, 田英杰. 数据挖掘中的新方法——支持向量机. 北京: 科学出版社, 2004.

[23] Pontil M, Verri A.Support vector machines for 3D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(6), 637~646.

[24] Bradley P S, Mangasarian O L, Street W N.Feature selection via mathematical programming. INFORMS Journal on Computing, 1998, 10: 209~217.

[25] Guyon I, Weston J, Barnhill S, et al.Gene selection for cancer classification using support vector machines. Machine Learning, 2002, 46(1~3): 389~422.

[26] Yu E, Cho S.GA-SVM wrapper approach for feature selection in keystroke dynamics identity verification. Proceedings of 2003 INNS-IEEE International Joint Conference on Neural Networks, 2253~2257.

[27] Weston J, Mukherjee S, Chapelle O, et al. Feature selection for SVMs. Advances in Neural Information Processing Systems, 13.Cambridge, MA: MIT Press, 2001.

[28] Chapelle O, Vapnik V, Bousquet O, et al. Choosing multiple parameters for support vector machines. Machine Learning, 2002, 46 (123): 131~159.

[29] Lee J H, L in C J.Automatic Model Selection for Support Vector Machines. Taipei: Taiwan University, 2000.

[30] Bengio Y.Gradient-based optimization of hyperparameters. Neural Computation, 2000, 12(8): 1889~1900.

[31] Keerthi S S.Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transon Neural Networks, 2002, 13(5):1225~1229.

[32] Hsu C W, Chang C C, Lin C J.A practical guide to support vector classification, 2003. Available at:http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.

[33] Jarecke P. EO-1/Hyperion Science Data User's Guide. TRW Space, Defense & Information System, 2001.

基于GA-SVM封装算法的高光谱数据特征选择

A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector maching

PDF (PC)

赞

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

Metrics

本文评价

推荐阅读 0