Dimension reduction is an important topic in data mining and machine learning. Especially dimension reduction combined with feature fusion is an effective preprocessing step when the data are described by multiple feature sets. Canonical Correlation Analysis (CCA) and Discriminative Canonical Correlation Analysis (DCCA) are feature fusion methods based on correlation. However, they are different in that DCCA is a supervised method utilizing class label information, while CCA is an unsupervised method. It has been shown that the classification performance of DCCA is superior to that of CCA due to the discriminative power using class label information. On the other hand, Linear Discriminant Analysis (LDA) is a supervised dimension reduction method and it is known as a special case of CCA. In this paper, we analyze the relationship between DCCA and LDA, showing that the projective directions by DCCA are equal to the ones obtained from LDA with respect to an orthogonal transformation. Using the relation with LDA, we propose a new method that can enhance the performance of DCCA. The experimental results show that the proposed method exhibits better classification performance than the original DCCA.
@article{bwmeta1.element.bwnjournal-article-amcv21i3p549bwm,
author = {Yong Joon Shin and Cheong Hee Park},
title = {Analysis of correlation based dimension reduction methods},
journal = {International Journal of Applied Mathematics and Computer Science},
volume = {21},
year = {2011},
pages = {549-558},
zbl = {1230.68173},
language = {en},
url = {http://dml.mathdoc.fr/item/bwmeta1.element.bwnjournal-article-amcv21i3p549bwm}
}
Yong Joon Shin; Cheong Hee Park. Analysis of correlation based dimension reduction methods. International Journal of Applied Mathematics and Computer Science, Tome 21 (2011) pp. 549-558. http://gdmltest.u-ga.fr/item/bwmeta1.element.bwnjournal-article-amcv21i3p549bwm/
[000] Anton, H. and Busby, R. (2003). Contemporary Linear Algebra, John Wiley and Sons, Denver, CO. | Zbl 1008.15001
[001] Baudat, G. and Anouar, F. (2000). Generalized discriminant analysis using a kernel approach, Neural Computation 12(10): 2385-2404.
[002] Billings, S. and Lee, K. (2002). Nonlinear fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm, Neural Networks 15(2): 263-270.
[003] Chen, L., Liao, H., M.Ko, Lin, J. and Yu, G. (2000). A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recognition 33(10): 1713-1726.
[004] Duda, R., Hart, P. and Stork, D. (2001). Pattern Classification, Wiley Interscience, New York, NY. | Zbl 0968.68140
[005] Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition, 2nd Edn., Academic Press, San Diego, CA. | Zbl 0711.62052
[006] Fukunaga, K. and Mantock, J. (1983). Nonparametric discriminant analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 5(6): 671-678. | Zbl 0542.62047
[007] Garthwaite, P. (1994). An interpretation of partial least squares, Journal of the American Statistical Society 89(425): 122-127. | Zbl 0793.62034
[008] He, X. and Niyogi, P. (2003). Locality preserving projections, Proceedings of the Advances in Neural Information Processing Systems Conference, Vancouver, Canada, pp. 153-160.
[009] Hotelling, H. (1936). Relations between two sets of variates, Biometrika 28(3): 321-377. | Zbl 0015.40705
[010] Hou, C., Nie, F., Zhang, C. and Wu, Y. (2009). Learning an orthogonal and smooth subspace for image classification, IEEE Signal Processing Letters 16(4): 303-306.
[011] Howland, P. and Park, H. (2004). Generalizing discriminant analysis using the generalized singular value decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8): 995-1006.
[012] Jolliffe, I. (1986). Principal Component Analysis, Springer, New York, NY. | Zbl 1011.62064
[013] Nie, F., Xiang, S. and Zhang, C. (2007). Neighborhood minmax projections, Proceedings of the International Joint Conference on Artificial Intelligence, Hyderabad, India, pp. 993-998.
[014] Pardalos, P. and Hansen, P. (2008). Data Mining and Mathematical Programming, CRM Proceedings & Lecture Notes, Vol. 45, American Mathematical Society, Montreal.
[015] Park, C. and Park, H. (2008). A comparison of generalized linear discriminant analysis algorithms, Pattern Recognition 41(3): 1083-1097. | Zbl 1132.68656
[016] Roweis, S.T. and Saul, L.K. (2000). Nonlinear dimensionality reduction by locally linear embedding, Science 290(5500): 2323-2326.
[017] Sugiyama, M. (2006). Local fisher discriminant analysis for supervised dimensionality reduction, Proceedings of the IEEE International Conference on Machine Learning, Pittsburgh, PA, USA, pp. 905-912.
[018] Sun, Q., Zeng, S., Liu, Y., Heng, P. and Xia, D. (2005). A new method of feature fusion and its application in image recognition, Pattern Recognition 38(12): 2437-2448.
[019] Sun, T. and Chen, S. (2007). Class label versus sample label-based CCA, Applied Mathematics and Computation 185(1): 272-283. | Zbl 1109.62053
[020] Sun, T., Chen, S., Yang, J. and Shi, P. (2008). A supervised combined feature extraction method for recognition, Procedings of the IEEE International Conference on Data Mining, Pisa, Italy, pp. 1043-1048.
[021] Tenenbaum, J.B., de Silva, V. and Langford, J.C. (2000). A global geometric framework for nonlinear dimensionality reduction, Science 290(5500): 2319-2323.
[022] Wegelin, J. (2000). A survey of partial least squares (PLS) methods, with emphasis on the two block case, Technical report, Department of Statistics, University of Washington, Seattle, WA.
[023] Yan, S., Xu, D., Zhang, B., Zhang, H.-J., Yang, Q. and Lin, S. (2007). Graph embedding and extensions: A general framework for dimensionlity reduction, IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1): 40-51.
[024] Yang, J. and Yang, J.-Y. (2003). Why can LDA be performed in PCA transformed space?, Pattern Recognition 36(2): 563-566.
[025] Yang, J., Yang, J., Zhang, D. and Lu, J. (2003). Feature fusion: Parallel strategy vs. serial strategy, Pattern Recognition 36(6): 1369-1381. | Zbl 1028.68152
[026] Ye, J. (2005). Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems, Journal of Machine Learning Research 6(4): 483-502. | Zbl 1222.62081
[027] Yu, H. and Yang, J. (2001). A direct LDA algorithm for highdimensional data-with application to face recognition, Pattern Recognition 34(10): 2067-2070. | Zbl 0993.68091