Cross-task code reuse in genetic programming applied to visual learning

Wojciech Jaśkowski ; Krzysztof Krawiec ; Bartosz Wieloch

International Journal of Applied Mathematics and Computer Science, Tome 24 (2014), p. 183-197 / Harvested from The Polish Digital Mathematics Library

Résumé

We propose a method that enables effective code reuse between evolutionary runs that solve a set of related visual learning tasks. We start with introducing a visual learning approach that uses genetic programming individuals to recognize objects. The process of recognition is generative, i.e., requires the learner to restore the shape of the processed object. This method is extended with a code reuse mechanism by introducing a crossbreeding operator that allows importing the genetic material from other evolutionary runs. In the experimental part, we compare the performance of the extended approach to the basic method on a real-world task of handwritten character recognition, and conclude that code reuse leads to better results in terms of fitness and recognition accuracy. Detailed analysis of the crossbred genetic material shows also that code reuse is most profitable when the recognized objects exhibit visual similarity.

Détail

Publié le : 2014-01-01

Zbl 1292.90333

EUDML-ID : urn:eudml:doc:271915

Bibliographie

[000] Bhanu, B., Lin, Y. and Krawiec, K. (2005). Evolutionary Synthesis of Pattern Recognition Systems, Springer-Verlag, New York, NY. | Zbl 1067.68135

[001] Caruana, R. (1997). Multitask learning, Machine Learning 28(1): 41-75.

[002] Chang, Y.F., Lee, J.C., Mohd Rijal, O. and Syed Abu Bakar, S.A.R. (2010). Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship model, International Journal of Applied Mathematics and Computer Science 20(4): 727-738, doi: 10.2478/v10006-010-0055-x. | Zbl 05869747

[003] Ciresan, D.C., Meier, U. and Schmidhuber, J. (2012). Multi-column deep neural networks for image classification, Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, pp. 3642-3649.

[004] Fabijańska, A. (2012). A survey of subpixel edge detection methods for images of heat-emitting metal specimens, International Journal of Applied Mathematics and Computer Science 22(3): 695-710, DOI: 10.2478/v10006-012-0052-3. | Zbl 1303.94006

[005] Galvan Lopez, E., Poli, R. and Coello Coello, C.A. (2004). Reusing code in genetic programming, in M. Keijzer, U.-M. O'Reilly, S.M. Lucas, E. Costa and T. Soule (Eds.), Genetic Programming-7th European Conference, EuroGP 2004, Proceedings, Lecture Notes in Computer Science, Vol. 3003, Springer-Verlag, Berlin/Heidelberg, pp. 359-368.

[006] Ghosn, J. and Bengio, Y. (2003). Bias learning, knowledge sharing, IEEE Transactions on Neural Networks 14(4): 748-765.

[007] Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I.H. (2009). The Weka data mining software: An update, SIGKDD Explorations 11(1): 10-18.

[008] Haynes, T. (1997). On-line adaptation of search via knowledge reuse, in J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba and R.L. Riolo (Eds.), Genetic Programming 1997: Proceedings of the Second Annual Conference, Morgan Kaufmann, San Francisco, CA, pp. 156-161.

[009] Holland, J. (1975). Adaptation in Natural and Artificial Systems, Vol. 1, University of Michigan Press, Ann Arbor, MI.

[010] Hornby, G.S. and Pollack, J.B. (2002). Creating high-level components with a generative representation for body-brain evolution, Artificial Life 8(3): 223-246.

[011] Howard, D. (2003). Modularization by multi-run frequency driven subtree encapsulation, in R.L. Riolo and B. Worzel (Eds.), Genetic Programming Theory and Practice, Kluwer, New York, NY, Chapter 10, pp. 155-172.

[012] Howard, D., Roberts, S.C. and Ryan, C. (2006). Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance, Pattern Recognition Letters 27(11): 1275-1288.

[013] Hsu, W.H., Harmon, S.J., Rodriguez, E. and Zhong, C. (2004). Empirical comparison of incremental reuse strategies in genetic programming for keep-away soccer, in M. Keijzer (Ed.), Late Breaking Papers at the 2004 Genetic and Evolutionary Computation Conference, Association for Computing Machinery, Seattle, WA.

[014] Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007a). Genetic programming for cross-task knowledge sharing, in D. Thierens (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, London, pp. 1620-1627.

[015] Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007b). Knowledge reuse in genetic programming applied to visual learning, in D. Thierens (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, London, pp. 1790-1797. | Zbl 1292.90333

[016] Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007c). Learning and recognition of hand-drawn shapes using generative genetic programming, in M. Giacobini (Ed.), EvoWorkshops 2007, Lecture Notes in Computer Science, Vol. 4448, Springer-Verlag, Berlin/Heidelberg, pp. 281-290.

[017] Koza, J. (1992). Genetic Programming, MIT Press, Cambridge, MA. | Zbl 0850.68161

[018] Koza, J.R. (1994). Genetic Programming II: Automatic Discovery of Reusable Programs, MIT Press, Cambridge, MA. | Zbl 0850.68160

[019] Koza, J.R., Bennett III, F.H., Andre, D. and Keane, M.A. (1996). Reuse, parameterized reuse, and hierarchical reuse of substructures in evolving electrical circuits using genetic programming, in T. Higuchi (Ed.), Proceedings of International Conference on Evolvable Systems: From Biology to Hardware (ICES-96), Lecture Notes in Computer Science, Vol. 1259, Springer-Verlag, Berlin.

[020] Krawiec, K. (2006). Learning high-level visual concepts using attributed primitives and genetic programming, in F. Rothlauf (Ed.), EvoWorkshops 2006, Lecture Notes in Computer Science, Vol. 3907, Springer-Verlag, Berlin/Heidelberg, pp. 515-519.

[021] Krawiec, K. (2007). Generative learning of visual concepts using multiobjective genetic programming, Pattern Recognition Letters 28(16): 2385-2400.

[022] Krawiec, K. and Bhanu, B. (2005). Visual learning by coevolutionary feature synthesis, IEEE Transactions on System, Man, and Cybernetics, Part B 35(3): 409-425.

[023] Kurashige, K., Fukuda, T. and Hoshino, H. (2003). Reusing primitive and acquired motion knowledge for gait generation of a six-legged robot using genetic programming, Journal of Intelligent and Robotic Systems 38(1): 121-134.

[024] Langdon, W.B. and Poli, R. (2002). Foundations of Genetic Programming, Springer-Verlag, New York, NY. | Zbl 0993.68039

[025] Li, B., Li, X., Mabu, S. and Hirasawa, K. (2012). Towards automatic discovery and reuse of subroutines in variable size genetic network programming, in X. Li (Ed.), Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, Australia, pp. 485-492.

[026] Louis, S. and McDonnell, J. (2004). Learning with case-injected genetic algorithms, IEEE Transactions on Evolutionary Computation 8(4): 316-328.

[027] Luke, S. (2002). ECJ evolutionary computation system, http://cs.gmu.edu/eclab/projects/ecj/.

[028] Mitchell, T.M. (2006). The discipline of machine learning, Technical Report CMU-ML-06-108, Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA.

[029] Montana, D.J. (1993). Strongly typed genetic programming, BBN Technical Report #7866, Bolt Beranek and Newman, Inc., Cambridge, MA.

[030] Moya, M.R., Koch, M.W. and Hostetler, L.D. (1993). One-class classifier networks for target recognition applications, World Congress on Neural Networks, Portland, OR, USA, pp. 797-801.

[031] O'Sullivan, J. and Thrun, S. (1995). A Robot That Improves Its Ability to Learn, Carnegie Mellon University, School of Computer Science, Pittsburgh, PA.

[032] Poli, R., Langdon, W.B. and McPhee I.N.F. (2008). A field guide to genetic programming, http://www.gp-field-guide.org.uk/.

[033] Perez, C.B. and Olague, G. (2013). Genetic programming as strategy for learning image descriptor operators, Intelligent Data Analysis 17(4): 561-583.

[034] Pratt, L.Y., Mostow, J. and Kamm, C.A. (1991). Direct transfer of learned information among neural networks, Proceedings of the 9th National Conference on Artificial Intelligence (AAAI-91), Anaheim, CA, USA, pp. 584-589.

[035] Rizki, M.M., Zmuda, M.A. and Tamburino, L.A. (2002). Evolving pattern recognition systems, IEEE Transactions on Evolutionary Computation 6(6): 594-609.

[036] Roberts, S.C., Howard, D. and Koza, J.R. (2001). Evolving modules in genetic programming by subtree encapsulation, in J.F. Miller (Ed.), Genetic Programming, Proceedings of EuroGP'2001, Lecture Notes in Computer Science, Vol. 2038, Springer-Verlag, Berlin, pp. 160-175. | Zbl 0989.68609

[037] Rosca, J.P. and Ballard, D.H. (1996). Discovery of subroutines in genetic programming, in P.J. Angeline and K.E. Kinnear, Jr. (Eds.), Advances in Genetic Programming 2, MIT Press, Cambridge, MA, Chapter 9, pp. 177-202.

[038] Seront, G. (1995). External concepts reuse in genetic programming, in E.V. Siegel and J.R. Koza (Eds.), Working Notes for the AAAI Symposium on Genetic Programming, AAAI/MIT, Cambridge, MA, pp. 94-98.

[039] Tackett, W.A. (1993). Genetic generation of 'dendritic' trees for image classification, Proceedings of the World Congress on Neural Networks, Portland, OR, USA, pp. IV 646-649.

[040] Teller, A. and Veloso, M. (1997). PADO: A new learning architecture for object recognition, in K. Ikeuchi and M. Veloso (Eds.), Symbolic Visual Learning, Oxford Press, New York, NY, pp. 77-112.

[041] Trujillo, L. and Olague, G. (2006). Synthesis of interest point detectors through genetic programming, in M. Cattolico (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, Seattle, WA, pp. 887-894.

[042] Vilalta, R. and Drissi, Y. (2002). A perspective view and survey of meta-learning, Artificial Intelligence Review 18(2): 77-95.

[043] Whitley, D., Rana, S. and Heckendorn, R. (1999). The island model genetic algorithm: On separability, population size and convergence, Journal of Computing and Information Technology 7(1): 33-47.