Machine-learning in optimization of expensive black-box functions

Yoel Tenne

International Journal of Applied Mathematics and Computer Science, Tome 27 (2017), p. 105-118 / Harvested from The Polish Digital Mathematics Library

Résumé

Modern engineering design optimization often uses computer simulations to evaluate candidate designs. For some of these designs the simulation can fail for an unknown reason, which in turn may hamper the optimization process. To handle such scenarios more effectively, this study proposes the integration of classifiers, borrowed from the domain of machine learning, into the optimization process. Several implementations of the proposed approach are described. An extensive set of numerical experiments shows that the proposed approach improves search effectiveness.

Détail

Publié le : 2017-01-01
EUDML-ID : urn:eudml:doc:288102

Bibliographie

[000] Arlot, S. (2010). A survey of cross-validation procedures for model selection, Statistics Survey 4: 40-79. | Zbl 1190.62080

[001] Benoudjit, N., Archambeau, C., Lendasse, A., Lee, J. and Verleysen, M. (2002). Width optimization of the Gaussian Kernels in radial basis function networks, Proceedings of the 10th European Symposium on Artificial Neural Networks, ESANN 2002, Bruges, Belgium, pp. 425-432.

[002] Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V. and Trosset, M.W. (1999). A rigorous framework for optimization of expensive functions by surrogates, Structural Optimization 17(1): 1-13.

[003] Büche, D., Schraudolph, N.N. and Koumoutsakos, P. (2005). Accelerating evolutionary algorithms with Gaussian process fitness function models, IEEE Transactions on Systems, Man, and Cybernetics C 35(2): 183-194.

[004] Chipperfield, A., Fleming, P., Pohlheim, H. and Fonseca, C. (1994). Genetic Algorithm TOOLBOX for Use with MATLAB, Version 1.2, Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield.

[005] Conn, A.R., Gould, N.I.M. and Toint, P.L. (2000). Trust Region Methods, SIAM, Philadelphia, PA. | Zbl 0958.65071

[006] Conn, A.R., Scheinberg, K. and Toint, P.L. (1998). A derivative free optimization algorithm in practice, Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, USA, Paper No. AIAA-1998-4718. | Zbl 1042.90617

[007] de Jong, K.A. (2006). Evolutionary Computation: A Unified Approach, MIT Press, Cambridge, MA. | Zbl 1106.68093

[008] Drela, M. and Youngren, H. (2001). Xfoil 6.9 user primer, Technical report, MIT, Cambridge, MA.

[009] Emmerich, M.T.M., Giotis, A., Özedmir, M., Bäck, T. and Giannakoglou, K.C. (2002). Metamodel-assisted evolution strategies, in J.J. Merelo Guervós (Ed.), 7th International Conference on Parallel Problem Solving from Nature- PPSN VII, Lecture Notes in Computer Science, Vol. 2439, Springer, Berlin, pp. 361-370.

[010] Forrester, A.I.J. and Keane, A.J. (2008). Recent advances in surrogate-based optimization, Progress in Aerospace Science 45(1-3): 50-79.

[011] Golberg, M.A., Chen, C.S. and Karur, S.R. (1996). Improved multiquadric approximation for partial differential equations, Engineering Analysis with Boundary Elements 18(1): 9-17.

[012] Gorissen, D., De Tommasi, L., Croon, J. and Dhaene, T. (2008). Automatic model type selection with heterogeneous evolution: An application to RF circuit block modeling, Proceedings of the IEEE Congress on Evolutionary Computation (CEC), Hong Kong, China, pp. 989-996.

[013] Handoko, S., Kwoh, C.K. and Ong, Y.-S. (2010). Feasibility structure modeling: An effective chaperon for constrained memetic algorithms, IEEE Transactions on Evolutionary Computation 14(5): 740-758.

[014] Hansen, N. and Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies, Evolutionary Computation 9(2): 159-195.

[015] Hicks, R.M. and Henne, P.A. (1978). Wing design by numerical optimization, Journal of Aircraft 15(7): 407-412.

[016] Jin, Y., Olhofer, M. and Sendhoff, B. (2002). A framework for evolutionary optimization with approximate fitness functions, IEEE Transactions on Evolutionary Computation 6(5): 481-494.

[017] Jones, D.R., Schonlau, M. and Welch, W.J. (1998). Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13(4): 455-492. | Zbl 0917.90270

[018] McKay, M.D., Beckman, R.J. and Conover, W.J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21(2): 239-245. | Zbl 0415.62011

[019] Molinaro, A.M., Simon, R. and Pfeiffer, R.M. (2005). Prediction error estimation: A comparison of resampling methods, Biometrika 21(15): 3301-3307.

[020] Muller, J. and Shoemaker, C.A. (2014). Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems, Journal of Global Optimization 60(2): 123-144. | Zbl 1312.90064

[021] Okabe, T. (2007). Stabilizing parallel computation for evolutionary algorithms on real-world applications, Proceedings of the 7th International Conference on Optimization Techniques and Applications (ICOTA 7), Kobe, Japan, pp. 131-132.

[022] Poloni, C., Giurgevich, A., Onseti, L. and Pediroda, V. (2000). Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for a complex design problem in fluid dynamics, Computer Methods in Applied Mechanics and Engineering 186(2-4): 403-420. | Zbl 0956.76023

[023] Powell, M.J.D. (2001). Radial basis function methods for interpolation of functions of many variables, Proceedings of the 5th Hellenic-European Conference on Computer Mathematics and Its Applications (HERCMA-01), Athens, Greece, pp. 2-24. | Zbl 1048.65502

[024] Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidyanathan, R. and Tucker, K.P. (2005). Surrogate-based analysis and optimization, Progress in Aerospace Science 41(1): 1-28.

[025] Rasheed, K., Hirsh, H. and Gelsey, A. (1997). A genetic algorithm for continuous design space search, Artificial Intelligence in Engineering 11(3): 295-305.

[026] Ratle, A. (1999). Optimal sampling strategies for learning a fitness model, 1999 IEEE Congress on Evolutionary Computation-CEC 1999, Washington, DC, USA, pp. 2078-2085.

[027] Regis, R.G. (2014). Particle swarm with radial basis function surrogates for expensive black-box optimization, Journal of Computational Science 5(1): 12-23.

[028] Regis, R.G. and Shoemaker, C.A. (2013). A quasi-multistart framework for global optimization of expensive functions using response surface models, International Journal of Global Optimization 56(4): 1719-1753. | Zbl 1275.90068

[029] Sacks, J., Welch, W.J., Mitchell, T.J. and Wynn, H.P. (1989). Design and analysis of computer experiments, Statistical Science 4(4): 409-435. | Zbl 0955.62619

[030] Sheskin, D.J. (2007). Handbook of Parametric and Nonparametric Statistical Procedures, 4th Edn., Chapman and Hall, Boca Raton, FL. | Zbl 1118.62001

[031] Smoczek, J. (2013). Evolutionary optimization of interval mathematics-based design of a TSK fuzzy controller for anti-sway crane control, International Journal of Applied Mathematics and Computer Science 23(4): 749-759, DOI: 10.2478/amcs-2013-0056. | Zbl 1284.93143

[032] Smołka, M., Schaefer, R., Paszyński, M., Pardo, D. and Álvarez Aramberri, J. (2015). An agent-oriented hierarchic strategy for solving inverse problems, International Journal of Applied Mathematics and Computer Science 25(3): 483-498, DOI: 10.1515/amcs-2015-0036.

[033] Sobieszczanski-Sobieski, J. and Haftka, R.T. (1997). Multidisciplinary aerospace design optimization: Survey of recent developments, Structural Optimization 14(1): 1-23.

[034] Tenne, Y. (2013). An optimization algorithm employing multiple metamodels and optimizers, International Journal of Automation and Computing 10(3): 227-241.

[035] Tenne, Y. (2015). An adaptive-topology ensemble algorithm for engineering optimization problems, Optimization and Engineering 16(2): 303-334.

[036] Tenne, Y. and Armfield, S.W. (2008). A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications, in A. Yang et al. (Eds.), Success in Evolutionary Computation, Studies in Computational Intelligence, Vol. 92, Springer-Verlag, Berlin/Heidelberg, pp. 43-72.

[037] Tenne, Y. and Goh, C.K. (Eds.) (2010). Computational Intelligence in Expensive Optimization Problems, Springer, Berlin. | Zbl 1187.90020

[038] Tenne, Y., Izui, K. and Nishiwaki, S. (2010). Handling undefined vectors in expensive optimization problems, in C. Di Chio (Ed.), Proceedings of the 2010 EvoStar Conference, Lecture Notes in Computer Science, Vol. 6024, Springer, Berlin, pp. 582-591.

[039] Tenne, Y., Izui, K. and Nishiwaki, S. (2011). A classifier-assisted framework for expensive optimization problems: A knowledge-mining approach, in C.A. Coello-Coello (Ed.), Proceedings of the 5th Learning and Intelligent Optimization Conference (LION 5), Lecture Notes in Computer Science, Vol. 6683, Springer, Berlin/Heidelberg, pp. 161-175.

[040] Viana, F.A.C., Haftka, R.T. and Watson, L.T. (2013). Efficient global optimization algorithm assisted by multiple surrogate technique, Journal of Global Optimization 56(2): 669-689. | Zbl 1275.90072

[041] Wortmann, T., Costa, A., Nannicini, G. and Schroepfer, T. (2015). Advantages of surrogate models for architectural design optimization, Artificial Intelligence for Engineering Design, Analysis and Manufacturing 29(4): 471-481.

[042] Wu, H.-Y., Yang, S., Liu, F. and Tsai, H.-M. (2003). Comparison of three geometric representations of airfoils for aerodynamic optimization, Proceedings of the 16th AIAA Computational Fluid Dynamics Conference, Orlando, FL, USA, pp. 1-11, Paper no. AIAA 2003-4095.

[043] Wu, X., Kumar, V., Quinlan, R.J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.-H., Steinbach, M., Hand, D.J. and Steinberg, D. (2008). Top 10 algorithms in data mining, Knowledge and Information Systems 14(1): 1-37.