Improving prediction models applied in systems monitoring natural hazards and machinery

Marek Sikora ; Beata Sikora

International Journal of Applied Mathematics and Computer Science, Tome 22 (2012), p. 477-491 / Harvested from The Polish Digital Mathematics Library

Résumé

A method of combining three analytic techniques including regression rule induction, the k-nearest neighbors method and time series forecasting by means of the ARIMA methodology is presented. A decrease in the forecasting error while solving problems that concern natural hazards and machinery monitoring in coal mines was the main objective of the combined application of these techniques. The M5 algorithm was applied as a basic method of developing prediction models. In spite of an intensive development of regression rule induction algorithms and fuzzy-neural systems, the M5 algorithm is still characterized by the generalization ability and unbeatable time of data model creation competitive with other systems. In the paper, two solutions designed to decrease the mean square error of the obtained rules are presented. One consists in introducing into a set of conditional variables the so-called meta-variable (an analogy to constructive induction) whose values are determined by an autoregressive or the ARIMA model. The other shows that limitation of a data set on which the M5 algorithm operates by the k-nearest neighbor method can also lead to error decreasing. Moreover, three application examples of the presented solutions for data collected by systems of natural hazards and machinery monitoring in coal mines are described. In Appendix, results of several benchmark data sets analyses are given as a supplement of the presented results.

Détail

Publié le : 2012-01-01

Zbl 1283.93043

EUDML-ID : urn:eudml:doc:208123

Bibliographie

[000] Bloedorn, E. and Michalski, R. (2002). Data-driven constructive induction, IEEE Intelligent Systems 13(2): 30-37.

[001] Boser, B., Guyon, I. and Vapnik, V. (1992). A training algorithm for optimal margin classifiers, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, Pittsburgh, PA, USA, pp. 144-152.

[002] Box, G. and Jenkins, G. (1994). Time Series Analysis: Forecasting and Control, Prentice-Hall, Upper Saddle River, NJ. | Zbl 0858.62072

[003] Breiman, L., Friedman, J.H., Olshen, R.A. and Stone, C.J. (1994). Classification and Regression Trees, Wadsworth, Belmont, CA. | Zbl 0541.62042

[004] Brockwell, P. and Davis, R. (2002). Introduction to Time Series Forecasting, Springer-Verlag, New York, NY. | Zbl 0994.62085

[005] Broyden, C. (1969). A new double-rank minimization algorithm, Notices of the American Mathematical Society 16: 670.

[006] Cao, L. and Tay, F. (2003). Support vector machine with adaptive parameters in financial time series forecasting, IEEE Transactions on Neural Networks 14(6): 1506-1518.

[007] Chen, X., Yang, J. and Liang, J. (2011). A flexible support vector machine for regression, Neural Computing & Applications, DOI 10.1007/s00521-011-0623-5.

[008] Chunshien, L. and Kuo-Hsiang, C. (2007). Recurrent neurofuzzy hybrid-learning approach to accurate systems modeling, Fuzzy Sets and Systems 158(2): 194-212. | Zbl 1110.93030

[009] Czogała, E. and Łęski, J. (2000). Fuzzy and Neuro-Fuzzy Intelligent Systems. Studies in Fuzziness and Soft Computing, Springer-Verlag, New York, NY. | Zbl 0953.68122

[010] Dembczyński, K., Kotowiski, W. and Słowiński, R. (2010). Ender: A statistical framework for boosting decision rules, Data Mining and Knowledge Discovery 21(1): 52-90.

[011] Dixon, W. (1992). A Statistical Analysis of Monitored Data for Methane Prediction, Ph.D. thesis, University of Nottingham, Nottingham.

[012] Duch, W., Adamczak, R. and Grabczewski, K. (2000). A new methodology of extraction, optimization and application of crisp and fuzzy logical rules, IEEE Transactions on Neural Networks 11(10): 1-31.

[013] Friedman, J., Kohavi, R. and Yun, Y. (1996). Lazy decision trees, Proceedings of AAAI/IAAI, Portland, OR, USA, pp. 717-724.

[014] Gale, W., Heasley, K., Iannacchione, A., Swanson, P., Hatherly, P. and King, A. (2001). Rock damage characterization from microseismic monitoring, Proceedings of the 38th US Symposium of Rock Mechanics, Lisse, The Netherlands, pp. 1313-1320.

[015] Goldberg, D. (1989). Genetics Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Publishing Company, Boston, MA. | Zbl 0721.68056

[016] Góra, G. and Wojna, A. (2002). Riona: A new classification system combining rule induction and instance-based learning, Fundamenta Informaticae 51(4): 369-390. | Zbl 1011.68114

[017] Grychowski, T. (2008). Hazard assessment based on fuzzy logic, Archives of Mining Sciences 53(4): 595-602.

[018] Hao, P. (2010). New support vector algorithms with parametric insensitive/margin model, Neural Networks 23(1): 60-73.

[019] Jang, J.-S. (1994). Structure determination in fuzzy modelling: A fuzzy cart approach, Proceedings of the IEEE International Conference on Fuzzy Systems, Orlando, FL, USA, pp. 480-485.

[020] Janssen, F. and Fürnkranz, J. (2010a). On the quest for optimal rule learning heuristics, Machine Learning 78(3): 343-379.

[021] Janssen, F. and Fürnkranz, J. (2010b). Separate-and-conquer regression, Proceedings of LWA 2010: Lernen, Wissen, Adaptivität, Kassel, Germany, pp. 81-89.

[022] Jonak, J. (2002). Hazard assessment based on fuzzy logic, Journal of Mining Sciences 38(3): 270-277.

[023] Kabiesz, J. (2005). Effect of the form of data on the quality of mine tremors hazard forecasting using neural networks, Geotechnical and Geological Engineering 24(5): 1131-1147.

[024] Katayama, N. and Satoh, S. (1997). The SR-tree: An index structure for high dimensional nearest neighbor queries, Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, pp. 369-380.

[025] Macleod, J., Luk, A. and Titterington, D. (1987). A reexamination of the distance-weighted k-nearest-neighbor classification rule, IEEE Transactions on Systems, Man and Cybernetics 17(4): 689-696.

[026] Malerba, D., Esposito, F., Ceci, M. and Appice, A. (2005). Topdown induction of model trees with regression and splitting nodes, IEEE Transactions on Pattern Analysis and Machine Intelligence 26(5): 612-625.

[027] Michalak, M. (2011). Adaptive kernel approach to the time series prediction, Pattern Analysis and Applications 14(3): 283-293.

[028] Nelles, O., Fink, A., Babuška, R. and Setnes, M. (2000). Comparison of two construction algorithms for Takagi-Sugeno fuzzy models, International Journal of Applied Mathematics and Computer Science 10(4): 835-855. | Zbl 0972.68168

[029] Oh, S. and Pedrycz, W. (2000). Identification of fuzzy systems by means of an auto-tuning algorithm and its application to nonlinear systems, Fuzzy Sets and Systems 115(2): 205-230. | Zbl 0965.93045

[030] Quinlan, J. (1992a). Learning with continuous classes, Proceedings of the International Conference on Artificial Intelligence, Singapore, pp. 343-348.

[031] Quinlan, J.R. (1992b). C4.5 Programs for Machine Learning, Morgan Kaufman Publishers, San Mateo, CA.

[032] Quinlan, J. (1993). Combining instance-based learning and model-based learning, Proceedings of the 10th International Conference on Machine Learning, San Mateo, CA, USA, pp. 236-243.

[033] Rutkowski, L. (2004). Generalized regression neural networks in time-varying environment, IEEE Transactions on Neural Networks 15(3): 576-596.

[034] Scholkopf, B., Smola, A., Williamson, R. and Bartlett, P. (2000). New support vector algorithms, Neural Computation 12(5): 1207-1245.

[035] Schuster, H. (1998). Deterministic Chaos, VCH Verlagsgesellschaft, New York, NY.

[036] Sikora, M. and Krzykawski, D. (2005). Application of data exploration methods in analysis of carbon dioxide emission in hard-coal mines dewater pump stations, Mechanizacja i Automatyzacja Górnictwa 413(6): 57-67, (in Polish).

[037] Sikora, M., Krzystanek, Z., Bojko, B. and Śpiechowicz, K. (2011). Application of a hybrid method of machine learning for description and on-line estimation of methane hazard in mine workings, Journal of Mining Sciences 47(4): 493-505.

[038] Sikora, M. and Sikora, B. (2006). Application of machine learning for prediction a methane concentration in a coal mine, Archives of Mining Sciences 51(4): 475-492.

[039] Sikora, M. and Wróbel, Ł. (2010). Application of rule induction algorithms for analysis of data collected by seismic hazard monitoring systems in coal mines, Archives of Mining Sciences 55(1): 91-114.

[040] Siwek, K., Osowski, S. and Szupiluk, R. (2009). Ensemble neural network approach for accurate load forecasting in a power system, International Journal of Applied Mathematics and Computer Science 19(2): 303-315, DOI: 10.2478/v10006-009-0026-2. | Zbl 1167.93338

[041] Tay, F. and Cao, L. (2002). Modified support vector machines in financial time series forecasting, Neurocomputing 48(1): 847-861. | Zbl 1006.68777

[042] Taylor, J. and Cristianini, N. (2004). Kernel Methods for Pattern Analysis, Cambridge University Press, Cambridge.

[043] Tong, H. (1990). Non-linear Time Series: A Dynamical Systems Approach, Oxford University Press, Oxford. | Zbl 0716.62085

[044] Torgo, L. (1997). Kernel regression trees, Proceedings of Poster Papers, European Conference on Machine Learning, Prague, Czech Republic, pp. 118-127.

[045] Vapnik, V. (1995). The Nature of Statistical Learning Theory, Springer, New York, NY. | Zbl 0833.62008

[046] Wang, Y. and Witten, I. (1997). Inducing model trees for continuous classes, Proceedings of Poster Papers, European Conference on Machine Learning, Prague, Czech Republic, pp. 128-137.

[047] Weigend, A., Huberman, B. and Rumelhart, D. (1990). Predicting the future: A connectionist approach, International Journal of Neural Systems 1(3): 193-209.

[048] Wess, S., Althoff, K. and Derwand, G. (1994). Using k-d trees to improve the retrieval step in case-based reasoning, in S. Wess, K.-D. Althoff and M. Richter (Eds.), Topics in Case-Based Reasoning, Springer-Verlag, Berlin, pp. 167-181.

[049] Wettschereck, D., Aha, D. and Mohri, T. (1997). A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms, Artificial Intelligence Review 11(1-5): 273-314.

[050] Wilson, D. and Martinez, T.R. (2000). An integrated instance-based learning algorithm, Computational Intelligence 16(1): 1-28.

[051] Witten, I. and Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, San Francisco, CA. | Zbl 1076.68555

[052] Wnek, J. and Michalski, R.S. (1994). Hypothesis-driven constructive induction in AQ17-HCI: A method and experiments, Machine Learning 14(2): 139-168. | Zbl 0804.68125

[053] Yager, R. and Filev, D. (1994). Essentials of Fuzzy Modeling and Control, John Wiley and Sons, New York, NY.