Extraction of fuzzy rules using deterministic annealing integrated with ε-insensitive learning

Czabański, Robert

International Journal of Applied Mathematics and Computer Science, Tome 16 (2006), p. 357-372 / Harvested from The Polish Digital Mathematics Library

Résumé

A new method of parameter estimation for an artificial neural network inference system based on a logical interpretation of fuzzy if-then rules (ANBLIR) is presented. The novelty of the learning algorithm consists in the application of a deterministic annealing method integrated with ε-insensitive learning. In order to decrease the computational burden of the learning procedure, a deterministic annealing method with a "freezing" phase and ε-insensitive learning by solving a system of linear inequalities are applied. This method yields an improved neuro-fuzzy modeling quality in the sense of an increase in the generalization ability and robustness to outliers. To show the advantages of the proposed algorithm, two examples of its application concerning benchmark problems of identification and prediction are considered.

Détail

Publié le : 2006-01-01

Zbl 1144.68338

EUDML-ID : urn:eudml:doc:207799

Bibliographie

[000] Bezdek J.C. (1982): Pattern Recognition with Fuzzy Objective FunctionAlgorithms. - New York: Plenum Press.

[001] Box G.E.P. and Jenkins G.M. (1976): Time Series Analysis. Forecasting and Control. -San Francisco: Holden-Day. | Zbl 0363.62069

[002] Czabański R. (2003): Automatic Fuzzy If-Then Rules Extraction from Numerical Data. -Ph.D. thesis, Silesian University of Technology, Gliwice, (in Polish).

[003] Czabański R. (2005): Neuro-fuzzy modeling based on deterministic annealing approach. - Int. J. Appl. Math. Comput. Sci., Vol. 15, No. 4, pp. 125-134. | Zbl 1107.68080

[004] Czogała E. and Łęski J. (1996): A new fuzzy inference system with moving consequents in if-then rules. Application to pattern recognition. -Bull. Polish Acad. Science, Vol. 45, No. 4, pp. 643-655. | Zbl 0954.68133

[005] Czogała E. and Łęski J. (1999): Fuzzy and Neuro-Fuzzy Intelligent Systems. -Heidelberg: Physica-Verlag. | Zbl 0953.68122

[006] Czogała E. and Łęski J. (2001): On equivalence of approximate reasoning results using different interpretations of if-then rules. - Fuzzy Sets Syst., Vol. 117, No. 2, pp. 279-296. | Zbl 0985.68067

[007] German S. and German D. (1984): Stochastic relaxation, Gibbs distribution and the Bayesian restoration in images. -IEEE Trans. Pattern Anal. Mach. Intell.,Vol. 6, pp. 721-741.

[008] Ho D. and Kashyap R.L. (1965): An algorithm for linear inequalities and its applications. -IEEE Trans. Elec. Comp., Vol. 14, No. 5, pp. 683-688. | Zbl 0173.17902

[009] Ho Y.C. and Kashyap R.L. (1966): A class of iterative procedures for linear inequalities. -SIAM J. Contr., Vol. 4, No. 2, pp. 112-115. | Zbl 0143.37503

[010] Jang J.S.R. (1993): ANFIS: Adaptive-network-based fuzzy inference system. - IEEE Trans. Syst. Man Cybern., Vol. 23, No. 3, pp. 665-685.

[011] Jang J.S.R. and Sun J.S.R. (1993): Functional equivalence between radial basis function networks and fuzzy inference systems. -IEEE Trans. Neural Netw., Vol. 4, No. 1, pp. 156-159.

[012] Jang J.S.R., Sun C.T. and Mizutani E. (1997): Neuro-Fuzzy and Soft Computing. A Computational Approach to Learning and Machine Intelligence. -Upper Saddle River: Prentice-Hall.

[013] Kirkpatrick S., Gelatt C. and Vecchi M. (1983): Optimization by simulated annealing. -Science, Vol. 220, No. 4598, pp. 671-680. | Zbl 1225.90162

[014] Łęski J. (2002): Improving generalization ability of neuro-fuzzy systems by ε-insensitive learning. -Int. J. Appl. Math. Comput. Sci., Vol. 12, No. 3, pp. 437-447. | Zbl 1062.68100

[015] Łęski J. (2003a): Neuro-fuzzy system with learning tolerant to imprecision. -Fuzzy Sets Syst., Vol. 138, No. 2, pp. 427-439.

[016] Łęski J. (2003b): ε-Insensitive learning techniques for approximate reasoning systems. -Int. J. Comput. Cognit., Vol. 1, No. 1, pp. 21-77.

[017] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H. and Teller E. (1953): Equation of state calculation by fast computing machines. -J. Chem. Phys., Vol. 21, No. 6, pp. 1087-1092.

[018] Mitra S. and Hayashi Y. (2000): Neuro-fuzzy rule generation: Survey in soft computing framework. -IEEE Trans. Neural Netw., Vol. 11, No. 3, pp. 748-768.

[019] Rao A.V., Miller D., Rose K. and Gersho A. (1997): Mixture of experts regression modeling by deterministic annealing. -IEEE Trans. Signal Process., Vol. 45, No. 11, pp. 2811-2820.

[020] Rao A.V., Miller D., Rose K. and Gersho A. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models. -IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159-173.

[021] Rose K. (1991): Deterministic Annealing, Clustering and Optimization. -Ph.D. thesis, California Inst. Tech., Pasadena.

[022] Rose K. (1998): Deterministic annealing for clustering, compression, classification, regression and related optimization problems. -Proc. IEEE, Vol. 86, No. 11, pp. 2210-2239.

[023] Vapnik V. (1998): Statistical Learning Theory. -New York: Wiley. | Zbl 0935.62007

[024] Vapnik V. (1999): An overview of statistical learning theory. -IEEE Trans. Neural Netw., Vol. 10, No. 5, pp. 988-999.

[025] Weigend A.S., Huberman B.A. and Rumelhart D.E. (1990): Predicting the future: A connectionist approach. -Int. J. Neural Syst., Vol. 1, No. 2, pp. 193-209.

[026] Xie X.L. and Beni G. (1991): A validity measure for fuzzy clustering. -IEEE Trans. Pattern Anal. Mach. Intell., Vol. 13, No. 8, pp. 841-847.

[027] Yen J., Wang L. and Gillespie C.W. (1998): Improving the interpretability of TSK fuzzy models by combining global learning and local learning. -IEEE Trans. Fuzzy Syst., Vol. 6, No. 4, pp. 530-537.