Neuro-fuzzy modelling based on a deterministic annealing approach

Czabański, Robert

International Journal of Applied Mathematics and Computer Science, Tome 15 (2005), p. 561-576 / Harvested from The Polish Digital Mathematics Library

Résumé

This paper introduces a new learning algorithm for artificial neural networks, based on a fuzzy inference system ANBLIR. It is a computationally effective neuro-fuzzy system with parametrized fuzzy sets in the consequent parts of fuzzy if-then rules, which uses a conjunctive as well as a logical interpretation of those rules. In the original approach, the estimation of unknown system parameters was made by means of a combination of both gradient and least-squares methods. The novelty of the learning algorithm consists in the application of a deterministic annealing optimization method. It leads to an improvement in the neuro-fuzzy modelling performance. To show the validity of the introduced method, two examples of application concerning chaotic time series prediction and system identification problems are provided.

Détail

Publié le : 2005-01-01

Zbl 1107.68080

EUDML-ID : urn:eudml:doc:207767

Bibliographie

[000] Bezdek J.C. (1982): Pattern Recognition with Fuzzy Objective Function Algorithms. - New York: Plenum Press. | Zbl 0503.68069

[001] Box G.E.P. and Jenkins G.M. (1976): Time Series Analysis. Forecasting and Control. - San Francisco: Holden-Day. | Zbl 0363.62069

[002] Chen J.Q., Xi Y.G. and Zhang Z.J. (1998): A clustering algorithm for fuzzy model identification. - Fuzzy Sets Syst., Vol. 98, No. 3, pp. 319-329.

[003] Cho K.B. and Wang B.H. (1996): Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction. - Fuzzy Sets Syst., Vol. 83, No. 3, pp. 325-339.

[004] Chung F.L. and Duan J.C. (2000): On multistage fuzzy neural network modeling. - IEEE Trans. Fuzzy Syst., Vol. 8, No. 2, pp. 125-142.

[005] Czogała E. and Łęski J. (1996): A new fuzzy inference system with moving consequents in if-then rules. Application to pattern recognition. - Bull. Polish Acad. Sci., Vol. 45, No. 4, pp. 643-655. | Zbl 0954.68133

[006] Czogała E. and Łęski J. (1999): Fuzzy and Neuro-Fuzzy Intelligent Systems. - Heidelberg: Physica-Verlag. | Zbl 0953.68122

[007] Czogała E. and Łęski J. (2001): On equivalence of approximate reasoning results using different interpretations of if-then rules. - Fuzzy Sets Syst., Vol. 117, No. 2, pp. 279-296. | Zbl 0985.68067

[008] German S. and German D. (1984): Stochastic relaxation, Gibbs distribution and the Bayesian restoration in images. - IEEE Trans. Pattern Anal. Mach. Intell., Vol. 6, No. 9, pp. 721-741.

[009] Jang J.S.R. (1993): ANFIS: Adaptive-Network-Based Fuzzy Inference System. - IEEE Trans. Syst. Man Cybern., Vol. 23, No. 3, pp. 665-685.

[010] Jang J.S.R. and Sun C.T. (1993): Functional equivalence between radial basis function networks and fuzzy inference systems. - IEEE Trans. Neural Netw., Vol. 4, No. 1, pp. 156-159.

[011] Jang J.S.R. and Sun C.T. (1995): Neuro-fuzzy modeling and control. - Proc. IEEE, Vol. 83, No. 3, pp. 378-406.

[012] Jang J.S.R., Sun C.T. and Mizutani E. (1997): Neuro-Fuzzy and Soft Computing. A Computational Approach to Learning and Machine Intelligence. - Upper Saddle River: Prentice-Hall.

[013] Juang C. and Lin C. (1998): An on-line self-constructing neural fuzzy inference network and its applications. - IEEE Trans. Fuzzy Syst., Vol. 6, No. 1, pp. 12-32.

[014] Kim E., Park M. and Ji S. (1997): A new approach to fuzzy modeling. - IEEE Trans. Fuzzy Syst., Vol. 5, No. 3, pp. 328-337.

[015] Kirkpatrick S., Gelatt C. and Vecchi M. (1983): Optimization by simulated annealing. - Science, Vol. 220, No. 4598, pp. 671-680. | Zbl 1225.90162

[016] Kosko B. (1987): Fuzzy associative memories In: Fuzzy Expert Systems (A. Kandel, Ed.). - Boca Raton: CRC Press.

[017] Łęski J. (2003): ε-insensitive learning techniques for approximate reasoning systems. - Int. J. Comput. Cognit., Vol. 1, No. 1, pp. 21-77.

[018] Lin Y. and Cunningham G.A. (1995): A new approach to fuzzy-neural modeling. - IEEE Trans. Fuzzy Syst., Vol. 3, No. 2, pp. 190-197.

[019] Mamdani E.H. (1974): Applications of fuzzy algorithms for control of simple dynamic plant. - Proc. IEEE, Vol. 121, No. 12, pp. 1585-1588.

[020] Mamdani E.H. (1976): Advances in the linguistic synthesis of fuzzy controller. - Int. J. Man-Mach. Stud., Vol. 8, No. 6, pp. 669-678. | Zbl 0344.68028

[021] Mamdani E.H. (1977): Applications of fuzzy logic to approximate reasoning using linguistic synthesis. - IEEE Trans. Comput., Vol. 26, No. 12,pp. 1182-1191. | Zbl 0397.94025

[022] Mamdani E.H. and Assilian S. (1975): An experiment in linguistic synthesis with a fuzzy logic controller. - Int. J. Man-Mach. Stud., Vol. 7, No. 1, pp. 1-13. | Zbl 0301.68076

[023] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H. and Teller E. (1953): Equation of state calculation by fast computing machines. - J. Chem. Phys., Vol. 21, No. 6, pp. 1087-1092.

[024] Mitra S. and Pal S.K. (1995): Fuzzy multi-layer perceptron, inferencing and rule generation. - IEEE Trans. Neural Netw., Vol. 6, No. 1, pp. 51-63.

[025] Pedrycz W. (1984a): An identification algorithm in fuzzy relational systems. - Fuzzy Sets Syst., Vol. 13, No. 2, pp. 153-167. | Zbl 0554.93070

[026] Pedrycz W. (1984b): Identification in fuzzy systems. - IEEE Trans. Syst. Man Cybern., Vol. 14, No. 2, pp. 361-366. | Zbl 0551.93015

[027] Rao A.V. and Rose K. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models. - IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159-173.

[028] Rao A.V., Miller D., Rose K. and Gersho A. (1997): Mixture of experts regression modeling by deterministic annealing. - IEEE Trans. Signal Process., Vol. 45, No. 11, pp. 2811-2820.

[029] Rose K. (1991): Deterministic Annealing, Clustering and Optimization. - Ph.D. Thesis, California Inst. Technol, Pasadena.

[030] Rose K. (1998): Deterministic annealing for clustering, compression, classification, regression and related optimization problems. - Proc. IEEE, Vol. 86, No. 11, pp. 2210-2239.

[031] Rose K. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models. - IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159-173.

[032] Rutkowska D. (2001): Neuro-Fuzzy Architectures and Hybrid Learning. - Heidelberg: Physica-Verlag. | Zbl 1005.68127

[033] Schuster H.G. (1984): Deterministic Chaos. - Weinheim: VCH Verlagsgesellschaft. | Zbl 0707.58004

[034] Sugeno M. and Kang G.T. (1988): Structure identification of fuzzy model. - Fuzzy Sets Syst., Vol. 28, No. 1, pp. 15-33. | Zbl 0652.93010

[035] Sugeno M. and Yasukawa T. (1993): A fuzzy-logic based approach to qualitative modeling. - IEEE Trans. Fuzzy Syst., Vol. 1, No. 1, pp. 7-31.

[036] Tong R.M. (1980): The evaluation of fuzzy models derived from experimental data. - Fuzzy Sets Syst., Vol. 4, No. 13, pp. 1-12. | Zbl 0453.93003

[037] Wang L. and Langari R. (1995): Building Sugeno-type models using fuzzy discretization and orthogonal parameter estimation techniques. - IEEE Trans. Fuzzy Syst., Vol. 3, No. 4, pp. 454-458.

[038] Xie X.L. and Beni G. (1991): A validity measure for fuzzy clustering.- IEEE Trans. Pattern Anal. Mach. Intell., Vol. 13, No. 8, pp. 841-847.

[039] Xu C.W. and Lu Y.Z. (1987): Fuzzy model identification and self-learning for dynamic systems. - IEEE Trans. Syst. Man Cybern., Vol. 17, No. 4, pp. 683-689. | Zbl 0643.93018

[040] Yager R.R. and Filev D.P. (1984): Essentials of Fuzzy Modeling and Control. - New York: Wiley.

[041] Yen J., Wang L. and Gillespie C.W. (1998): Improving the interpretability of TSK fuzzy models by combining global learning and local learning. - IEEE Trans. Fuzzy Syst., Vol. 6, No. 4, pp. 530-537.

[042] Zadeh L.A. (1965): Fuzzy sets. - Inf. Contr., Vol. 8, No. 3, pp. 338-353. | Zbl 0139.24606

[043] Zadeh L.A. (1971): Towards a theory of fuzzy systems, In: Aspects of Network and System Theory (R.E. Kalman and N. DeClaris, Ed.). - New York: Holt, Rinehart & Winston.

[044] Zadeh L.A. (1973): Outline of a new approach to the analysis of complex systems and decision processes. - IEEE Trans. Syst. Man Cybern., Vol. 3, No. 1, pp. 28-44. | Zbl 0273.93002

[045] Zikidis K.C. and Vasilakos A.V. (1996): ASAFES2: A novel, neuro-fuzzy architecture for fuzzy computing, based on functional reasoning. - Fuzzy Sets Syst., Vol. 83, No. 1, pp. 63-68.