Neuro-fuzzy systems with inference based on bounded product
Danuta Rutkowska, Leszek Rutkowski, Robert Nowicki
Department of Computer Engineering, TechnicalUniversity of Czestochowa, Poland
Abstract:
The paper refers to neuro-fuzzy systems, also called fuzzy inference neural networks. The connectionist architectures that represent fuzzy systems based on Mamdani approach with the bounded product operation applied to fuzzy inference are determined. They are compared with the architectures of the well known systems which employ max-min or max-product inference.
- Introduction
The best known and most often applied fuzzy systems are Mamdani approach systems with the max-min or max-product type of inference. The max operation refers to the aggregation of the inferred fuzzy sets. The min and product operators are T-norm operators, used as inference rules. They correspond to Mamdani and Larsen rule of inference, respectively (see e.g. [10], [3], [5}, [7]).
The min and product operators are very convenient from their mathematical form point of view. Therefore, they have been chosen as the representation of IF-THEN rules in the fuzzy systems based on Mamdani and Larsen type of inference.
In this paper we consider fuzzy systems with the inference based on bounded productT-norm operator. The inference process are illustrated in Section2 and the connectionist neuro-fuzzy architectures are presented in Sections 3 and 4. Computer simulations that show performance of the systems are described in Section 5.
It will be shown, in Section 3, that in the special case, the neuro-fuzzy systems based on bounded productT-norm are equivalent to the systems based on Mamdani and Larsen inference rules.
- Fuzzy inference
Mamdani approach to fuzzy inference applies Tnorm operators (min or product) to represent the fuzzy IF-THEN rules
(1)
where and are linguistic variables, and are antecedent and consequent fuzzy sets, characterized by the membership functions and , respectively.
The Mamdani and Larsen rule of inference are expressed as follows
(2)
where the previous equation corresponds to Mamdani (min operator) and the latter one to Larsen (product operator) type of inference.
Analogously, we can employ the bounded productT-norm
(3)
where denotes .
The inference based on the individual rules(1) refers to so-called FITA approach, i.e. First Inference Then Aggregate. The fuzzy sets , referred from the individual rules , where , are aggregated to obtain one output fuzzy set. Another kind of inference, called FATI, i.e. First Aggregate Then Inference, leads to the same result when Mamdani approach is applied. Different results are obtained using logical approach to fuzzy inference, based on implications in a logical sense, instead of the Tnorms [1].
The FITA fuzzy inference process based on bounded productT-norm operator, is illustrated in Fig.1. The inferred fuzzy sets , , are obtained according to the following formula
(4)
and Equation (3), where
(5)
and is a crisp input of the fuzzy system; is called antecedent matching degree or rule firing strength.
Formula (4) comes from the compositional rule of inference (the supstar composition) [11], assuming that the singleton fuzzifier is employed in the fuzzy system [10], [6].
Fig.1. Fuzzy inference based on
bounded product
Three Gaussian-shaped consequent fuzzy sets , , are portrayed in Fig.1. We assume that these fuzzy sets are normal, so the membership values for their centers equal to 1. The fuzzy sets, , inferred from the individual rules , according to the bounded product inference rule (3) are presented in the figure. The membership functions of these fuzzy sets are obtained from the membership functions of fuzzy sets by shifting them down, so the maximal values of the shifted membership functions equal to, and of course, the membership values for all are greater than or equaled to 0.
At the bottom of Fig.1 there is the membership function of fuzzy set , obtained by aggregation of fuzzy sets , by use of max operation, . Similar fuzzy set is determined by applying algebraic or bounded Snorm operator instead of max operator.
It is worth comparing the shapes of fuzzy sets , inferred based on bounded productT-norm, with the fuzzy sets inferred in the systems with Mamdani and Larsen rules of inference. In these cases, the membership functions of fuzzy sets are “clipped” and “scaled” versions of the membership functions of fuzzy sets , respectively [2].
In spite of different shapes of membership function of the output fuzzy set , obtained by use of different Tnorm operators (min, product, or bounded product), the crisp output value can be the same if the special form of the center of area (COA) defuzzification method is applied. This situation will be explained in the Section 4.
3. CA neuro-fuzzy architectures
Let us consider the fuzzy system, introduced in Section 2, with the singleton fuzzifier, and center average (CA) defuzzification method
(6)
where , , are centers of membership functions , i.e. the points in such that
(7)
From formulas (6) and (4) we have the expression
(8)
For the Mamdani and Larsen rule of inference, expressed by Equation (2), as well as for the bounded product rule given by Equation (3), from formula (7) we have
(9)
Substituting (9) into (8) we obtain
(10)
The connectionist architecture, which represents the systems described by Equation(10), is shown in Fig.2.
Fig.2. The architecture of Mamdani approach
systems; in the general form
The elements of the first layer of the architecture portrayed in Fig.2 realize membership functions . The special cases of this architecture are obtained for the specific definition of the Cartesian product , by means of min or product operation. These neuro-fuzzy systems, for Mamdani and Larsen rule of inference, have been introduced in [10], and studied in [6], [7]. Figure 3 illustrates this architecture. The second layer contains elements which realize the Cartesian product. Thus, these elements perform min or product operations, respectively. The last layers conduct the CA defuzzification method, so they are composed of summation and division elements, according to Equation(10).
Fig.3. The architecture of Mamdani approach
systems; with Cartesian product layer
Let us notice that in the case of product operation as the Cartesian product, and Gausssian membership functions (with the same width parameters), the architecture shown in Fig.3 represents the normalized version of radial basis function (RBF) network [4].
It is worth emphasizing that the neuro-fuzzy systems with the singleton fuzzifier, the inference based on the bounded product, and the CA defuzzification method, is equivalent to the well known neuro-fuzzy systems based on Mamdani and Larsen rule of inference, presented in [10].
- COA neuro-fuzzy architectures
Let us consider the fuzzy system, introduced in Section 2, with the singleton fuzzifier, and the following defuzzification method
(11)
where is the crisp output of the system, and , , are centers of membership functions , defined by Equation (7).
Formula (11) is a special case of the discrete version of COA defuzzification.
It was said in Section2 that fuzzy set was obtained by aggregation of fuzzy sets , , by use of max or other Snorm operators. Hence
(12)
where is the Snorm operator, e.g. max, algebraic or bounded Snorm operator.
From formulas (12), (4), and (11), it is easy to obtain the following expression
(13)
The norm in Equation (13) can be written as follows
(14)
where denotes .
For the Mamdani and Larsen rule of inference, and for the bounded product rule, equality (9) is satisfied. Hence, we obtain the following form of Equation (14)
(15)
where is defined by formula (2) or (3), respectively, i.e.
(16)
where
(17)
Formulas (13), (15), and (16) constitute the mathematical description of the fuzzy systems based on Mamdani, Larsen, and bounded product rule of inference.
Figure 4 illustrates the neuro-fuzzy architecture of the system described by Equations (13), (15), and (16). The T-norm elements perform the T-norm operations in formula (16), i.e. Mamdani, Larsen, or bounded product rule of inference. The S-norm elements realize the S-norm operator in Equations (13), (15), i.e. the aggregation operator, usually chosen as max operator, but others are also possible to apply, for example algebraic or bounded Snorm.
Fig.4. The COA neuro-fuzzy architecture
Now, let us assume that
(18)
for and .
It is easy to notice that if assumption (18) is fulfilled then Equation (13) will take the form of formula (10). In this case, the neuro-fuzzy architecture presented in Fig.4 is reduced to the architecture depicted in Fig.2.
For the fuzzy sets , , portrayed in Fig.1, assumption (18) is satisfied. In this case, the CA defuzzification method leads to the same result (the same crisp output ) as the special discrete version of the COA method, defined by Equation (11).
- Computer simulations
The neuro-fuzzy systems presented in this paper can solve different tasks. The truck backer-upper control problem, a function approximation, as well as classification tasks have been chosen to illustrate performance of the systems. The computer simulations have shown that the neuro-fuzzy system with the inference based on bounded product performs similarly to the systems based on Mamdani and Larsen rules of inference. The neuro-fuzzy architectures depicted in Fig.2, 3, or 4 have been learnt by use of the gradient algorithm, which is similar to the back-propagation method, commonly applied to train artificial neural networks [12]. The FLiNN software [5] have been employed in order to train the neuro-fuzzy architectures [6], [7].
The truck backer-upper control problem is the very well known example of control tasks, solved by the Mamdani approach fuzzy or neuro-fuzzy systems; see e.g. [10]. Figure 5 shows trajectories of the truck, successfully controlled by the system with the inference based on bounded product (the architecture presented in Fig.4 with the bounded productTnorm). These trajectories are similar to that obtained by use of the neuro-fuzzy architecture depicted in Fig.4 with min or product Tnorm operators. They are also similar to the trajectories of the Mamdani approach systems represented by the architectures illustrated in Fig.2 or 3.
Fig.5. Trajectories of the truck in the
truck backer-upper control problem
Figure 6 portrays the result of the approximation of linear function , by use of the neuro-fuzzy system with the architecture shown in Fig.4, where the Tnorm operators are bounded productTnorm. The similar result have been obtained, applying the neuro-fuzzy architecture depicted in Fig.4 with min or product Tnorm operators, as well as the Mamdani approach systems illustrated in Fig.2 or 4.
Fig.6. Illustration of the approximation
of linear function
The next example of the applications of the neuro-fuzzy systems is the classification problem. The task is to classify the points located in the square area to three different classes: two semi-rings and the area of the square beyond the semi-rings [9]. Figure 7 shows the classification result obtained by the neuro-fuzzy system with the inference based on bounded product (the architecture presented in Fig.4 with the bounded productTnorm). The similar result have been carried out by use of the neuro-fuzzy architecture depicted in Fig.4 with min or product Tnorm operators, as well as the systems represented by the architectures portrayed in Fig.2 or 3.
Fig.7. Illustration of the classification problem
Another classification task, solved by the neuro-fuzzy systems, refers to the medical diagnosis problem, concerning a tumor of mucous membrane of uterus [7], [8]. The neuro-fuzzy system with the inference based on bounded product provides a perfect diagnose, even better than the systems which employ Mamdani and Larsen rules of inference, i.e. min or product Tnorm.
6. Conclusions
If assumption (18) is satisfied, the neuro-fuzzy architectures of the systems based on bounded productT-norm are the same as the architectures of the systems that employ min Tnorm (Mamdani rule) and product Tnorm (Larsen rule). In this case, the same connectionist architectures are obtained for the systems with CA defuzzifier and for the systems which use the special discrete versions of COA defuzzification method, given by Equation (11). From this point of view, we can say that these neuro-fuzzy systems are equivalent. However, if condition (18) is not fulfilled, the neuro-fuzzy architectures will be different, and more complicated.
The architecture presented in Fig.4 contains more elements than the architecture depicted in Fig.2 but it constitutes the more general case of the neuro-fuzzy systems. Each of the systems can be trained by means of the gradient method, similar to the back-propagation learning of neural networks. This learning algorithm can be conducted by the FLiNN software, based on the system architectures. Thus, it is not necessary to determine the mathematical formulas of the gradient learning methods.
The computer simulations, described in Section 5, have shown that results of performance of the neuro-fuzzy systems based on bounded productT-norm is similar or even better than the performance of the corresponding systems with min or product Tnorm.
The system based on bounded product is an alternative to Mamdani approach system with Mamdani (min) or Larsen (product) type of inference. In the logical approach systems there are many different choices concerning the type of inference, based on different logical implications [1], [9].
References
- Czogala E. and Leski J., Fuzzy and Neuro-Fuzzy Intelligent Systems, Physica-Verlag, Heidelberg, New York, 2000.
- Driankov D., Hellendoorn H., and Reinfrank M., An Introduction to Fuzzy Control, Springer-Verlag, Berlin, Heidelberg, 1993.
- Jager R., Fuzzy Logic in Control, Thesis Technishe Universiteit Delft, Delft, 1995.
- Moody J. and Darken C., Fast learning in networks of locally-tuned processing units, Neural Computation, Vol.1, No.2, 1989, pp.281-294.
- Pilinski M., FliNN – User Mannual, Polish Neural Network Society, Czestochowa, Poland, 1997 (in Polish).
- Rutkowska D., Pilinski M., and Rutkowski L., Neural Networks, Genetic Algorithms, and Fuzzy Systems, PWN, Warsaw, Poland, 1997 (in Polish).
- Rutkowska D., Intelligent Computational Systems, PLJ Academic Publishing House, Warsaw, Poland, 1997 (in Polish).
- Rutkowska D. and Nowicki R., New neuro-fuzzy architectures, Proc. Int. Conf. on Artificial and Computational Intelligence for Decision, Control and Automation in Engineering and Industrial Applications, ACIDCA’2000, Intelligent Methods, Monastir, Tunisia, March, 2000, pp. 82-87.
- Rutkowska D. and Nowicki R., Implication – based neuro-fuzzy architectures, International Journal of Applied Mathematics and Computer Science, No.4, 2000.
- Wang L.-X., Adaptive Fuzzy Systems and Control, PTR Prentice Hall, Englewood Cliffs, New Jersey, 1994.
- Zadeh L.A., Outline of a new approach to the analysis of complex systems and decision processes, IEEE Trans. on Systems, Man, and Cybern. SMC-3, No.1, 1973, pp.28-44.
- Zurada J. M., Introduction to Artificial Neural Systems, West Publishing Company, New York, 1992.