An Improved Performance Phase Fuzzy ARTMAP Algorithm (Fuzzy ARTVar) in Power System Applications

SHAHRAM JAVADI

AzadUniversity – Tehran Central Branch

Electrical Engineering Department
Moshanir Power Electric Company
IRAN
Abstract:
This paper presents an improved Fuzzy ARTMAP Neural Network algorithm named Fuzzy ARTVar for short term load forecasting assessment.

In this paper we improve the performance phase of Fuzzy ARTMAP by an algorithm named ARTVar.

In particular, we consider one of the major limitations of Fuzzy ARTMAP, its dependence on tuning parameters. The performance of Fuzzy ARTMAP depends on the values of two parameters called the choice and vigilance parameters, and also on the order of pattern presentation for the off-line mode of training. This property was improved by one of methods, which works on order of data fed to network named ordered Fuzzy ARTMAP and it has shown in my last paper. Another algorithm has developed for improving performance phase of Fuzzy ARTMAP in this approach. Experimental results have shown that Fuzzy ARTVar exhibits superior generalization performance, compared to Fuzzy ARTMAP, for a variety of machine learning database.

What is worth nothing is that the performance of Fuzzy ARTVar is independent of the tuning of network parameters, in contrast with the other version of Fuzzy ARTMAP such as ARTEMAP, ARTEMAPQ and etc whose performance depends on the choice of certain network parameters.

To demonstrate the effectiveness of the proposed neural network, short term load forecasting is performed on the IRAN power system. Test results indicate that the special neural network is very effective in improving the accuracy of the forecast hourly loads. For on-line training, the Fuzzy ARTMAP network was found to be a better choice than the other neural networks.

Keywords: Fuzzy ARTMAP, Fuzzy ARTVar, Neural Network, Power Systems, Short Term Load Forecasting,

1. Introduction

Pattern classification is a key element to many engineering solutions. Sonar, radar, seismic, and diagnostic applications all require the ability to accurately classify data. Control, tracking and prediction systems will often use classifiers to determine input – output relationships. Because of this wide range of applicability, pattern classification has been studied a great deal. Simpson in his Fuzzy Min - Max paper [1] identified a number of desirable properties that a pattern classifier should possess. These properties are listed below:

Property 1: On-Line Adaptation,

Property 2: Non-Linear separability,

Property 3: Short Training Time,

Property 4: Soft and Hard Decision,

Property 5: Verification and Validation,

Property 6: Independence from Tuning Parameters, Property 7: Nonparametric Classification,

Property 8: Overlapping Classes.

A neural network classifier that satisfies most of these properties is Fuzzy ARTMAP [2]. Fuzzy ARTMAP is a member of the class of neural network architectures referred to as ART-architectures developed by Carpenter, Grossberg, and their colleagues at BostonUniversity. The ART-architectures are based on the ART theory introduced by Grossberg in [3]. In this paper, we focus on a variation of the performance phases of the Fuzzy ARTMAP algorithm that provides, in many instances, an improved performance. We refer to this variation of the Fuzzy ARTMAP algorithm as Fuzzy ARTVar. It is also worth mentioning that recently, modifications of the performance phase of Fuzzy ARTMAP algorithm have appeared in the literature (e.g., ARTEMAP (power rule in [4]), ARTEMAPQ (Q-max rule in [4]), and ARTMAP-IC [5]) that improved the performance of Fuzzy ARTMAP. One of the important difference between Fuzzy ARTVar and ARTEMAP, ARTEMAPQ and ARTMAP-IC is that the performance of the latter three algorithms depends on network parameters ( for ARTMAP, Q for ARTEMAPQ, and  and Q for ARTMAP-IC), while Fuzzy ARTVar dose not. Hence, Fuzzy ARTVar satisfies one of the important properties of classifier system that is independence from tuning parameters, while ARTMAP, ARTEMAPQ, and ARTMAP-IC do not. Note that in the simulations reported in this paper the choice parameters (a) and the baseline vigilance parameters (a) of Fuzzy ARTVar, ARTEMAP and ARTEMAPQ are chosen equal to zero.

The organization of the paper is as follows: In Section 2, we discuss briefly the Fuzzy ARTMAP architecture, its training phases, and its performance phase. In Section 3, we introduce the modification of the performance phase of the Fuzzy ARTMAP algorithm that leads us to the algorithms that we called Fuzzy ARTVar. In section 4 we experimentally demonstrate the superiority of Fuzzy ARTVar versus Fuzzy ARTMAP for a familiar application in electrical distribution network, short term load forecasting. In the same section, we provide performance comparisons between Fuzzy ARTVar, ARTEMAP, ARTEMAPQ, and Gaussian ARTMAP for the same set of databases. Gaussian ARTMAP is an ART-based algorithm ([6]) whose training phase and performance phase differ from the corresponding phase in Fuzzy ARTMAP, but Gaussian ARTMAP’s operation in the training phase resemble the operations of Fuzzy ARTVar in its performance phase. We see in section 4, that Fuzzy ARTVar compares very favorably with ARTEMAP, ARTEMAPQ, and Gaussian ARTMAP, despite the fact that each one of these algorithms depends on the choice of a parameter, whose optimum value is database dependent. In Section 5, we provide a review of the paper and some conclusive remarks.

2. The Fuzzy ARTMAP Neural Network

A detailed description of the Fuzzy ARTMAP neural network can be found in [2]. For completeness, in the following, we present only the necessary details.

The Fuzzy ARTMAP neural network consists of two Fuzzy ART modules, designated as ART and ART, as well as an inter–Art module as shown in figure 1. Inputs (a’s) are presented at the ARTa module, while their corresponding outputs (b’s) are presented at the ARTb module. The inter–ART module includes a MAP field whose purpose is to determine whether the correct mapping has been established from inputs to outputs.

Fig.1. A typical Fuzzy ARTMAP architecture

Fuzzy ARTMAP can operate in two distinct phases: the training phase and the performance phase. In this paper we focus on classification tasks, where many inputs are mapped to a single, distinct output. It turns out that for classification tasks, the operations performed at the ARTb and inter–ART modules can be ignored, and the algorithm can be described by referring only to the top-down weights and the parameters of the ARTa module.

3. The Fuzzy ARTVar Algorithm

The training phase of the Fuzzy ARTVar algorithm is identical with the training phase of the Fuzzy ARTMAP algorithm. After the training is over in Fuzzy ARTVar, we go through another phase that we call pre-performance phase. The purpose of the pre-performance phase is to compute, for every committed node in , the sample mean vector and the sample standard deviation vector of all the input training patterns that chose this node. The sample standard deviation vector of the node j in

and the sample standard deviation vector of the node j are denoted by and ,

Respectively. These vectors are then used in the performance phase of Fuzzy ARTVar to produce the outputs of the test patterns.

The main difference between Fuzzy ARTMAP is that in Fuzzy ARTMAP every node (category) j in is represented by the weight, or the two endpoints u and v of its corresponding hyperectangle (for more details see [7]). On the other hand, in Fuzzy ARTVar every node j is represented by its mean vector , and its standard deviation vector. It is therefore, very reasonable to refer to these values as weight values during the performance phase of Fuzzy ARTVar.

Performance Phase of Fuzzy ARTVar

1. Initialize the values of the committed weight vectors in to the values that they had at the end of pre-performance phase. A node in is committed if it had coded at least one training input pattern during the Fuzzy ARTVar training phase.

Also, associate every committed node in of the trained Fuzzy ARTVar with the output pattern that it was mapped to at the end of the Fuzzy ARTVar training phase.

Initialize the index r to the value of one.

2. Choose the r-th input pattern from the test list.

3. Calculate the Mahalanobis distance of from each in, according to the following equation. When calculating the Mahalanobis distance consider only the committed nodes in (i.e., nodes with index j, such that 1< j< Na-1)

(1)

In the above equationrepresents the covariance matrix of the members of node j in , with diagonal elements equal to the aforementioned variances (calculated in the pre-performance phase), and off-diagonal elements equal to zero.

4. Choose the node in that produces the minimum Mahalanobis distance. Assume that this node has index j. that is,

(2)

Designate the output of the presented input pattern equal to the output pattern that node j was associated to at the end of the Fuzzy ARTVar training phase.

5. If this is the last input/output pair in the performance phase is considered complete. Otherwise, go to step 2, to present the next in line input pair, by increasing the value of the index r by one.

4. Experimental Results-Load Forecasting problem

In order to demonstrate the superior performance of Fuzzy ARTVar compared to Fuzzy ARTMAP we chose to conduct experiments on one of familiar application in Electrical distribution network, load forecasting.

A number of algorithms have been suggested for the load-forecasting problem. Previous approaches can be generally classified into two categories in accordance with techniques they employ. One approach treats the load pattern as a time series signal and predicts the future load by using various time series analysis techniques [8-14].

The idea of the time series approach is based on the understanding that a load pattern is nothing more than a time series signal with known seasonal, weekly, and daily periodicities. These periodicities give a rough prediction of the load at the given season, day of the week, and time of the day. The difference between the prediction and the actual load can be considered as a stochastic process. By the analysis of this random signal, we may get more accurate prediction. The techniques used for the analysis of this random signal include the Kalman filtering, the Box-Jenkins method, the autoregressive moving average (ARMA) model, and spectral expansion technique.

The Kalman filter approach requires estimation of a covariance matrix. The possible high nonstationarity of the load pattern, however, typically may not allow an accurate estimate to be made.

These methods are very time consuming and difficult. More recently the application of neural network has developed in many of engineering problems. One of these problems is forecasting of load hourly by back propagation method or KOHONEN neural network classifier [15].

In this paper, a different approach is proposed for load forecasting. This approach is based on Fuzzy ARTMAP network. Because of self-organized characteristic of these networks, they can be used online in power systems for load forecasting. It is shown in figure (2) [16].

Fig.2. On-Line Training

One of the measures of performance that we used in comparing Fuzzy ARTVar and Fuzzy ARTMAP is the generalization performance of these networks. The generalization performance of these networks is defined to be the percentage of patterns in the test set that are correctly classified by a trained network. Since the performance of Fuzzy ARTMAP and Fuzzy ARTVar depends on the order of the pattern presentation in the training set, ten different random orders of pattern presentation will be investigated, and performance measures such as the average generalization performance, the minimum generalization performance, the maximum generalization performance, and the standard deviation of the generalization performance will be produced.

Also, another measure of performance for comparing neural networks is the size of the networks created. In order to compare the sizes of the networks that Fuzzy ARTVar and another algorithm create (e.g., Fuzzy ARTMAP) we compare the average compression ratio of the other algorithm versus the average compression ratio of Fuzzy ARTVar. The average compression ratio for Fuzzy ARTVar (other algorithm) is defined to be the ratio of the average number of nodes created in versus the number of patterns used in the training of Fuzzy ARTVar (other algorithm).

Since in Fuzzy ARTVar, the criterion of choosing a node in during the presentation of the test pattern is the minimization of (1), nodes with zero variances across some dimension will never be chosen (because then the corresponding covariance matrix inverses will be infinite). To alleviate this problem if some node variances were found to be zero, we substituted these zero variances with the minimum of the positive variance corresponding to this node. The resulting algorithm, we named Fuzzy ARTVarc. From now, when we refer in the main text to Fuzzy ARTVar we will imply either Fuzzy ARTVar or Fuzzy ARTVarc.

5. Simulations

In order to test the algorithm for its effectiveness in Load Forecasting of a power system, we chose data, which is obtained from dispatching center of TAVANIR Co.

We study 2 cases. In cases 1, we use Fuzzy ARTMAP Network and in case 2, an improved Fuzzy ARTMAP Network is used. Finally the obtained results are compared.

In each case, performance error of neural network is calculated according to the following formula:

(3)

Where,

ydi : Desired output of Neural Network.

yai : Actual output of Neural Network.

N : Number of Data Set for Training.

Case 1 (Fuzzy ARTMAP network):

In this case we use a Fuzzy ARTMAP neural network to predict load of next day according current day. In this test, parameter ρwas chosen to be ρa=0.95, ρb=0.94, ρab=0.93. A set of 1000 training patterns was selected from the entire set. After training the network with 1000 patterns, the set of 1000 remained patterns was used to test network. It can be shown in figure (3).

Case 2 (Fuzzy ARTVAR):

In this case we used animproved Fuzzy ARTMAP Neural-Network, Fuzzy ARTVar. Also we used the same input bit patterns. Error in this case is higher than the above cases and computing time for training is too high. It is shown in figure (4).

The major motivation for our work was the design of a Fuzzy ARTMAP algorithm that is independent of the tuning of parameters, and achieves good generalization by avoiding excessive experimentation. The dependence of Fuzzy ARTMAP on the choice parameter and the vigilance parameter is an inherent characteristic of the algorithm. Choosing these parameters equal to zero frees the experimenter from the tedious task of optimizing the network performance with respect to these two parameters. With the choice parameter and the vigilance parameter chosen equal to zero, one ends up with a fuzzy ARTMAP algorithm that exhibits a significant variation in generalization performance for different orders of training pattern presentations. Furthermore, it is not an easy task to guess which one of the exceedingly large number of orders of pattern presentations exhibits the best generalization.

5. Review – Conclusions

We introduced a variation of the performance phase Fuzzy ARTMAP, that we called Fuzzy ARTVar. We demonstrated that for a number of classification problems the performance of Fuzzy ARTVar is superior to the performance of Fuzzy ARTMAP (see Table 1). We have also implemented other variations of the performance phase of Fuzzy ARTMAP that have appeared in the literature (such as ARTEMAOP, ARTEMAPQ), as well as the Gaussian ARTMAP with Fuzzy ARTVar we observed that Fuzzy ARTVar compares favorably with each one of these algorithms despite the fact that the performance of these algorithms was optimized with respect to an appropriate network parameter. Knowing though there are very few instances that a new classification algorithm will outperform existing classification algorithms for every possible classification problem, we believe that Fuzzy ARTVar is a good algorithm to consider in conjunction with or instead of algorithms such as Fuzzy ARTMAP, ARTEMAP, ARTEMAPQ, or Gaussian ARTMAP.

Fig.3. Actual Load respected to Predicted Load by FAM neural network

Fig.4. Actual Load respected to Predicted Load by Fuzzy ARTVar neural network

Table 1: Comparison of Fuzzy ARTMAP and Fuzzy ARTVar generalization performances

Network / No. of Node in ARTa / Compression rate* / Minimum / Maximum / Average / Std. dev.
Fuzzy ARTMAP / 240 / 0.76 / 89.2 / 95.3 / 94.1 / 1.9
Fuzzy ARTVar / 110 / 0.89 / 92.3 / 97.2 / 96.2 / 1.4

*: Compression Ratio = (No. of test data – No. Node created in ARTa) / No. of Test data

Acknowledgement

The author is grateful to MOSHANIR, Power Electric Consultant Company, for supporting expenses in order to attend this conference.

MOSHANIR is the most important Power Electric Company in IRAN, which is working on various projects about Transmission Lines, Substation, Distribution and Power plant projects.

References

[1] P.K. Simpson, “Fuzzy min-max neural networks- part1: Classification,‘ IEEE Transactions on Neural Networks, Vol.3, No.5 pp. 776-786, September 1992.

[2] G. A. Carpenter, S. Grossberg, N. Markuzon, J. H. Reynolde, and D. B. Rossen, “ Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps” IEEE Transactions on Neural Networks, Vol. 3, No. 5, pp. 698-713, September 1992.

[3] S. Grossberg, “ Adaptive pattern recognition and universal recording II: Feedback, expectation, olfaction, and illusions, Biological Cybernetics, Vol. 23, pp. 187-202, 1976.

[4] G. Carpenter, and W.D. Ross, “ART-EMAP: A neural network architecture for object recognition by evidence accumulation,” IEEE Transaction on Neural Networks, Vol. 6, No. 4, pp. 805-818, July 1995.

[5] G. A. Carpenter, Ross, N. Markuzon, “ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases,’ Technical Report CAS/CNS- 96-017, BostonUniversity, May 1996.

[6] J. R. Williamson, “Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps,” Neural Networks, Vol. 9, No. 5, pp. 881-897, 1996.

[7] M. Georgiopoulos, H. Fernlund, G. Bebis, and G. L. Heileman, “Order of search in Fuzzy ART and Fuzzy ARTMAP: Effect of the choice parameter,”

Neural Networks, Vol. 9, No. 9, pp. 1541-1559, 1996.

[8] J. Toyoda, M. Chen, and Y. Inoue, “An Application of State Estimation to short-Term Load Forecasting, Partl: Forecasting Modeling”,”Part2: Implementation,” IEEE Tr. on Power App. and Sys., vol. PAS-89. pp. 167S-1688, Oct., 1970