Header Space reserved for Publication
Control chart pattern recognition using wavelet analysis
and neural networks
Chuen-Sheng Cheng 1, Hui-Ping Cheng 2, Kuo-Ko Huang 1
1 Dept. of Industrial Engineering and Management, Yuan-Ze University, Tao-Yuan, Taiwan 320, ROC
1 E-mail:
2 Department of Business Administration, Ming-Dao University, Changhua, Taiwan 52345, ROC
2 E-mail:
Abstract
Control charts are useful tool in detecting out-of-control situations in process data. There are many unnatural patterns that may exist in process data indicating the process is out of control. The presence of unnatural patterns implies that a process is affected by assignable causes, and corrective actions should be taken. Identification of unnatural patterns can greatly narrow the set of possible causes that must be investigated, and thus the diagnostic work could be reduced in length.
This paper presents a modified self-organizing neural network developed for control chart pattern analysis. The aim is to develop a pattern clustering approach when no prior knowledge of the unnatural patterns is available. This paper also investigates the use of features extracted from wavelet analysis as the components of the input vectors. Experimental results and comparisons based on simulated and real data show that the proposed approach performs better than traditional approach. Our research concluded that the extracted features can improve the performance of the proposed neural network.
Keywords: Control chart; Pattern recognition; Wavelet analysis; Feature extraction
1. Introduction
Statistical process control charts have been widely used for many years to monitor the quality characteristics of a process. A process is considered out of control if a point falls outside the control limits or a series of points exhibit an unnatural pattern (also known as nonrandom variation). Analysis of unnatural patterns is an important aspect of control charting. These unnatural patterns provide valuable information regarding potentialities for process improvement. It is well documented that a particular unnatural pattern on a control chart is often associated with a specific set of assignable causes (Western Electric Company, 1958). Therefore, once any unnatural patterns are recognized, the scope of process diagnosis can be greatly narrowed to a small set of possible causes that must be investigated, and thus the diagnostic search could be reduced in length.
Fig. 1 displays examples of the types of unnatural pattern, the reader is referred to the Western Electric Handbook (Western Electric Company, 1958) for a detailed description of these unnatural patterns. The typical unnatural patterns on control charts are defined as follows.
1. Trends. A trend can be defined as a continuous movement in one direction (either upward or downward).
2. Sudden shifts. A shift may be defined as a sudden or abrupt change in the average of the process.
3. Systematic variation. One of the characteristics of a natural pattern is that the point-to-point fluctuations are unsystematic or unpredictable. In systematic variations a low point is always followed by a high one or vice versa.
4. Cycles. Cyclic behaviour of the process mean can be recognized by a series of high portions or peaks interspersed with low portions or troughs.
5. Mixtures. In a mixture the points tend to fall near the high and low edge of the pattern with an absence of normal fluctuations near the middle. A mixture is actually a combination of data from separate distributions.
Numerous approaches have been proposed to control chart pattern recognition. These include statistical (Cheng & Hubele, 1996; Yang & Yang, 2005), rule-based expert system (Cheng, 1989) and artificial neural network techniques (Al-Assaf, 2004; Cheng, 1997; Guh & Hsieh, 1999; Guh & Tannock, 1999; Hassan et al., 2003; Hwarng & Hubele, 1993; Hwarng & Hubele, 1993; Jang et al., 2003; Pacella et al., 2004; Perry et al., 2001; Pham & Chan, 1998; Pham & Chan, 2001; Pham & Oztemel, 1992; Pham & Oztemel, 1994; Pham & Wani, 1997; Smith, 1994; Wani & Pham, 1999; Yang & Yang, 2002). Supervised neural networks have been successfully employed in references (Al-Assaf, 2004; Cheng, 1997; Guh & Hsieh, 1999; Guh & Tannock, 1999; Hassan et al., 2003; Hwarng & Hubele, 1993; Hwarng & Hubele, 1993; Jang et al., 2003; Perry et al., 2001; Pham & Oztemel, 1992; Pham & Oztemel, 1994; Pham & Wani, 1997; Smith, 1994; Wani & Pham, 1999; Yang & Yang, 2002). These neural networks learn to recognize patterns by being presented with representative samples during a training phase. Ideally, sample patterns should be developed from a real process. A common approach adopted by previous researches was to generate training samples based on predefined mathematical model. An implicit assumption of this approach is that the groups of unnatural patterns are known in advance. In actual cases, sufficient training samples of unnatural patterns may not be readily available. In addition, the use of pre-defined models may create problems for patterns not previously encountered. Unsupervised neural networks can be used to cluster data into groups with similar features. This approach has been studied in (Pacella et al., 2004; Pham & Chan, 1998; Pham & Chan, 2001).
Figure. 1 - Examples of control chart patterns.
This paper presents a self-organizing neural network developed for control chart pattern recognition. The aim is to develop a pattern clustering approach when no prior knowledge of the unnatural patterns is available. This paper also investigates the use of features extracted from wavelet analysis as the components of the input vectors.
The paper comprises six main sections. Section 2 briefly reviews self-organizing map. In Section 3, the SOM-based approach is explained in detail. Section 4 shows the results of using the proposed neural network to classify control chart patterns. Section 5 reports the performance evaluation based on a small set of patterns collected from manufacturing process. Finally, conclusions are made in Section 6.
2. Self-organizing map
Fig. 2 illustrates a typical architecture of the Kohonen self-organizing map. There are m cluster units, arranged in a one- or two- dimensional array; the input vectors are n-tuples. The weight vector for a cluster unit serves as an exemplar (or template) of the input patterns associated with that cluster. During the self-organization process, the cluster unit whose weight vector matches the input vector most closely is chosen as the winner. The winning unit and its neighbouring units (determined by a radius, R and neighbourhood topology) update their weights. The SOM neural network can be used to cluster a set of p continuous-valued vectors into m clusters.
The training of a Kohonen self-organizing map can be summarized as follows:
1. Set topological neighbourhood parameter R. Set learning rate parameter .
2. Initialize weights . The weights are initialized to random values chosen from the same range of values as the components of the input vectors.
3. Present an input vector from the training set to the map.
4. For each , compute .
5. Find index such that is a minimum.
6. For all units within a specified neighbourhood of , and for all :
.
7. Update learning rate when all input vectors have been presented.
Reduce radius of topological neighbourhood.
8. Repeat steps 3 to 7 until a specified number of iterations are completed.
The number of input units in the network, , is equal to the dimension of the vectors to be classified by the network. Note that the components of the input vector could be raw (unprocessed) data and/or a set of features extracted from the data. The number of output nodes, , is set arbitrarily. The value of represents the maximum number of clusters to be formed. If it is higher than the number of actual classes, only part of the output nodes will be committed. If is too high, many trivial class templates will be created. The learning rate is a slowly decreasing function of training epochs. The radius of the neighborhood around a cluster unit also decreases as the clustering process progresses.
Figure. 2 - Architecture of the Kohonen self-organizing neural network.
In the recall mode, only steps 4 to 6 are used. When a new input vector is presented to the network, the trained neural network will determine the best matched template. For more details on the Kohonen self-organizing map, refer to reference (Kohonen, 1989).
3. The proposed methodology
This paper presents a self-organizing neural network developed for control chart pattern recognition. As with other types of self-organizing neural networks, no desired output is provided with any input in the training phase. The network segregates the input vectors into clusters as training proceeds. The clusters identify the different classes present in the input data set.
A good classifier should have high classification accuracies with both the training and the test data sets and should involve only a small number of committed nodes by comparison with the number of patterns in the data set. The problem of the basic SOM is that many nodes are needed to obtain good accuracies (Pham & Chan, 1998). In order to alleviate this problem, we propose a modification to the training of the traditional SOM. The operation of the proposed network involves the following steps:
1. Formation of clusters
Apply SOM to cluster the training set into clusters. The value of is set substantially larger than the number of potential classes in the data set.
2. Merging of clusters
Treat the exemplars of the committed output nodes as the input vectors. Set the number of output nodes to , where , and apply the SOM procedure.
The rationale of the proposed modification is explained as follows. In the phase of cluster formation, the number of maximum clusters to be formed () is set at a high value so that input vectors with differing prominent features will cause new cluster nodes to be formed. In other words, the value of should be kept high to ensure that different patterns are separated into distinct classes. This approach will establish the most representative class template. However, a high value of will cause many trivial class templates to be created at the beginning of the training. The second phase can be thought of as merging clusters which seem to represent the same pattern class. It is intuitive that a clustering algorithm will perform better using a smooth version of input patterns (i.e., the class templates).
The components of the input vector could be raw (unprocessed) data and/or a set of features extracted from the data. Previous researches (Hassan et al., 2003; Pham & Wani, 1997; Smith, 1994; Wani & Pham, 1999) indicated that the performance of the neural network-based pattern recognizer can be improved using extracted statistical features as the input vector. Al-Assaf (2004) showed that neural network works on a reduced set of coefficients obtained from the wavelet analysis performs better than using raw data as input.
The present work investigates the use of features extracted from wavelet analysis. The characteristics would be extracted from the process data and added to the input vector before presenting to the neural network. Fast Haar transform (FHT) (Kaiser, 1998; Newland, 1993) was used to de-noise and extract important features from unnatural patterns. Consider a data vector consisting of numbers , where . We can form two new vectors, defined as follows:
(1)
(2)
where . Applying the same process, we obtain
(3)
(4)
where . Both and have components. In this research we will study the effect of input vector augmented by and . As an example, consider the following process data:
(5)
The extracted characteristics, and , are:
(6)
(7)
(8)
(9)
The overall (or augmented) input vector presented to the network would be:
(10)
4. Simulation results and discussion
The implementing of the proposed neural network-based algorithm requires no knowledge of the unnatural patterns. However, to estimate the performance of the proposed neural network, unnatural process data were simulated. The patterns include normal, cyclic (sine wave and cosine wave) pattern, increasing trend, decreasing trend, upward shift and downward shift.
The training data set consists of 462 input vectors (66 for each type). The testing data set contained the same number of input vectors as the training set. An input vector includes a time series of 32 data points. The generation of the different types of patterns for the training and test data sets is described in reference (Cheng, 1989). The parameters used for simulating unnatural patterns are given in Table 1. The topology and training parameters are given in Table 2. The proposed neural network was implemented in Matlab (MathWorks, 2004).
The proposed neural network will be evaluated according to the following criteria: it should have the minimum number of established clusters and the highest classification accuracy. A set of experiments was conducted to estimate the performance of the proposed methodology. The performance of the basic SOM was selected as the benchmark. In order to have a fair comparison, the number of maximum clusters was kept the same for all competitive procedures.
Table 1 - Parameters for simulating patterns
Pattern type / Parameters (in terms of )Increasing trend / gradient: 0.1 to 0.3
Decreasing trend / gradient: -0.3 to -0.1
Upward shift / shift magnitude: 0.5 to 3.0
Downward shift / shift magnitude: -3.0 to -0.5
Cyclic pattern / amplitude:0.5 to 3.0; period: 12
Table 2 - Topology and training parameters