The Use of Artificial Neural Networks for Data Analysis

THE USE OF ARTIFICIAL NEURAL NETWORKS FOR DATA ANALYSIS

A.V. Gavrilov, V.M. Kangler

Novosibirsk state technical university, Novosibirsk, Russia, Email:

Abstract.

The principles of application of Hopfield neural network for Data Analysis is proposed in this report. Here the artificial neural networks are used for realization of associative memory representing information contents of a relational database (DB). This approach is used for solve the task of detection of associative interrelations between of fields of Data Base and forecasting of values of its.

Introduction.

Critical competitive struggle, aspiration of the companies to receive profit from the investments in information technologies, growth of number of the employees accepting the decisions, stimulated active development of new area of computer science - detection of knowledge in databases (KDD - Knowledge Discovery in Databases). Its basic assignment is automated search earlier unknown laws in databases storing the information (for example, about activity of the companies), and use of the extracted knowledge during acceptance of the decisions. With the help of methods KDD it is possible to reveal, for example, structure of the consumers of the given goods to prevent frauds with credit cards or to predict change of a situation in the financial market. The basic task of systems KDD is the realization of the analysis of the data contained in databases, with the purpose of detection of the latent or unobvious laws. Allocate five standard types of laws, which allow to reveal automatic intellectual methods of the analysis and representation of the data: association, sequence, classification, clustering and forecasting. The means of detection of knowledge in databases basically use five methods: an induction of associative rules; trees of the decisions; K - nearest neighbours; neural networks; genetic algorithms. Their combination is sometimes applied. The report is devoted to the decision of two tasks KDD, detection of associations and forecasting, by artificial neural networks (ANNs).

The analysis of Data Base by Hopfield neural network.

Data Analysis in the given work is understood as a complex of the measures directed on revealing of latent laws in databases. Use of the artificial neural networks for the decision of a task is considered which within the framework of tasks of the automated intellectual analysis of the data can be treated as follows.

1.  The qualitative solving of a task of find of values of fields of DB, associated to other fields with known values.

2.  The solving of a task of forecasting of values of a set of fields of DB with the given values of other set of fields.

Having system similar to associative memory, and also formal rules of carry of information contents of an analyzed database in this memory, it is possible to solve this task. Thus is used the association nature of such system and on values of some factor as "key" are taken is associative close values of other factors. One of possible ways to realize of such associative memory is to construct the distributed dynamic system or network of discrete elements with attractors as the typical patterns-images, contained in DB [1]. Each such pattern will have the area of an attraction, and any entry condition representing any allowable pattorn, is obliged to get in one of its areas of an attraction. With current of time during evolution this initial structure is transformed to the closest of structures - attractors, stored in memory, namely in that, it belonged to area of which attraction. Hence, submitting on an input as the entry condition for such distributed system some structure, we shall carry out its automatic recognition, which will be parallel. In a role of such distributed dynamic system it is offered to use artificial neural network.

The decision, offered us, in a general view looks as follows. Artificial neural network is trained on records of an analyzed relational database. During training neural network becomes gnoseological model of the DB. The received thus model the user can use for forecasting and research of associative connections latent in a database. As such model it is offered to use a network representing particular realization of dynamic system with associative memory, proposed by Hopfield [2-4].

The primary preparation given for the decision of the put task consists in representation of set of the factors as the contents of a relational database. The structural relations are thus set and the task is reduced to definition of semantic affinity of contents of fields of the given structure. As binary artificial neural network operate with binary vectors, the additional processing initial given is required with the purpose of reduction to a kind represented by such vector.

The vector needed for artificial neural network turns out the concatination of binary subvectors representing the fields of a relational database. Each field of a relational database can be referred to one of two types: numerical or symbolical. The value of a symbolical field is brought in the dictionary, appropriate each field. To raise efficiency of procedures of search of value in the dictionary and their coding, contents of the dictionaries are ranging, the repeating values are excluded. The value of numerical fields are represented differently. A range of value of each of numerical fields (to numerical fields those concern also, which contain dates) is broken into intervals, and each single value is represented by an interval, by which it belongs. The point on a numerical axis can be presented by a degeneration interval. The replacement of numerical value with intervals provides large flexibility of representation. Analogue of the dictionaries for numerical fields is the list of intervals. After the dictionaries and sets of intervals were generated, they are exposed to coding. That is each value in the dictionary or interval in a set the binary code representing subsequently this value or an interval in neural network is put in conformity. The dimension of a code (length of a code) for each dictionary and set (in other words for each field) is defined by capacity of set of various values in the dictionary or set, and also way of coding. The various variants of coding can essentially have an effect for quality of work of associative memory submitted by Hopfield neural network. The examples to that can be served by experimental data received as a result of model modeling during preparation of the given work. The formation of the dictionaries and their coding is a preparatory grade level of neural network. Training is carried out according to a Hebb rule [5].

Each record of a database is represented as a binary vector and participates in formation of weight factors of connections between formal neurons of a network. It is desirable to remove from training repeating records of a database. As the repeated record of a vector in associative memory submitted by neural network adverse has an effect for a power landscape of this network. Namely, the depth of a power minimum appropriate to such vector is doubled. The deeper minimum usually has wider area of an attraction, and it reduces areas of an attraction of the next minima and deforms the appropriate drawing vectors. In a limiting case it can result that in system there will be only one very deep minimum of energy, the area of which attraction grasps all possible spin configurations. At system the persuasive idea " is formed as though ", that, certainly, it is extremely undesirable.

There are various versions of Hopfield networks with similar structure, but some differences are in operation of them. Here are considered only binary condition neuron: + 1 and -1) Hopfield networks with discrete functioning in time. Specifity uniting such network is asynchronous of operation of them. That is as against synchronous functioning, with which the condition all neurons of a network is defined simultaneously, the asynchronous functioning means in each particular moment of time an opportunity of switching only one neuron. For modeling we deal with four algorithms of operation of Hopfield's neural network:

1.  The determined algorithm;

2.  Stochastic algorithm;

3.  The determined algorithm with annealing imitation;

4.  Stochastic algorithm with annealing imitation.

The process of operation of neural network is not depended from mentioning algorithms is multiiterative. Each of iterations includes two steps: a choice of the neuron-candidate, formation of a condition of the chosen neuron-candidate. The difference of stochastic functioning from determined consists in a method of a choice of the neuron-candidate. With stochastic - the candidate for switching is neuron chosen casually with the help of the random-number generator. Thus, such situations are possible, that during operation the condition of any neurons may be never analyzed. With the determined operation neurons become the candidates for change of the condition by way of following numbers. In algorithms with annealing imitation the change of a condition of the neuron-candidate carries probabilistic character. If as a result of change of a condition on opposite the complete energy of a network will be lowered, the condition of neuron varies on opposite. Differently neuron's condition varies on opposite with the certain probability dependent on parameter, named in "temperature" of a network simulating a level of thermal noise. On the first iteration of operation of the neural network the initial value of temperature (maximal value) is established, then gradually through some of iterations the value of temperature decreases. The iterative process stops, when temperature reaches final value (minimal value). If to compare among themselves algorithms without and with annealing imitation, at the first set of advantages before second (it is easier, require smaller amount of iterations, have greater accuracy - if the power landscape near to a global minimum is not cut up by local minima), but if the power landscape contains set of local minima, the algorithms with annealing imitation behave better (from the point of view of criterion of quality of functioning). The confirmation of last statement can be served by the data of model modeling. If to consider operation of system as a whole, it includes the following stages:

1.  the formation of inquiry,

2.  the preparation of an input vector of a network,

3.  the cycles of functioning of the neural network,

4.  the formation of a target vector.

Let's consider each stage.

The Formation of inquiry. At this stage the user chooses a part of fields DB and gives them some values (using the dictionaries, appropriate to this fields, of values and sets of intervals). During functioning the system will predict values of other fields (distinct from set by the user) similarly to associative memory.

The preparation of an input vector of a neural network. At this stage the input vector is formed for the neural network. It "is going" from subvectors, appropriate to the fields of Data Base. With formation of a vector the zero categories subvectors are replaced on (-1). Those subvectors, whose value sets the user, are marked as "frozen". The values of other subvectors are formed with use of the random-number generator.

The cycles of operation of a neural network. This stage includes a number of cycles of operation of a neural network. Before each cycle on the neural network moves an input vector generated at the previous stage of operation. Each cycle of operation of neural network consists of repeated iterations, thus each iteration in turn includes two above described stages (choice of the neuron-candidate, formation of a condition of the chosen neuron-candidate). It is necessary to notice neurons that appropriate "frozen" subvectors, cannot be chosen as the neuron-candidate, and, hence, they do not change the condition during all functioning. On end of the first cycle received by a neural network a vector and the value of energy of a network is remembered in the special buffer. At the end of each subsequent cycle the value of energy of a network is compared with value stored in buffer. If the current value of energy is less then in the buffer, a current condition of a network (the condition of a network represents a vector in quality which component the condition neurons act) replaces in the buffer earlier stored vector.

The formation of a target vector. It is a formal stage, when a vector kept in the buffer is given out by system as the vector - answer restored by associative memory by a fragment, given by the user.

Conclusions

To the basic advantages of the approach, proposed here, it is possible to relate the following.

·  Organic association of the decisions of such tasks as forecasting and detection of associations within the framework of the uniform approach.

·  The training of neural network with the purpose of the subsequent modeling requires only two continuous passes on a simulated database. While with the traditional approach, namely with accumulation statistics about combinations of values contained in DB, the multiiterative complex procedures of search are required.

·  The decision of a task in those situations, when the traditional approach appears statistically insolvent. For example, when analyzed DB contains small number of similar records.

·  Universality of Hopfield network, used as a basis of construction of associative memory.

·  Simplicity of program realization of Hopfield model.

This approach was realized in the program of Data Base Analysis (AnalDB). There the SQL-query is the source of data for learning of the neural network in this program. This program is realized on Delphi 3.0 and Cbuilder 3.0. It may be used for find of most probable values of any fields in context determined by other fields, and for forecasting of fields of also unexisting record (if the records are the patterns of process in time). This program is used for research of the charge of water in river Ob for last 104 years with the purpose of forecasting inflow of water and research of a demographic situation in one district of Novosibirsk. This work is supported by the program "Universities of Russia - fundamental researches" and grant of Russian Federation on a direction "Computer Science and cybernetics".