Artificial Neural Network Project
(Graduate Study University Wisconsin Madison 539)
By Jeff Schmidt
815-226-6410
Hamilton-Sundstrand
May 17 2001
ABSTRACT
When developing software for flight worthiness, the FAA requires that software must go through a life cycle of requirements, design, implementation and verification. For most Avionic programs, the verification phase has become one of the most expensive phases. The implementation phase is where the code is developed. Most software used in Avionic Equipment is written using Boolean logic and if statements for conditional decisions. To test the code to verify it has been developed correctly, tests are done in 2 steps. The first is the system test to verify that the software and hardware perform to the requirements created for the system. The second suite of tests is on the code to exorcize each Boolean logic equation, as well as all the possible paths the conditional decisions provide. At times this must be done at the object code and not the source level. This project will create an Artificial Neural Network (ANN) engine, which will run all the ANNs created. Since the ANNs created are strictly data structures, no executable code is create except for the engine. This will only require the ANN engine to be part of the second, low level tests. Since there is only data structures created, the creation of code for the implementation is reduced as well. The ANNs created in the project will be trained to replace convention software and must be 100% accurate.
- Table of Contents
- ABSTRACT 1
- Problem Statement 3
- Method/Setup 4
- Results 12
- Further Study. 15
- Proposal 17
Problem Statement
The proposal for this project has been attached as appendix A.In order to have ANNs replace convention software in certifiable equipment, two areas of concern must be overcome. The first is to have the ANN that is going to be placed in a safety critical box must be developed to 100% accuracy to its requirements. The ANN must be able to handle many different types of mappings such as time-delays, analog input boundary decisions, input relations, and previous states of the ANN. Some of these mappings are used in this project. The ANNs created must be able to execute in a real-time environment. With the latest embedded processors now providing multiplications and floating point arithmetic in a few clock cycles along with the processor’s cache, real-time execution is achievable. The second area of concern is how to trace the requirements to trained ANN. This process must be provided in an intuitive way to allow many engineers to develop a system at the same time.
Although the proposal stated the project would create a GUI simulator to run the ANN control software against the convention software, a second GUI interface to provide an engineer with an environment to provide requirements to the ANN and train it. The second GUI interface allows the second area of concern to be addressed. All the requirements are provided to create the ANN.
This project will take the requirements for the proposal and create conventional software to run in the simulator as well as the ANNs to control the power quality of an aircraft. As stated in the proposal, a generator is attached to an airplane engine and the generator control unit (GCU) is used to control the generator to provide good quality power for the aircraft. The normal parameters are 400Hz and 115V.
The first goal is to create an ANN, which will control the power quality according to the requirements of GCU #1, section 6 of the proposal. This ANN will be trained to handle 100% of these requirements.
The second goal is to create additional requirement for GCU#1 such that no executable code is added, deleted, or changed. The new requirements are adding hysteresis to the frequency and voltage. By adding hysteresis, the ANN now must use its own output as one of its inputs, which not gives it a mapping of its previous state.
The third goal is to produce a second GCU controller with different requirements without creating any new executable code.
Method/Setup
The tools used for this project were created using Borland C++ builder environment. Two GUIs were created to help in developing the ANNs required and run a simulation of the control. The following files were used for this project:Project.exe / Program containing the two GUIs.
gcu1mann_err.txt / File created using the Neuron Simulator GUI to contain the requirements for Generator Control Unit 1GCU1. The under speed threshold was set in error to be 0-3000 instead of 0-2999. This file is used to show an error when comparing the conventional software to the ANN logic.
Gcu1mann_err.ann / Trained network of ANNs for GCU1 with the under speed error.
Gcu1mann.txt / File created using the Neuron Simulator GUI to contain the requirements for Generator Control Unit 1GCU1.
Gcu1mann.ann / Trained network of ANNs for GCU1
Gcu2mann.txt / File created using the Neuron Simulator GUI to contain the requirements for GCU2.
Gcu2mann.ann / Trained network of ANNs for GCU2
Gcu1cann.txt / File created using the Neuron Simulator GUI to contain the requirements for GCU1 with hysterisis added to the frequency and voltage levels. This requires the output of the top-level neuron to be fed back into its inputs. Uses a previous state.
Gcu1cann.ann / Trained network of ANNs for GCU1 with hysteresis.
The implementation of the ANN to run the controller was broken down into several expert ANNs and one top level ANN providing the final output. The architecture is shown below.
FIGURE A.
The top-level ANN was designed as a Multi-Layered Perceptron[1] using one hidden layer and one output layer. Each neuron used an activation function of a sigmoidal function. The number of hidden neurons can be adjusted. This was to allow the network to be trained using the back-propagation algorithm to a supervised set of samples. The inputs to the top level ANN are all discrete inputs in order to bound the number of input samples required to train the network. To allow for analog data as part of the control, expert threshold networks were created to provide input into the top-level neuron.
The back-propagation algorithm for calculation of weights [2]:
Weigh Learning Local Input signal of
Correction rate Gradient neuron j
Connect
Neuron i to j
Dwij(n) = h * dj(n) * yi(n)
The local gradient would change based on whether the neuron is hidden [3] or the neuron is an output [4]. The forward pass algorithm to calculate the neuron output [5]:
Output Activation function (Summation of weights * previous layer inputs)
yj(n) = j(å wij(n) * yi(n))
j() Activation function.
The threshold ANNs were designed as a Multi-Layered Perceptron using one hidden layer and one output layer. Each neuron used an activation function of a threshold function. There are two hidden neurons that sample the same input. The weights of a threshold ANN are calculated from the required thresholds. One hidden neuron is calculated to respond to the low threshold and one hidden neuron is calculated to respond to the high threshold. The threshold ANN is shown below.
FIGURE B.
The controller network is based on the top-level ANN that in turn gets its inputs form either the threshold ANNs or discrete inputs. A visual presentation of the top-level ANN and its inputs can be seen in Fig A.
The process to create the required networks uses the Neuron Editor program. Although not part of the original project, the Neuron Editor was created to help architect the neuron layout. This program was designed to prototype a process in which a designer can enter all the inputs, requirements and neuron network parameters. The Neuron Editor is not a commercial quality product but does allow for simple editing of the controller network.
FIGURE C.Edit Neuron GUI Overview
The following is not intended to show how to create a layout from scratch but to display an existing one. To view a layout of a complete network, go to the Neuron Editor window and enter in “gcu1mann”, without quotes, located at the top right corner of the window. Then hit the Load Network button.- At this point under the Inputs tab, all the inputs for the simulation are presented. A range form 0 to 1 indicates that the input is a discrete input.
- The Levels tab shows all the thresholds ANNs created for this controller network. With a mouse click on a listed Level Neurons will display its required thresholds.
- The Top Level tab displays 3 steps in setting up the top neuron. Going to Step 2 shows the requirements that the top-level ANN will train with. A cell with a “1” indicates that the input is required to be a “1” while a “0” indicates the input shall be a “0”. An “X” is showing that this input is a don’t care input. The end column holds the required output for the controller network. This layout has 7 requirements with only one of them turning the output relay on. Step 3 is used to show the layout of the top-level ANN. Here the number of hidden neurons can be selected.
- The Train tab is used to create the training samples from the requirements table. These training samples can be seen in a file tmp.txt after the Create Training Samples button is selected. The file holds a column for each input and one for the output. The last column is used as a tag to remove duplicated samples. After the samples are created, the top-level neuron is ready for training by selecting the Training button. This will train the top-level neuron and create a file gcu1mann.ann, which is then used by the simulator program. The file created contains all the weights for all the neurons used in the network. All files have been created to be view with an editor.
Loading Simulator with ANNs.
To load a new file into the simulator, follow the steps below:
- Select the NO ANN radio button to stop any networks from running
- Type in the file to be loaded in the ANN FILE edit box, top right in GUI. (do not add extension)
- Select the LOAD ANN button
Conventional Software
The conventional software logic was created using the requirements in the proposal, appendix A. Each Generator Controller has its own code, shown below, and is selectable on the Simulator.
Common code for both GCUs
#include "common.h"
//------
// Return if under threshold
U16 UnderVoltage(U16 voltage, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(voltage < threshold)
returnCondition = 1;
return(returnCondition);
}
//------
// Return if over threshold
U16 OverVoltage(U16 voltage, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(voltage > threshold)
returnCondition = 1;
return(returnCondition);
}
//------
// Return if under threshold
U16 UnderFrequency(U16 frequency, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(frequency < threshold)
returnCondition = 1;
return(returnCondition);
}
//------
// Return if over threshold
U16 OverFrequency(U16 frequency, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(frequency > threshold)
returnCondition = 1;
return(returnCondition);
}
//------
// Return if over threshold
U16 OverCurrent(U16 current, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(current > threshold)
returnCondition = 1;
return(returnCondition);
}
//------
// Return if under threshold
U16 UnderSpeed(U16 speed, U16 threshold)
{ //Check parameter to threshold
U16 returnCondition = 0;
if(speed < threshold)
returnCondition = 1;
return(returnCondition);
}
GCU1 code
U16 GCU1_SW(U16 frequency, U16 voltage, U16 speed, U16 current, U16 switchValue)
{
//local variables
static U16 closeRelay;
static U16 UV,OV,OC,OF,UF,US;
//Check Voltage
UV = UnderVoltage(voltage, 90);
OV = OverVoltage(voltage, 130);
UF = UnderFrequency(frequency, 370);
OF = OverFrequency(frequency, 430);
OC = OverCurrent(current, 7);
US = UnderSpeed(speed, 3000);
//Close relay only when power is in the good quality zone
if(((UV || OV || UF || OF || OC || US) == 0) & switchValue)
closeRelay = 1;
else
closeRelay = 0;
return(closeRelay);
}
GCU2 Software
U16 GCU2_SW(U16 frequency, U16 voltage, U16 speed, U16 current, U16 switchValue )
{
//local variables
static U16 closeRelay;
static U16 UV,OV,OC,OF,UF,US, USS, UFF;
//Check Voltage
UV = UnderVoltage(voltage, 90);
OV = OverVoltage(voltage, 130);
UF = UnderFrequency(frequency, 380);
OF = OverFrequency(frequency, 420);
OC = OverCurrent(current, 9);
US = UnderSpeed(speed, 3000);
USS = UnderSpeed(speed, 100);
UFF = UnderFrequency(frequency, 350);
if(USS & (UFF == 0))
{// The input speed wire is broken
US = 0;//No underspeed
}
//Close relay only when power is in the good quality zone
if(((UV || OV || UF || OF || OC || US) == 0) & switchValue)
closeRelay = 1;
else
closeRelay = 0;
return(closeRelay);
}
Results
To show the goals have been met to the requirements of the controllers, test procedures have been used.
Goals
- To create an ANN that can replace the conventional software for the contactor control in a GCU. This requires the ANN software to learn 100% of the contractor functional requirements.
- To create an ANN algorithm such that the code to calculate the ANN output does not change if the contactor requirements change. It is ok for the inputs to have their thresholds change or new inputs, this should not change the algorithm required to calculate the output.
- To create an generic ANN algorithm such that this network will run the contactor control requirements from 2 different GCUs by only changing the ANN data. This would consist of changing only the ANN weights, again no code change.
- To validate the ANN data to the requirements using mathematics/graphs. Use of Matlab can provide a visual display.
The results form the 1st three goals have been met with success. The 4th goal to have the create ANN be verified with graphs can now be done for each threshold ANN. Since each threshold ANN has only one input the graph would show the same result as the level requirements used to create it.
Goal 1 success
The first goal of creating an ANN to be trained to meet 100% of the requirements from logic developed using conventional software. The requirements were taken from GCU 1 of the proposal. The first test loads file gcu1mann_err, which was trained using the Neuron Editor GUI, into the simulator GUI. The following test was used to verify goal one was met.