Enhancement of CVIP-FEPC and Color Normalization Program

An

Independent Study Report

on

Enhancement of CVIP-FEPC and Color Normalization Program

Independent Study

(ECE 591)

SUBMITTED BY

Kumari Heema Poudel(800517296)

December 12, 2013

1

Abstract

Image analysis is an important part of Digital Image Processing which comprises of computer vision and human vision applications.Usually, feature analysis and pattern classification are the final and significant steps in image analysis. The Computer Vision and Image Processing Feature Extraction and Pattern Classification Tool, CVIP-FEPC, was developed for exploring both human and computer vision applications. The main function of this tool is to analyze feature extraction and pattern classification process. This enables the user to perform batch processing with large sets of image fileswhich is much more efficient than processing one image at a time.This toolallows the user to load multiple images and run experiments based on different parameters by specifying the classes, selecting the features, selecting the test set, choosing the pattern classification parameters and then letting the program to process on the entire image set. Result files are stored in the output folder. One of the main objectives ofthis independent study is to develop the next stable version of CVIP-FEPC that includesfixing the bugs present in current CVIP-FEPC and to enhance the tool by adding more features.

Another main focus of this study is on the Color Normalization Program, which is used to do a remapping to the color values of thermographic images so that all of the images are mapped to a common temperature scale. Graphical User Interface (GUI) has been added to theColor Normalization program to make it more user friendly and a mechanism has been developed to organizethe generated output files.

Acknowledgement

I would like to express my deepest appreciation to Dr. Scott E. Umbaugh for his continuous guidance and encouragement throughoutthis independent study.Without his great supervision, constant help and valuable suggestions this study would not have been fruitful.

Additionally I wish to thank Samrat Subedi and Jiyuan Fu; members of Computer Vision and Image Processing Lab, Southern Illinois University Edwardsville for their help and support in testing the application with the experiment on real images. I would like to thank Long Island Veterinary Specialists Dr. Dominic J. Marino and Dr. Catherine A. Loughin for providing images which have been used in testing the application. Last but not the least I want to thank Ravneet Kaur and Krishna Regmi for their help.

Kumari Heema Poudel

Table of Contents

Abstract

Acknowledgement

Table of Contents

List of Figures

1.Introduction

2.Materials and Methods

2.1Development Environment

2.2Color Normalization

2.3CVIP-FEPC

3.Overall Flow of FEPC

4.Known Bugs

5.Conclusion

6.Future Work

7.Appendices

7.1Color Normalization

7.2Screenshots

8.References

List of Figures

Figure 2.1: Color Normalization GUI and help file………………………………………………………...4

Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test……………………...5

Figure 2.3: Help File of FEPC……………………………………………………………………………...6

Figure 2.4: Feature Selection screen with Variable Texture Distance……………………………………...7

Figure 4.1: Exception occurred while using the different feature category in second experiment……….11

Figure 7.1: Original Thermographic image and their normalized equivalents…………………………....14

Figure 7.2: Main window of CVIP-FEPC………………………………………………………………..14

Figure 7.3: Pattern Classification window of CVIP-FEPC ……………………………………………...15

Figure 7.4: Class List window of CVIP-FEPC …………………………………………………………15

1

1.Introduction

This study is about the further development on Color Normalization program and CVIP–FEPC.The CVIP–FEPC tool is extensively being usedfor the project ‘Veterinary Thermographic Image Analysis’ from Long Island Veterinary Specialists, in which the feature analysis and pattern classification is done for cats’ and dogs’ thermographic images to classify the cats and dogs for normal and abnormal conditions of bone cancer, hyperthyroid and the gate analysis.

CVIP-FEPC is not limited for the analysis of thermographic images only. This can be used for the analysis of any imagetypes. One area of concern may be to run the experiments with multiple texture distances at the same time.In previousversionof CVIP-FEPC, this feature of variable texture distance was not available.So, this is the one feature that has been added in the CVIP-FEPC.A help file is always very useful for any application to make the user familiar with the tool and this has been added in the current CVIP-FEPC.

Other than FEPC, work has been done on Color Normalization program which was developed for the ‘Veterinary Thermographic Image Analysis’ to normalize the temperature range according to the color of the thermographic image. A graphical user interface has been developed to make the program user friendly and some other features have been added to mitigate the manual work to manage the output files in folders.The new version of Color Normalization program automatically creates the folders and keeps all the output files to their respective folders.

This report will describe the changes made in FEPC to the program detail and their reflection in GUI together with the changes in Color Normalization program.

2.Materials and Methods

2.1Development Environment

CVIP-FEPC was developed with C#.NET programminglanguage using Visual Studio 2008.The further enhancements are done using the same programming language but with the Visual Studio 2010 and the CVIP-FEPC is converted to C#.NET 2010. The following FEPC files are modified to achieve the variable Texture Distance feature: MainForm.cs, FeatureHeader.cs, FeatureChooser.cs, FeatureVector.cs, and ImageObject.cs.

Color Normalizationprogram has been tested with the thermographic images provided by Long Island Veterinary Specialists taken with a MedithermMed2000IRIS. The unit testing is done by the developer whereas thefunctional testingis done by the researchers of the Veterinary Thermographic Image Analysis.

Similarly for the FEPC features, unit and functionaltesting have been done with small set of images by the developer, whereas researcher performed the functionaltesting on the real set of thermographic images.

2.2Color Normalization

The research project undergoing in SIUE, ‘Veterinary Thermographic Image Analysis’, uses thermographic images of cats and dogs to classify them into different categories like normal and abnormal. Within any thermographic image, each color represents a specific temperature. But in different images, the same color may represent different temperatures, which can introduce noise in the pattern classification process. To overcome this, a remapping is done to ensure that all the images are mapped to a common temperature scale which is done in Color Normalizationprogram. More about Color Normalization methods can be found in the appendix [1].

2.2.1Output Folder Structuring

In the previous version of Color Normalization, all the output normalized files were stored at the same folder as input.It was a tedious task to manually manage the folders for placing normalized output files and their masks for each category together. Now this version of the program makes this task easier by automatically managingthe organization of file.

Following foldersare created automatically at the location where input files are located:

• orig: All the input files are kept here together with the Mask folder.
• lum: All lum normalized files areplaced here concatenating ‘–lum’ in the original file name.Similarly, mask filenamesare also changed by concatenating ‘-lum’ in mask file name and these mask files arealso copied together with the Mask folder within the Lum folder.
• normGrey: All normGrey normalized files are kept here with concatenating ‘–normGrey’ in the original file name. Similarly mask files name are also changed by concatenating ‘-normGrey’ in mask file name and these mask files are also copied together with the Mask folder within the normGrey folder.
• normRGB: All normRGB normalized files are kept here with concatenating ‘–normRGB’ in the original file name. Similarly mask files name are also changed by concatenating ‘-normRGB’ in mask file name and these mask files are also copied together with the Mask folder within the LnormRGB-lum folder.
• normRGB-lum: All normRGB-lum normalized files will be kept here with concatenating ‘–normRGB’ in the original file name.Similarly mask files name are also changed by concatenating ‘-normRGB-lum’ in mask file name and these mask files are also copied together with the Mask folder within the normRGB-lum folder.

2.2.2GUI based Color Normalization

A simple GUI has been designed to make it user friendly.This GUI will load a *.txt file which contains the path of the images followed by maximum and minimum temperatures of that image.The exact file information is available in Help file of the application as shown in Figure 2.1.

Figure2.1: Color Normalization GUI and help file

2.3CVIP-FEPC

CVIP - FEPC is composed of four graphical interfaces: the Main window, the Feature Selection window,the Pattern Classification Selection window and the Class List window.The main interface is the Main window which is seen when the application is executed.The Feature Selection window appears when the user clicks on Features button of Main window and Pattern Classification window appears when the Classification button of the Main window is clicked.There is a setting menu in the main window which directs to the Class List window.

The following works have been done in CVIP-FEPC:

2.3.1 Bug Fixing

The previous version of CVIP-FEPC was not working forLeave One Out method with Combinatoric test.The following error was encountered when the user tried to execute CVIP-FEPC with the Leave one out method and Combinatoric test, which is shown in Figure 2.2.

Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test

After debugging, it was found that the training and test classes were being used without initializing them, which was causing the error of “Object reference not set to an instance of an object”. For using any method of a class, the class needs to be initialized and this was the missing part. After writing the codes for initializing the class and calling its function in an appropriate manner, issue was fixed.

2.3.2HelpFile

A help file has been added in the FEPC to make it easier to use and to havea betterunderstanding of the software.The help file is basically a Compiled HTML file (CHM file) which is designed and compiled using ‘HTML Help Workshop’ software. Snapshot for the help file is shown in Figure 2.3 below.

Figure 2.3: Help Fileof FEPC

2.3.3Variable Texture Distance:

In the current version of FEPC, we havethe option of selecting one texture distance at a time.This means any experiment even if it is combinatoric, it will run with only one texture distance.What if a user wants to analyze the output of experiment with several texture distances with the same feature set? For this, the user has to run the experiment several times by passing the texture distance of choice.A better optionis to incorporate the choice of selecting multiple texture distances in FEPC itself.

With the option of selecting variable texture distance, the feature selection window looks like as shown in Figure2.4.

Figure2.4: Feature Selection screen with Variable Texture Distance

This variable texture distance feature is available only if the experiment is to be run for ‘Combinatoric’ option and not for the ‘Single Test’.There is a button Switch to Single Test Mode, which can be selected if the user wants to run Single Test experiment only.

With the addition of this new feature the first file that gets affected is MainForm.cs. In this file the program checkswhether the user has selected the variable texture option or not. If the multiple texture distance is selected, the program calculates thedifferent texture distances (d1, d2,…dn) and the experiment runs with all the texture distances.

The second file that getsaffected is FeatureDlg.cs. This is the interface for feature selection and three boxes have been added here which are the parameters for variable texture distance:

1. Lower: This is the first texture distance value with which the experiment will start its execution.
2. Increment:This is the increment on the lower texture distance.
3. Upper:This is the final limit of the texture distance.

To illustrate this process, let us take an example where lower is set as 2, increment is set as 2 and upper is set as 8.

Then formula used to find the iteration is

N= ((Upper –Lower)/Increment)+1

D[1]=lower;

Where n varies from 2 to N

So in our example N= ((8-2)/2)+1=4

So d1=lower=2

D2=d1+increment=2+2=4

D3=d2+increment =4+2=6

D4=d3+increment=6+2=8

So the experiment will run with these 4 texture distances along with all other features and classification algorithms selected at the beginning.

The output files of the experiment will be saved into fourfolders, with four different texture distances.There can be cases where the user wants to run the combinatoric experiments but want to select just one texture distance. For this, the desired texture distance need to be typed in lower box and upper box should have the same value.Value entered in the increment box does not matter here.

FeatureChooser.csand FeatureHeader.csare the files where output files are being written.So,appropriate texture distance is being passed to be written in the output files in case of multiple texture distances.

FeatureVector.cs and ImageObject.cs are the files which calculate the feature vectors, so proper distance is being passed while calculating the texture features for the variable texture distance case.

3.Overall Flow of FEPC

The Main Window (MainForm.cs) appears when a user runs the application. In this window, the user can add multiple images at the same time. Now the user can select the features by clicking on Features button and this is the point where program control goes over to FeatureDlg.cs which is the interface for feature selection window. All the binary, histogram, RST invariant, texture and spectral features are listed here which can be selected.

After selecting the features user can set the parameters: data normalization methods, distance and similarity measures and classification algorithm for pattern classification. If combinatoric test has been selected, the user can choose multiple methods for every category otherwise the user can select only one method for each category. After selecting the classification methods, the user will classify the images to any classes like normal and abnormal and hit the button Run Test.

MainForm.cs is the file where all the set up for running the experiment gets done.This file handles the deleting of images (response of Delete button in the same window) and changing the output location (action of clicking on Browse button). FeatureVector.cs is the file which calculates all the feature vectors. Normalizer.cs is the file where all the calculations regarding data normalizations are done. FeatureHeader.cs is the file which generates all the text output feature files. FeatureFile.cs is used for getting the feature files which are to be written in the output directory. This file checks if the user has selected any of the normalization method, if yes this file directs the program control to do the normalization and then get the feature file otherwise will get the feature file without any normalizationbeing done. FeatureChooser.cs file checks for the features which are selected by the users and calculates the number of combinations with the number of features selected. For any combinatoric test, the total number of combinations will be: 2n, where n is the total number of the features and pattern classification methods selected for each texture distance. TheClassifier.csfilechecks for the classification methods being selected by the user and recalculates the number of combinations since the number of features is increased with the selection of classification methods and thusprepares the content for the output file.

4.Known Bugs

A bug was encountered in the previous version of CVIP-FEPC. If the user runs the application for the first time and has selected a few categories of the features, then the application will work perfectly, but if the user tries to run another experiment without restarting the application and selects the features of the categories not selected previously then the error will pop up.

For example, if the user has selected features from binary features, histogram and texture features categories in the first experiment and runs the experiment on these feature set, the experiment runs successfully.But now with the addition of any feature from the RST-invariant categoryinto the feature set and running the experiment will producean error. The error message is shown in the snapshot in Figure 4.1 below.This was noticed at the later stage of implementing the variable texture distance, so this still exists in the current application.

Figure 4.1: Exception occurred while using the different feature category in second experiment

5.Conclusion

All the bugs except for the one mentioned in the Known Bugs section are successfully fixed. The major concern was the bug which was preventing the use of Leave One Out method with Combinatoric Test, which is very important functionality of CVIP-FEPC, is now fixed. The newly added help file includes information about overview of CVIP-FEPC and how to use it. The variable texture distancefeature is also working successfully for both Training and Test Set and Leave One Out, except of the bug mentioned above.