Andrija Krtalic, Milan Bajic
Faculty of Geodesy University of Zagreb, Croatia
Masa Stojcic
Geofoto, Zagreb, Croatia
Keywords: long-wave IR, visible and near IR, spatial resolution, criteria, fusion
ABSTRACT: Recent research in projects ARC and SMART (European Commission IST programme) shows that long wave thermal IR images enable detection of the objects that are otherwise not detectable. However, the important conclusion from this research is that the spatial resolution of available thermal IR cameras is not satisfactory. The possible way to compensate partially this deficiency can be data fusion. Due to the experience gained and the lessons learned in many years of the airborne remote sensing with thermal IR cameras we decided to analyze the fusion of the thermal IR images with images from visible and near IR wavelengths. The first goal was improvement of spatial resolution (sharpening) and this has been achieved. More important goal was the evaluation the radiometric quality of the fusion due to different physical background of the images. For this purpose many statistical parameters of different objects in the source thermal IR image were analysed and compared to the respective statistical parameters set for the same objects in the image obtained by fusion. The objects used in the analysis selected by use of the ground truth data that were available for the identification. The thermal IR contrast C between object “o” and background “b” used for comparison of the fused and source images too. The results of the analysis approved that the applied fusion method provides improvement of the spatial resolution and that thermal information of the analyzed objects conserved. Due to approved quality of the fusion method, we recommend its application.
1 introduction
Digital system for airborne image acquisition collect data about some scene with two cameras: DuncanTech MS3100 (MS3100) and Thermovision®1000 (THV1000) [M. Bajic, 2003]. The first camera collects data in four wavelengths: three visible (V: B – blue, G – green, R – red) and one near infrared (NIR: IR) part of electromagnetic spectra (together: VNIR)) [DuncanTech, MS3100]. The second camera collects data in long wave thermal infrared range (TIR: 8-12 μm) [AGEMA, 1993]. Beside differences in sensors and wave lengths there are also differences in way of genesis of images. MS3100 is matrix camera with sensor resolution of 1392x1040 pixels [DuncanTech, MS3100] and developing image in manner of central projection. THV1000 is collecting thermal radiance with parallel linear scanning, and sensor resolution is 600x400 pixels. The images used in this paper have been collected with relative height of 500 m above the region of Velebit, Tulove grede. The idea of this paper is to improve the spatial resolution of TIR image collected from 500 m with spatial resolution of VNIR channels also collected from 500 m. For this purpose the fusion of image on pixel basis was done. The image fusion was done by Principal Components Analysis (PCA) [Richards, 1986].
2 RESOLUTION OF IMAGES
Airborne data acquisition with MS3100 was done with focal length of 24 mm, and the field of view (FOV) was 17°.99 x 14°.72. Sensor resolution is 1392x1040 pixels. FOV for lens on THV1000 was 20°x13°.33, and sensor resolution is 60x400 pixels. Based on this information the size of pixels was calculated for digital VNIR (≈0.12 m) and TIR images (≈0.30 m) [Krtalic, 2006]. These calculations of pixel sizes refer to stationary data acquisition (acquisition from stationary platform 500 m above terrain).
3 REGISTRATION AND CUTTING OF IMAGES
Images witch cover the same area as big as possible was chosen from a set of images collected with both cameras from relative height of 500 m. With this action the set of images (figure 1a, 1b) for image fusion process was created. But, before comparison and combining of multisensors and multiresolutions images has taken place the registration in the same reference system most be done. In other words, geometrical and radio metrical transformations need to be done within a set of images. VNIR image was chosen as reference image because better resolution and TIR image was registered and resampled on it [Krtalic, 2006].
a b Figure 1. a) Original VNIR (RGB) image, b) original TIR image
3.1 Geometrical transformation
The precision in geometrical transformation is mandatory because of the fusion on images which was done on pixel basis. That was the reason for using Thin Plate Spline transformation [URL1], within software package Descartes [Bentley, 2000], for geometrical transformation. This transformation is insuring zero deviation on reference points. So, the registered points on product of fusion in transformation process get exactly the same value of reference points (the same points on VNIR and TIR images) as the original VNIR image. That is the characteristic of this method of transformation. Between these reference points the interpolation was carried out.
3.2 Radiometric transformation
The original information from multispectral images must be preserved as much as possible. Because of that radiometric transformation in preprocessing (before fusion process) must be used as little as possible. However, registration of images is mandatory for image fusion on pixel basis. Within the frame of this action, the resampling of original TIR image was unavoidable. Radiometric transformation was done by Nearest Neighbor transformation [Castelman, 1996] within software package Descartes [Bentley, 2000]. The characteristic of this transformation method is in fact that no new spectral information in resample image has been introduced.
3.3 Cutting of images to the same size
After overlapping VNIR and TIR images (after geometrical and radiometric transformation) areas that show the same part of scene were cut and split in channels (VNIR split in B, G, R and IR). Cutting was done by means of software package Image Analyst [Intergraph, 1998]. That action has created a set of images with different physical background of the images, and different radiometric and spectral caracteristic of the same terrain. All images in that set have the same size and also the same size of pixel (≈0.12 m) (figure 2).
a b Figure 2. a) Cut original VNIR (RGB) image, b) cut original TIR image [Krtalic, 2006].
4 IMAGE FUSION
The aim of image fusion in this paper is improving spatial resolution of TIR image with spatial resolution of VNIR image but providing that thermal information of the analyzed objects must be preserved as much as possible. The product of image fusion must look like original TIR image and preserve thermal information. In this case the image fusion was done by PCA method [Richards, 1986].
The set of images for fusion in this paper consists of VNIR channels and registered and resampled TIR images with the same image resolution (figure 2). Well, the fusion of images with different physical background (reflection of visible and near infrared part of electromagnetic spectra – VNIR, long wave thermal radiation – TIR) was done. The fusion was carried out in every combination of cut images (channels) within mentioned set of images acquired from 500 m relative height above the region of Velebit, Tulove grede. The criterion for selection combinations of images for fusion was completeness of all original data, visual criterion (product with heights degree of likeness), and degree of improvement of spatial resolution of TIR image and correlation product of fusion with original TIR image.
a b c Figure 3. a) Cut original TIR image, b) pc2 of B-G-R-IR-TIR combination, c) pc2 of B-G-R-TIR combination [Krtalic, 2006].
5 ANALYSIS OF RESULTS
The aim of the analysis of resulting images is confirmation of spatial resolution improvement and conservation of original thermal information after applied fusion.
5.1 Visual and statistical criteria
The best visual result gives the second principal components from combinations B-G-R-IR-TIR and B-G-IR-TIR (figure 3) which are almost the same (visual and statistical parameters). Further more, principal components which gave the best visual results (best improvements of spatial resolution) have the smallest correlation coefficient compared with original TIR image. That leads to conclusion that this images show the scene in different ways. The differences of other statistical parameters are insignificant. The best improvements of fusion products refer to the parts of image that show rocks, bigger stones and dry stones.
5.2 Analysis of object characteristic on thermal image
The original TIR image and the best fusion product (pc2 from B-G-R-IR-TIR combination of images) are divided into target objects and reference objects. Target objects are dominant objects on the images with these characteristics. It is confirmed by means of the basic statistical quantities that make it different from the other objects. Each object has got its own thermal records being its own feature. The aim is to find out the thermal radiation of each simple object and to derive certain conclusions connected with individual objects by means of statistical values and the accompanying histograms. Reference objects are parts of image witch are in contrast relationship towards target objects. Besides the division into target and reference objects there is also basic division into natural and man made objects [Stojcic, 2006].
a b c
d e f Figure 4. Histograms of extracted objects from original TIR image (the first row) and from the best fused product (the second row): a) and d) = stones, b) and e) = dry stones, c) and f) = background areas [Stojcic, 2006].
The analysis of the best fused product was carried out in few steps. In the first step on each image (original TIR and the best fused product) were chosen target and reference objects. After that for each object basic statistical parameters were calculated: minimum and maximum value of temperature, their interrelations, mean and standard deviation within software package ThermaCAM Researcher [ThermaCAM, 2001]. The last step of this kind of analysis was the calculation of contrast value (C) between object in chosen combinations:
where there are: μ0 - mean value of target object, μb - mean value of reference object, σ0 - variances of target object, σb - variances of reference object.
a b c Figure 5. Diagrams and of standard deviations for samples: a) stones, b) dry stones, c) background areas on original TIR image (left-blue) and from the best fused product (right-red) [Stojcic, 2006].
a b c Figure 6. Diagrams of contrast value for samples: a) stones – dry stones, b) stones - background areas, c) dry stones - background areas on original TIR image (left–light green) and from the best fused product (right–dark green) [Stojcic, 2006].
After that calculation the comparison between given values on original TIR image and the best fused product was done. Three types of objects were chosen on images. They are stones, dry stones and background area (low vegetation). Within software Image Java there were minimal, maximal and mean value and standard deviation for original TIR image given and the best fused product. Contrast values were calculated for three combinations. The first one is stones – dry stones, the second is stones – background area and the third is dry stone - background area.
6 CONCLUSION
Image fusion of physically different backgrounds (images from visible (VNIR) and thermal (TIR) range) gives good results. Some fused products with better spatial resolution have preserved characteristic information of original TIR image. That was the aim of this fusion, conservation of specific thermal information because only in that case it has sense to use this products in further process of remote sensing (classification for example) for extraction of data which do not exist in other wavelengths. PCA method of image fusion was used for getting presented results in this paper. Histograms show (Figure 4) bigger mean value of samples for the best fused product than on the original TIR image. Standard deviations of sample objects are also significantly bigger on the best fused product than on the original TIR image (Figure 5). Such increase of values confirms reinforcement of variety information on the best result of fusion. Visual comparison of the fused products and original TIR image shows significant improvement in spatial resolution of the fused products according the original TIR image (Figure 3). Contours of object are clearer and differentiation of objects is better and easier. The whole fused products are sharper, nuanced and easier for interpretation and more suitable for further processing. Calculated measures of contrast value confirm these facts.
REFErences
AGEMA Infrared Systems (1993): ®1000 Operating Manual, Sept. 1993.
Bajić, M. (2003): Survey of suspected mined areas from a helicopter, Journal of Mine Action, James Madison University, Issue 7.3, 2003, pp. 54-58.
Bentley (2000): Descartes, 2000.
Castelman, K. R. (1996): Digital Image Procesing, Prentice Hall International, Inc., 1996.
DuncanTech, MS2100/MS2150&MS3100, Digital Multispectral Camera, User Manual, Document Number: 9000-0001-04
Intergraph (1998): Image Analyst, 1998.
Krtalić, A. (2006): Fuzija i interpretacija zrakoplovnih digital-nih snimaka za vidljivo, infracrveno blisko i termalno valno područje, magistarski rad, Geodetski fakultet Sveučilišta u Zagrebu, Zagreb, prosinac 2006.
Richards, J.A. (1986): Remote Sensing Digital Image Analysis, An Introduction, Springer-Verlag, Berlin, 1986.
Stojčić, M. (2006): Obrada, analiza i interpretacija zrakoplovnih infracrvenih termalnih snimaka, diplomski rad, Geodetski fakultet Sveučilišta u Zagrebu, Zagreb, veljača 2006.
ThermaCAM Researcher (2001): Operation Manual
URL1: Wolfram MathWorld, http://mathworld.wolfram.com/ThinPlateSpline.html (27.06.2005)