Extracting Temperature Estimates from Photographs

Extracting Temperature Estimates from Photographs

© Richard Harrison, 17 October 2016

Introduction

I have an interest in measuring the temperatures of pyrotechnic phenomena in the approximate range of 600K to 2000K. Since the temperature varies significantly in relatively short time intervals, I need a measuring instrument with a very rapid response. Clearly, a camera (either still or – better – video) can provide such a response, but can it give reliable temperature measurements?

A sufficiently hot object emits light, approximating to black body radiation, in the infrared and visible regions. In principle, it ought to be possible to deduce a reasonably accurate value for the object’s temperature from the RGB data in a digital photographic image. In practice, however, the task is made more complicated by the fact that a digital camera typically performs sophisticated modifications of the raw data in order to generate an image that looks natural to the eye. In addition to adjusting the colour balance, the camera must compress the brightness range present in the original scene to one that can be expressed by numbers in the range 0 to 255. My first attempts to make temperature measurements, which took no account of this kind of post processing, failed miserably.

Early Attempts

Colour temperature

Initially, I looked at a standard mapping between the absolute temperature and the corresponding colour temperature, expressed in terms of RGB values. The mapping that I found works only for temperatures of 2000K and above, which is somewhat higher than I would have liked. Also, it produced values with R always close to 255 and I couldn’t work out how to relate them to the more varied RGB values in an actual photograph. Eventually I came up with an ad-hoc method but, perhaps not unsurprisingly, the temperatures I obtained were always greater than 2000K, even when I knew that was definitely the wrong answer.

Wien’s Displacement Law

The black body spectrum has a peak of intensity at a wavelength, λmax, which is related to the absolute temperature, T, in a very simple way:

λmaxT = const.

If that peak lies within or close to the visible range, it ought to be possible to determine its location by looking at the relative intensities of R, G and B (assuming that one knows the wavelength band that each of the three detectors is sensitive to). Unfortunately, it turns out that the peak is in the visible region for temperatures that exceed 3500K, so even if the method works it won’t give meaningful data in the temperature range of interest.

Planck’s Law

In 1900, Max Planck derived an equation that described the precise distribution of radiation in the spectrum of a black body at any given temperature, a physical observation that had previously been totally inexplicable. In developing his theory he used what he originally considered to be a mathematical trick, but it later turned out to be the first step in the development of quantum theory.

With his law, given the temperature, it is possible to calculate the intensity of black body radiation at any wavelength. Conversely, knowing the relative intensities at two or more different wavelengths (or in two or more wavelength bands) should allow the calculation of the corresponding temperature.

I attempted to apply this reasoning to the relative values in the three spectral bands represented by R, G and B, but without success. I concluded that the most likely reason for this failure is the unknown processing used to adjust the data during its conversion to an RGB image.

An Approach that Works?

Because of the unknown differences between the incident radiation and the processed RGB values, the application of pure physical principles is not likely to lead to success. Perhaps an empirical approach, based more closely on the concepts surrounding colour and RGB images, might prove more fruitful.

Colour Temperature Revisited

With that thought in mind, I returned to the idea of colour temperature. Another search discovered a more precise algorithm, derived by Tanner Helland, for calculating RGB values. Not only is it more accurate than the one I had previously used, but it is claimed to be valid for temperatures as low as 1000K (and maybe a bit lower, but possibly with less precision).

I discovered that his algorithm could produce sensible-looking colour values to temperatures as low as about 650K (roughly 400°C) so I decided to extend the range down to this value, and attempt find out – again empirically – how valid the data was.

By this time I had come to realise that the problems I had encountered in my previous attempts to use colour temperature were largely due to the fact that, in the RGB system, the concepts of colour and intensity are inseparably mixed. I therefore had the idea of switching the description of colours to the Hue, Saturation and Luminosity (HSL) coordinate system. As shown in the diagram, this system represents colours on a colour circle. ‘Hue’ is represented by the angle of rotation from the vertical and ‘Saturation’ by the distance along the radius from the centre. The diagram doesn’t show the ‘Luminosity’, which is how “bright” the colour is.

The standard algorithm used by Microsoft to convert from RGB to HSL gives values between 0 to 240 for S and L, and 0 to 239 for H (240 being equivalent to 0 in a circular measurement system). I believe the maximum value is chosen to be compatible with the one-byte storage used for RGB values, with the additional condition that the algorithm works better if the maximum value is divisible by 6. The calculation itself is a little bizarre, using different algorithms depending on which of R, G or B has the largest value, and is undefined for any colour for which R = G = B. In that case, the algorithm uses an arbitrary Hue value of 160. For some values of R, G and B the calculated Hue is negative and is converted to the corresponding positive value by adding the full-circle value of 240 (so that, for example, -1 becomes 239).

The following table gives the RGB values, calculated with the aid of the Tanner Helland formula for temperatures from 651K to 3000K. It also shows the actual sRGB colours and their coordinates in the HSL system.

T (K) / R / G / B / Colour / Hue / Sat. / Lum.
651 / 255 / 0 / 0 / 0 / 240 / 120
700 / 255 / 11 / 0 / 2 / 240 / 120
800 / 255 / 29 / 0 / 5 / 240 / 120
900 / 255 / 45 / 0 / 7 / 240 / 120
1000 / 255 / 58 / 0 / 9 / 240 / 120
1100 / 255 / 70 / 0 / 11 / 240 / 120
1200 / 255 / 81 / 0 / 13 / 240 / 120
1300 / 255 / 90 / 0 / 14 / 240 / 120
1400 / 255 / 99 / 0 / 16 / 240 / 120
1500 / 255 / 107 / 0 / 17 / 240 / 120
1600 / 255 / 114 / 0 / 18 / 240 / 120
1700 / 255 / 121 / 0 / 19 / 240 / 120
1800 / 255 / 127 / 0 / 20 / 240 / 120
1900 / 255 / 133 / 7 / 20 / 240 / 123
2000 / 255 / 139 / 20 / 20 / 240 / 129
2100 / 255 / 144 / 32 / 20 / 240 / 135
2200 / 255 / 149 / 43 / 20 / 240 / 140
2300 / 255 / 154 / 53 / 20 / 240 / 145
2400 / 255 / 158 / 62 / 20 / 240 / 149
2500 / 255 / 162 / 71 / 20 / 240 / 153
2600 / 255 / 166 / 79 / 20 / 240 / 157
2700 / 255 / 170 / 87 / 20 / 240 / 161
2800 / 255 / 174 / 94 / 20 / 240 / 164
2900 / 255 / 177 / 102 / 20 / 240 / 168
3000 / 255 / 180 / 108 / 20 / 240 / 171

As can be seen, the colours are clearly distinguished by Hue up to a temperature of 1800K, but not thereafter. A plot of T against H (Hue) over this range shows a good fit to an exponential curve with the equation:

T = 631.17e0.0512H (1)

In what follows, I assume that this relationship holds, regardless of the values of H (or T). The justification for this assumption will be entirely dependent on whether or not it proves to be of value in providing a believable result.

The Data

The experimental data comes from night-time video recordings of the ignition of primed crackle grains. A range of exposure times was used, with the eventual aim of finding the most appropriate settings. For each recording, RGB values were measured in every video frame, at a selected point in the image of the burning grain. A typical set of results is shown in the accompanying diagram, which plots total illumination (R+G+B) against time. The large fluctuations visible during the first second are a result of the burning prime, which generates both flame and smoke. That is followed by a steady decrease in brightness, as the surface of the grain cools, followed by a sudden increase in brightness as the grain explodes.

The RGB values were transformed into the HSL coordinate system, although the Saturation and Luminosity values were ignored. Any Hue value that turned out to be negative was not converted to a positive one, on the grounds that this preserves the physical – and mathematical – relationship of one value with another far better than would be the case if the essentially arbitrary value of 240 were added. The Hue was then converted to a temperature, using equation (1), regardless of whether or not the values were within the range over which that relationship had been established.

A further complicating factor might come from the fact that the camera’s white balance processing could modify the Hue in some unknown way. As far as I know, the general idea of such processing is to make the image of a ‘white’ object appear white (or, at least, a shade of grey, for which R=G=B) regardless of the colour temperature of the incident illumination. I am hoping that the fact that all but a very small number of pixels in the image are black (R = G = B = 0) and therefore lie on the grey scale will result in no – or very little – such correction taking place.

The accompanying graph shows a plot of the ‘temperature’ against time for the same set of results that are shown in the previous diagram. The values appear reasonable, at least in the region below 2000K, where the relationship ought to have the greatest validity. Note that there appears to be a cut-off at a value a little over 5000K. It corresponds with positions where the RGB values approach (255, 255, 255). Remember that Hue is undefined if R=G=B, and bear in mind that its calculated value can vary widely and erratically if R, G and B are close in value. In other sets of data, the same effect appears in underexposed images, where R, G and B are all small numbers and therefore may be close in value.

Is This Reasonable?

The argument up until now has a few obvious holes, notably that equation (1) has been assumed to hold for all values of Hue, both positive and negative, despite evidence that it may not be valid for Hues outside the range 0 to 20.

If we make the (to me) reasonable assumption that the overall intensity (R+G+B) is a monotonic function of temperature (i.e. hotter always means brighter) the first graph gives one way of looking at how temperature varies with time. Unfortunately, the scale is unknown and the relationship is highly likely to be non-linear. The second graph, although derived from the hue in a dubious fashion, and with a highly speculative temperature scale, gives a second way of looking at the same variation.

Note that these two views are largely independent of each other because, in general, there is no intrinsic relationship between a light’s intensity and its hue. If they are related in this case, it can only be because they are both derived from images of the same source of thermal radiation. A simple way to compare them is to plot one (R+G+B) against the other (hue ‘temperature’). The resultant graph, shown below (after removing the problematic data points for which R, G and B are close in value) is remarkable.

The relationship between the intensity, I = R+G+B, and the temperature, T, is as clear as it was unexpected. It is so tightly defined that it was not difficult to find a suitable mathematical expression:

I = a.loge(T + b) + c (2)
where a, b and c are constants.

The inverse relationship, giving T as a function of total intensity is:

T = e(I-c)/a – b (3)

As shown in the diagram, plotting the value of this expression for measured values of I (=R+G+B) against the corresponding temperatures obtained from the Hue values shows that there is very little scatter around the expected relationship.

So far, I have very few sets of data, and not all of the exposure settings are ideal. However, they all give the same broad picture and, perhaps more significantly, give remarkably similar values for the constants a, b and c. To me, that suggests that the results are physically meaningful, although it is not yet clear to me exactly what the meaning is.

But Does it Work?

The accompanying graph shows the time variation of the temperature for the same example that was shown earlier, calculated by use of equation (3). Bear in mind, however, that this relationship was derived from data that might only be valid for temperatures up to about 2000K, so values that are significantly higher than that might well be subject to a greater degree of uncertainty.