To Obtain Radiosonde Files

28 April 2004

Landsat Thermal Calibration ProcedureNina Raqueño – Brian Staab

To obtain radiosonde files:

1)Go to the website

2)Enter the date you are looking for in the From – Thru fields

3)Enter the rest of the fields with desired information.

Hours of access and Data levels: All (usually).
Wind Units: Knots (usually)
UTC Time is used. I put in range 0-23 to get both radiosondes – Radiosonde observations are taken twice daily at 0000 hours (midnight) and 1200 hours (noon) UTC/GMT. For Buffalo(Eastern) 0-4= 8PM (2000 military) previous night (daylight savings april-oct) or 12UTC-4= 8 AM DS (0800) Date should be entered as the date of the scene plus the next day to insure you get the evening radiosonde as well

Note:

UTC – Universal Time Co-ordinate

GTM – Greenwich Mean Time

eg. For July 10, 2002 enter in date range July 10 and 11 Hours 0 – 23 Radiosonde for a given day is available at 8 AM and 8PM Eastern Savings

eg. Take a look at both the July 10th at 1200 and July 11 at 0 to see which profile is most appropriate. Usually best to just get both. For summer months especially, use the evening radiosonde for humidity reasons.

4)For the field: Select Radiosonde Sites by:

The easiest option is WBAN Station Identifier.

The closet radiosonde station to Rochester is Buffalo who’s WBAN# is 14733.

5)For the field: Sort order – If you intend to use Julia Barsi’s suit of programs to create a carddeck, you should be sure to select FSL format (Ascii text) since this is the only format that is compatible with her code. Otherwise, it doesn’t really matter what is selected.

6)If you had entered the WBAN# earlier then you will see your radiosonde data output to the screen window.

The easiest way to save this data is to copy it and paste it into a text file. Save the text file with a “.rsd” extention ex. 20030518_00.rsd

If you intend to use this data with Julia Barsi’s code then be careful that the first line of the text file is the first line of the radiosonde data.

To obtain weather data:

1)Go to web page:

Note: Since being a school, we get free access so don't worry about subscribing.

2)Select New York, then Rochester Greater

3)Select Month and Year

4)Select the format ASCII Download (Hourly Observations)

5)Since this page is difficult to read, save the comma delimited text file and open in Excel (as a comma delimited file) for easier extraction of:

Dry Bulb Temperature (temperature) Dew Point Station Pressure Sea Level Pressure

Dry Bulb Temperature (temperature)

9th column of data when opened in Excel. Get degrees Celsius.

Dew Point

13th column in Excel. Get degrees Celsius.

Station Pressure

19th column. Inches in Hundredths

Sea Level Pressure

21st column. Tenths of hectopascals (millibars).

The measurements/ values are Shown as last 3 digits only without decimal point.

To enter the Sea Level Pressure in the text file “weather.txt”, be sure to enter the values inthe form given in the 2 examples below i.e preceeded by ‘10’ and a ‘decimal point’ before the last digit

eg. 013=1001.1 MB263=1026.3 MB

Example Excel File

6)Get the all data for the 2 closest times of the Landsat overpass.
(11:45 AM or 10:45 LST ) so get: 954 and 1054.

Note:

LST – Local Standard Time

7)Enter the data into a text file named “weather.txt” Clicking on this link will open the existing file “weather.txt”. You can either save it on to your PC as a text file with the same name and enter the values from your weather data or you may copy & paste the text from this document and then enter the data appropriately. Be sure to save the file as “weather.txt” only.

The existing file “weather.txt” has the following format:

THis file contains ground information for low_atm.pro

Replace numbers with new values, but do not change the format of the file!!

Time elapsed between readings of ground truth:

Time elapsed between first sample and Landsat overpass:

Enter the units of Temp either C ,F ,or K:

First Ground Temperature:

18.3

Second Ground Temperature:

18.9

First Dew Point:

8.3

Second Dew Point:

7.2

Enter level of lake in km, Ont= 0.074, Sup=0.183, Mich=.1765:

0.074

Enter elevation of station in km, .222 buffalo, .238 flint

.222

Enter Units of SeaLevel Pressure mb or in: M or I

First SeaLevel Pressure (note if unavailable just enter station pressure):

1026.3

Second SeaLevel Pressure (note if unavailable just enter station pressure):

1025.9

Enter Units of Station Pressure mb or in: M or I

First Station Pressure in units of in:

29.68

Second Station Pressure in units of in:

29.66

How to Create a Card Deck:

To make a card deck with Julia Flags you need to:

1)Format the Radiosonde file

2)Add local ground weather

3)Create card deck

4)Add standard template layers from where the radiosonde stopped to Sensor Altitude (ie. 100km)

5)Change DOY manually:

1. Format the Radiosonde File:

A)Run these commands at a UNIX terminal:

To change the directory to the radiosonde formatting program -

cd /dirs/home/landsat/programs/radiosonde

To start IDL -

/cis/bin/idl

Compile the Radiosonde programs (@rad) -

@rad

cd, 'to_wherethe_radiosonde_file_is' -

cd, '/to where the radiosonde data file is'

ex. cd, ‘/dirs/home/landsat/2005/L7/20050131/rad’

Runs radiosonde program -

radiosonde

After having run these commands the radiosonde program will display a number of questions. This is what the screen will display/look like:

IDL> radiosonde

% Compiled module: LOADCT.

% Compiled module: FILEPATH.

% Compiled module: PATH_SEP.

% LOADCT: Loading table RAINBOW

RADIOSONDE AND RADIOSONDE CARDDECK MODIFIER

Which action would you like to take?

0) filter original radiosonde file to standard 4 columns and changes units to km & kelvin

1) change units of standard 4 col radiosonde (to C)

2) create carddeck from standard radiosonde

3) add layers to existing radiosonde

4) add layers to an existing carddeck with radiosonde

5) AUTO find the inversion layer and interpolate below it

6) MANUAL select inversion layer and interpolate below it

7) modify a column of dew point radiosonde values

8) modify a column of dew point radiosonde values in carddeck

9) replace nodata dewpts with standard model dewpoints

10) plot data from standard radiosonde file

11) get day of year from radiosonde filename

12) check for high humidities in radiosonde

99) end program

At the colon prompt you are supposed to enter the option you want to execute. Before creating the card deck a couple of runs of different options to clean up the data and get it into the desired processing format will have to be performed.

B)First choose‘Option 0’:

This will remove all the extra columns and rows off the radiosonde data that you got off the web.

In the radiosonde file – “FSL format (ASCII text)”; ‘No data’ values are coded 9999.

The rsd_filter.pro only checks for the 9999

Note:In some rare cases, ex. Original FSL format (ASCII text),‘no data’ values may have a different code like 32767 - if this is the case modify the .pro or manually edit the file.

The following questions will be asked for ‘Option 0’:

Enter radiosonde file to filter (straight from the noaa web site): 20030518_00.rsd

Enter output radiosonde filename: 20030518_00.rad

This option converts 20030518_00.rsd to 20030518_00.rad file

This is how the output of ‘Option 0’ looks:

Enter radiosonde file to filter (straight from the noaa web site): 20030518_00.rsd

Enter output radiosonde filename: 20030518_00.rad

MAY

lines before clean up : 74.0000

lines after clean up : 38

Units are now in kilometers, millibars, kelvin and kelvin

C)Next choose ‘Option 10’: This option plots the Radiosonde Data

You can check visually if the data needs more cleaning

The following questions will be asked for ‘Option 10’:

Enter standard radiosonde filename: 20030518_00.rad

Plots values so you can see:

Choose ‘plot option 0’ to plot -

1. Pressure vs. Altitude - Check if radiosonde reached altitude ~30km choice 0 plot pressure altitude

Choose ‘plot option 1’ to plot –

2. Plot air and dew point vs. altitude - look for any obvious errors such as where lines cross and high humidity clouds.

D)Choose ‘Option 12’ to detect high humidity entries:

Find any High Humidities data entries and remove the high humidity data entries if possible.

The following questions will be asked for ‘Option 12’:

Enter standard filtered Radiosonde: 20030518_00.rad

Initial air temperature value: 289.750

0: kelvin

1: celcius

: 0

Initial dew point value: 281.750

0: kelvin

1: celcius

: 0

This option displays the data in 2 columns. The first column is the altitude indices titled “ALT” and the second column is the corresponding “Relative Humidity”. The data is as displayed below:

ALT Relative humidity

0.218000 58.8106

0.195000 100.000

1.21900 100.000

1.55200 95.8830

Note:If any data entries are found (over 95 % humidity) remove the data entries directly from *.rad file. Directly means manually here!

A warning as the one listed below will be displayed in this case:

CHECK HUMIDITY VALUES

You have relative humidities exceeding 95%

You might consider modifying your dewpt

This option will display a plot of the humidities in the existing IDL plot window. Run this option 12 again after having manually removed the unacceptable data entries to verify that there are no more high humidity values.

Note: To manually remove means to open the *.rad file and to eliminate the row corresponding to the altitude index for which the relative humidity value is higher than 95%.

2. Add Local Weather Data

A)To add the local weather data choose ‘Option 6’:

Find the inversion layer and interpolate below it (option 6). I recommend doing it manually.

You will also need ground data to input into the program. This is the weather data obtained from above.

Use the standard file format for the surface data, called weather.txt. Just copy the file and change the entries to the conditions at Rochester for 10:00 and 11:00 am on that day, if they weren’t already done. Times are in local standard times, add units in weather file.

LST temp dew stat press(in) sea lvl

954 18.3 8.3 29.68 263 (1026.3)

1054 18.9 7.2 29.66 259 (1025.9)

The following questions will be asked for ‘Option 6’:

Enter standard filtered radiosonde: 20030518_00.rad

Enter output radiosonde filename: 20030518_00.wrs

Do you wish to enter data from a data file: 0: Yes

Enter data filename: weather.txt

You will be prompted to enter an altitude to start the inversion by looking at the graph and estimating a point to use. Start from bottom of the graph or from the x-axis upwards, and find first inflection/inversion point, and then interpolate across to the correct altitude. You will need to supply one of the exact altitudes from the file.

Note: The first inversion/inflection point in the graph is the index value or y-axis point where the graph falls backwards or the rise of the graph appears to fall back to the ground. It can also be the first point where the 2 graphs cross each other if there is no falling back phenomenon. To supply the exact altitude check the ALT index from the file where there is a sudden/considerable change in relative humidity for 2 consecutive indices.

Ex. ALT Relative humidity

0.218000 58.8106

0.853000 74.0930

1.92300 86.7913

2.01300 70.0846

Here the 1st inversion point is 1.923 because after increasing for 3 indices the relative humidity suddenly falls.

The following questions asked for ‘Option 6’ after entering the “weather.txt” file:

Enter alt at which to begin inversion: 1.923

The inversion_index is: 2

This option converts 20030518_00.rad to 20030518_00.wrs file (radiosonde weather file)

B) Check values for Correctness

Now check that the values in the file that you just made are correct

(i.e. if there are any values like 32787 or 9999 remove these line - these are ‘no data’ values)

At this point I normally plot them (option 10) to make sure there isn’t any problems with the data (filler numbers like 32787 or 9999 that may have snuck by).

If you have to check for high humidities now is a good time (option 12). If there are problems, fix it manually.

3. Create the Card Deck

You need to have a 'template' card deck available, find one here:

( /dirs/home/ngrpci/L7/template/temp.cdk or ~/temp.cdk )

It grabs the first 4 lines and last 5 lines from an existing card deck, and just generates the radiosonde lines.

A) Option 2

Enter standard filtered radiosonde: 20030518_00.wrs

Enter card deck file to use as model: temp.cdk

You will also need to answer units questions.

(Usually: altitude = km , temp = Kelvin , pressure = millibars)

Is this a carddeck for Modtran 3.5? 1 NO

Select 1 NOT for Modtran 3.5 even though we are running Modtran 3.5 ???

You may also be prompted to remove layers if there are more than 33. Just select layers that seem to be similar or already well represented.

Creates 20030518_00.cdk (card deck)

4. Add standard template layers

From where the radiosonde stopped to Sensor Altitude (i.e. 100 km)

A) Characterize to Landsat Space

At this point a .cdk has been created based on radiosonde levels.

However, you need to also characterize the atmosphere to Landsat space.

So cut and paste the higher altitudes from a standard atmosphere.

You will also need to edit the file to have the right number of layers index.

From 33 to 39 usually

Example:

T 7 2 2 1 0 0 0 0 0 0 1 1 1 %temp% 0.00

1 1 0 0 0 0 23.000 0.000 0.000 0.000 0.074

39 0 0Jun0799 INTERP below inversion layer

Grab the layers 35-100 from Space_fake.cdk and copy them into bottom layers of the cdk.

/dirs/home/ngrpci/L7/template/space_fake.cdk

5. Change DOY manually

Next edit the cdk file with the correct day of the year (May 18, 2003 = 138)

(see Barsi 2000, figure 60).

DOY can be found at

Example:

100 0.074 180.000 0.000 0.000 0.000 0

1 2 138 0

43.190 78.670 0.000 0.000 15.350 0.000 0.000 0.000

775 1000 5 5

(even though there seems to be an option in Julia’s radiosonde formatting tool it does not seem to edit this value in the file??)

RESULT

Now this should give you a new card deck file that you can now use to run in Julia's programs. This will not run on its own in Modtran, because there are replacement flags (%F% and others) that Julia's programs look for to substitute for real numbers.

EXAMPLE: /dirs/home/ngrpci/L7/20030518/ 20030519_00e.cdk

Run Modtran with Julia’s Program Suite:

The program(s) returns a filethat includes an intercept and slope; which is then appliedto sensor reaching radiance to give ground radiance.

Basic Steps:

1)Run the program: make_modtranprof

2)Run the program: make_adb

3)Calculate Integrated Downwelled Radiance

1.Run the Program ‘make_modtranprof’ -

A) At the UNIX prompt run IDL:

cd /dirs/home/landsat/programs/pro

/cis/bin/idl

B) Add to the current IDL path:

!path = !path + ‘:/dirs/home/landsat/programs/pro’

C) Compile

@com

D) Run the program ‘make_modtranprof’

The call in IDL will be in the form:

make_modtranprof,[A],[B],’C’,’D’,’E’, F,’G’,’H’

EXAMPLE:

make_modtranprof,[277.7,278.4,284.6,287.7,290.4],[.075,100],'20030518_00.out','/cis/phd/sag2937/summer/carddeck/20030518_00smita.cdk','/dirs/home/ngrpci/L7/template/L7.rsp',-1, '/cis/phd/sag2937/summer','/dirs/home/landsat/programs/pro'

It has a number of input parameters just for you to know while running the program:

A=the array of ground truth thermistor temperatures in Kelvin.

Note: (Must usetherm_cal_curve.xls to convert thermistor to Kelvin)

B=the array of altitudes (indices from the radiosonde data) from which the atmosphere is

removed from and to (lake level to satellite altitude)

C=the name of the outputfile containing the intercept and slope will be saved in

D=the carddeck and where to find it (ex. /cis/phd/sag2937/summer/carddeck/20030518_00smita.cdk)

E=the sensor responsivity and where to find it

F=-1 because you don’t want the multiangle technique

G=the path to the destination where you want the output file Cto be written to

H=the path indicating where the program files are

The program will output a file that looks like this:

Ex: 20030518_00.out, which has intercept and slope values

alt 0.0750000km rad 6.62847 6.70674 7.42203 7.79542

8.12917

alt 100.000km rad

6.71307 6.77885 7.38128 7.69584 7.97702

intercept, slope

-1.34164 1.18727

Intercept and slope will go into Excel Sheet for Sensor Reaching vs Predicted Graphs

Note: Make sure the file titled mod_run.rad is not writable!

2.How to Run the Program ‘make_adb’

This program gives us the downwelled radiance, .

Before you run the make_adb program,you have to do the following:

A) Replace the flags in the carddeck file with actual numbers.

i.e. replace %temp% with a temperature

replace %/F/% with the highest altitude

replace %/W0/% with the first (smallest) wave number from the responsivity file

replace %/W1/% with the last (largest) wave number from the responsivity file

Save the carddeck as a new carddeck file

Example:

T 7 2 2 1 0 0 0 0 0 0 1 1 1 280.0 0.00

1 1 0 0 0 0 23.000 0.000 0.000 0.000 0.074

39 0 0Jun0799 INTERP below inversion layer

100 0.074 180.000 0.000 0.000 0.000 0

1 2 138 0

43.190 78.670 0.000 0.000 15.350 0.000 0.000 0.000

775 1000 5 5

B) Make a *.cfg file for the cardeck.

- Make a copy of the temp.cfg file and rename it.

(i.e. cp temp.cfg 'date'.cfg)

Example:

cp /dirs/home/ngrpci/L7/template/temp.cfg /cis/phd/sag2937/summer/modtran/20030518_00.cfg

-- Open the new .cfg file ('date'.cfg). You need to change the TAPE5_FILENAME and the ADB_FILENAME in the “environment” section.

This section of the .cfg file will look like:

ENVIRONMENT {

TAPE5_FILENAME = /cis/phd/sag2937/summer/carddeck/20030518_00smita.cdk

ADB_FILENAME = /cis/phd/sag2937/summer/modtran/20030518_00smita.adb

ADB_FILENAME = 20030518_00smita.adb (if in the current working directory)

WEATHER_FILENAME = mls.wth

}

The TAPE5_FILENAME is the name of the new carddeck created in step 2.A.,