Remote Sensing in Hydrology, Spring 2009

Lab 3#

Remote Sensing in Hydrology, Spring 2009

Batch Processing NEXRAD MPE/Stage III Data and Statistic Analysis (1 of 2)

Purpose:

This lab helps you guys to understand the algorithm, the naming tradition, the storage and distributed mechanism of radar precipitation, and the methods to automatically retrieve and preprocess the NEXRA MPE/Stage III precipitation data. In the following lab 4, you will also learn to (1) preprocess raingauge data using VB in Excel; (2) use geostatistic approaches to evaluate the accuracy of radar precipitation using gauge data.

Step 1: preparation

· Download any one month (2004) of the MPE rainfall data from http://dipper.nws.noaa.gov/hdsb/data/nexrad/wgrfc_mpe.php (the very last table, XMRG format: a binary format). Save the data into your own working Folder.

· Download the scripts for processing the data from here http://www.utsa.edu/LRSG/Teaching/EES6973-06/mpe.zip, then unzip, you will find three files: ,;. The first one is the c source code for converting the MPE XMRG format to ASCII, the second one is the executable file, the third one is the Perl script doing all the untar, unzip, and repeating work.

· Login to the Linux system using the direction by Stuart’s email or I will show you in the class. Create a name call lab3, copy the first two files to here, and create a Data subdirectory under lab3, copy the Munch-e.pl and your downloaded MPE data to there.

(Note: to better understand the scripts and procedure, please read the reading assignment paper)

Step 2: Convert the tarred and compressed monthly file into hourly ASCII file using the Perl and C program ran in Linux platform

· Click “New terminal window” in fig.1. A new window of Unix or Linux type will be opened. For the first time, as required as Stuart’s email, I want you to change your password from here by typing “passwd”, which will lead you to change your password.

· Type ‘ls” –display directory and files, and ‘cd’ – change directory to your Data folder, where input “./munch-E.pl” and click enter. This will automatically untar and uncompress your monthly file into hourly ASCII file.

(Note: before you run the ./munch-E.pl, please change the permission to both mpe2ascLinux and Munch-e.pl, by typing chmod 744 thefilename, click enter)

Fig.1.

Step3: convert the ASCII file into GIS Grid file using ArcInfo workstation in Window platform

· At your local computer, create your own folder using your first name, then create two folders under your folder: GridData and ASCData.

· Transfer the converted hourly XMRG radar rainfall ASCII files from the Sever (from the Window of Fig 1) to your folder ASCData in the local computer.

· Open Notepad from Program->Accessories->Notepad, copy the following code to the NotePad,

o change the &Workplace C:\Wang\RadarProject\RadarData\ASCData to your own ASCData folder,

o change the COPY %tempt% C:\Wang\RadarProject\RadarData\GridData\%tempt% to your own GridData directory

o then save the code as asc2grid.aml into C:/workspace.

· Download the “z0101200018.zip” file from http://www.utsa.edu/LRSG/Teaching/EES6973-06/z0101200018.zip and save it in your C:/workspace. Then unzip it. Please be caution if you already have an info file in the workspace directory. If you do not know how to deal with it, come to ask Dr. Xie.

· Open ArcInfo from: Programs->ArcGIS->ArcInfoWorkstation->Arc

· Run code: &r asc2grid

/* Authors: Hongjie Xie, Xiaobing Zhou, Enrique Vivoni, Jan Hendrickx, Eric Small

/* Title of the paper: GIS-based NEXRAD Stage III precipitation database: automated approaches for data processing and visualization

/* PURPOSE: batch process to transfer ASCII files to ArcInfo grid files, define the polar stereographic projection for the grid files, reproject it to geographic (sphere) coordinate, and then to UTM 13, WGS 84 (ellipsoidal earth datum) coordinate;

/* &run asc2grd1.aml

&workspace C:\Wang\RadarProject\RadarData\MPE2005_ASCData /* change to your own workspace where store ascRadar data and codes

&type [show &workspace]

&s count = [filelist *.asc asc.list -file]

&if %count% <= 0 &then &return Error generating output file.

&s unit := [open asc.list ok -read]

Precision Single Highest

&do I := 1 &to %count%

&s temp := [read %unit% ok] /*temp get the name like xmrg01019600z.asc

&s tempn = [before %temp% z] /*tempn get the name like xmrg01019600z

&s tempt = r[after %tempn% _]z /*tempn get the name like 01019600z

/*&s tempn = r[e %tempn% z] /*tempn get the name like r01019600

/*&s suffix = t

/*&s tempt = %tempn%%suffix% /*tempn get r01019600u

&s tempg = g[after %tempn% _]

&s tempp = p[after %tempn% _]

/* Begin: ASCII to Grid

ASCIIGRID %temp% %tempp% FLOAT

/* Begin: Define Projection (copy from existing one)

PROJECTCOPY GRID c:\WorkSpace\z0101200018 GRID %tempp%

/* Begin: Reproject to geo

Project GRID %tempp% %tempg% # NEAREST

output

projection geographic

spheroid sphere

units dd

parameters

end

/* Begin: Reproject to Texas Centric Mapping System, Albers Conic Equal Area,

/* Datum: NAD83, Ellipsoid: GRS80

Project GRID %tempg% %tempt% # NEAREST 4000 /*reproject tempn to tempt

OUTPUT

Projection ALBERS

Units METERS

Spheroid GRS80

parameters

27 30 00

35 00 00

-100 00 00

18 00 00

1500000.00

6000000.00

end

COPY %tempt% C:\Wang\RadarProject\RadarData\MPE2005_GridData\%tempt%

/* change to your own workspace where store grid data

kill %tempt% all

kill %tempg% all

kill %tempp% all

/*&s [delete %temp%]

&end

&s ok := [close %unit%]

&s ok := [delete arc.list -file]

&return

/* Authors: Hongjie Xie, Xiaobing Zhou, Enrique Vivoni, Jan Hendrickx, Eric Small

/* Title of the paper: GIS-based NEXRAD Stage III precipitation database: automated approaches for data processing and visualization

/* &run asc2grd1.aml

Step 4: Prepare raingauge coverage in ArcCatalog and ArcToolBox

· Download the raingague file from here: http://www.utsa.edu/LRSG/Teaching/EES6973-06/GPGauge.txt, randomly choose about 10 raingauges, input their gauge ID and lat/long and save it as SiteData.txt file in your own work folder.

· Create feature class and define projection (This is a step you should always remember and know how to do it. For the first time, you can follow the step below, but you should remember it)

o Open ArcCatalog, Navigate to your own folder, right click SiteData (Fig.2 and 3), click Create Feature Calss->From XY table, Click Spatial Reference of input coordinates->Select ->Geographic Coordinate System->North America-> click North America Datum 1983 Clikc Ok. The defined coordinate system is GCS_North_American_1983. Specify the output into your own folder.

· Convert it to Texas Central Mapping System Albers (similar to the above, if you do not know, you can follow the step below, but you should remember it as well)

o Click ArcToolBox tab, click Data Management Tool->Projections and transformation-> Features->Click Project, click Output coordinate system->Select->Projected Coordinate System -> State System->Click NAD 1983 Texas Central Mapping System Albers. Click ok. Click Ok.

· Convert shape file to coverage (again you should remember it)

o In ArcCatalog, Right Click Sitedata shapfile->Export-> to Coverage, Navigate the output into your own folder and Naming as SiteCov, click ok.

Fig. 2

Fig.3

Step 5: Extract the radar rainfall in radar cells collocated with gauges

· Open Notepad from Program->Accessories->Notepad, copy the following code to the NotePad, change the &Workplace C:\Wang\RadarProject\RadarData\GridData \%tempt% as your own directory that GridData lies, then save it as ConvGrid.aml in C:/workspace.

· Open ArcInfo from: Programs->ArcGIS->ArcInfoWorkstation->Arc

· Run code: &r ConvGrid

· The cellvalue.txt file in the GridData folder is the hourly radar rainfall (mm)

&workspace C:\Wang\RadarProject\RadarData\MPE2005_GridData

/* change to your own workspace where store grid data

/* copy the sitecov to the directory where you store the grid data

&type [show &workspace]

&sv outfile = cellvalue.txt

&sv fileunit = [open %outfile% openstatus -write]

&s count = [filelist * grid.list -grid]

&type %count%

&if %count% <= 0 &then

return Error generating output file.

&s unit := [open grid.list ok -read]

Precision Single Highest

&do I := 1 &to %count%

&s temp1 := [read %unit% ok] /*temp get the name like r01019500u

&s temp2 = [before %temp1% z]

&sv raintime = %temp2%

/* add time stamp for each hour

&sv record = %raintime%

&if [write %fileunit% %record%] = 0 &then

&type %outfile% written successfully

/* latticespot will end if spot item (vv) exists.

/* So each time, the item, which is in the coverage.pat file, has to be dropped before/after using.

DROPITEM sitecov.pat sitecov.pat w

latticespot %temp1% sitecov w

/* Add X and Y values to the attribute table.

addxy sitecov point

tables

select sitecov.pat

unload cellvalue.txt rgid w

quit

&end

&s ok := [close %unit%]

&return

Step 6: Rain gauge rainfall preparation, statistical analysis, and discussion to be followed in Lab 4. (in this lab3, I do not require you to report your results, but I hope you follow all the steps and get all the data you want for the next lab for analysis).