Appendix 1
Preprocessing Data
Reformatting Matlab Data
Water level data were reformatted from Matlab matrix arrays, which stored data by gauge station ID, to separate tabular delimited text files formatted by date, as described below. The raw data were obtained were stored in a matlab .m file as a matrix array. Within the matrix array, there were four variables stored as cell arrays; latitude, longitude, time, and water level. The time cell array corresponds to the date and the latitude and longitude cell arrays corresponds to the location, in decimal degrees, of each river or well water gauge. These data were edited manually, removing outliers, as to not skew results. Next, cell arrays within each matrix array were exported to a 4 column matrix. This new four column matrix contained the combined raw data for the 10 year time series. The combined raw data matrix had four unique columns; date, latitude, longitude, and water level. For bookkeeping purposes, a fifth column containing each dates corresponding serial date number was added. For example, January 7, 2002’s serial date number is 731223 and January 8, 2002’s serial date number is 731224. For these data to be used in Arcmap, they needed to be reformatted so that there was a single file containing the serial date number, date, latitude, longitude, and water level for each date in the time series. In order to separate these combined raw data by date, a matlab script was used for reformatting purposes. The end result of the matlab script was the partitioning of 1 matrix to 539 separate matrices that correspond to a certain date in the time series. After that, the 539 five-column matrices were saved in tab-delimited text format (.txt) using a string corresponding to the serial date number for when data were recorded. Upon completion of this conversion to a text file, data were ready to be loaded into ArcMap using the “Create XY event layer” tool.
Loading Water Level Data into Arcmap.
Before beginning spatial analyses, the 539 text files required loading into Arcmap. A series of Arcmap tools are required to convert these data from text files to feature classes. The first step consisted of using the “Create XY Event Layer” tool which will load any data containing x,y coordinates in decimal degrees. The tool requires several inputs; 1) the table containing the data, 2) the column holding the x-coordinates, 3) the column holding the y-coordinates, 4) the column holding the z- coordinates, and 5) the specified coordinate system. The output of this tool is a temporary “event layer” that will be deleted when Arcmap closes if the event layer is not exported and saved to a different format. There are multiple tools in order to do this, with the most time efficient being the “Copy Features” tool. The “Copy Features” tool requires only 2 inputs, the file you wish to input and convert, which in this case is the event layer described previously, and an output path containing the name and extension you wish the new file to be saved to. The resulting file would contain all water gauge readings for a single date in the time series.
To automate this process, a python script was used to iterate through all 539 text files. The script reads each text file and creates an event layer using the “Create XY Event Layer” tool and then copies and stores this event layer as a feature class in a geodatabase using the “Copy Features” tool. The last line of the script saves the feature classes so that the feature class name corresponds to the serial date number for when data were recorded.
DEM Preparation
A digital elevation model (DEM) is the most essential basemap in this study. It is critical to use a DEM with the highest spatial resolution in order to prevent data from becoming skewed. Three DEM products were tested in this study, ASTER, SRTM non-void filled, and SRTM void-filled. 2010 ASTER GDEM V2 30 meter tiles were available for download from NASA REVERB (http://reverb.echo.nasa.gov). Void filled SRTM 90m tiles were available for download from the CGIAR-Consortium for Spatial Information (srtm.csi.cgiar.org). A third non-void filled SRTM DEM that was compiled in Matlab and used in other studies (Steckler et al. 2010) was compared with the most recent void-filled 3-arc second SRTM DEM made available by CGIAR-CSI.
i. ASTER Processing:
The bounding coordinates of the study site, North 26.627126, South 20.57500, East 92.684655, and West 87.994418, query 38 individual DEM “granules” (1o x 1o tiles) from NASA REVERB and are downloaded in bulk as a zip file received through email. Each ASTER GDEM tile includes a DEM (.dem) file and a quality assessment (QA, .num) file in a georeferenced tagged image file format (GeoTIFF). All 38 .dem GeoTIFF’s were converted to individual geodatabase raster datasets using a batch script in PyScripter. Next, a script was created in order to fill the sinks in each ASTER DEM grid using the “Fill” tool in the spatial analyst toolbox. The new filled ASTER DEM grids were saved in a new geodatabase, “FILL” for further processing.
The ASTER DEM grids were then mosaicked together into one smooth continuous image. First, a raster catalog, “Mosaic,” was created and then all filled ASTER DEM’s in the “FILL” geodatabase were loaded into the new raster catalog. A raster catalog allows the user to manage up to thousands of individual raster datasets in a table format and also allows them to be displayed in one heterogeneous layer. After this, the raster catalog, “Mosaic,” was converted to a raster dataset, “ASTER_DEM.” This conversion mosaics the multiple raster datasets in the raster catalog to one homogenous raster dataset. The new mosaicked ASTER DEM was then ready for analysis in ArcMap 10.1.
ii. SRTM Processing:
On CGIAR-CSI’s website, Bangladesh is contained in four void-filled SRTM tiles. These four raw data tiles were downloaded as .tifs. Using a batch script in PyScripter IDE, each SRTM DEM .tif was converted to a raster dataset and stored in a geodatabase. Next, the newly created SRTM raster datasets were smoothed by filling the sinks and then stored in a new geodatabase. The filled SRTM DEM’s were then stored in a raster catalog. Finally, the SRTM DEM raster datasets stored in the raster catalog were mosaicked using the “convert raster catalog to raster dataset” tool. Upon the completion of this step, the four raw SRTM DEM tiles were now one homogenous raster dataset, “SRTM_DEM,” ready for analysis in Arcmap 10.1.
iii. Matlab SRTM Processing:
A 3 arc second SRTM DEM that was compiled and successfully used in a study by Steckler et al. 2010 was used to test the other DEM’s obtained for this study. This DEM was compiled for use in Matlab as a “.mat” file. Matrices storing latitude, longitude, and elevation values needed to be converted to a .txt file for use in ArcMap. A Matlab script was written which transposed the latitude matrix and then combined it with the longitude matrix. The new latitude and longitude matrix then had the same dimensions as the z-value elevation matrix. Next, a new three column matrix, “LatLonElev,” was created where each latitude, longitude, and corresponding z-value were appended. Due to file size restraints in ArcMap, the LatLonElev matrix was divided into two separate smaller matrices which were then saved as .txt files for input into ArcMap. The .txt files were imported into ArcMap using the “add xy data” tool and then saved as point feature classes. The two point feature classes were interpolated using kriging methodology and then mosaicked together to produce a single 3 arc second SRTM DEM raster grid. This new 3 arc second SRTM DEM raster grid was then compared to the 3 arc second void filled SRTM DEM obtained from CGIAR-CSI. As expected, they matched exactly except where there were data voids in the 3 arc second SRTM DEM.
The three digital elevation model (DEM) products; ASTER, SRTM 3-arc second 90m DEM, and SRTM void filled 3-arc second 90m DEM, were then tested against one another. The ASTER DEM contained many discontinuities between pixels and was therefore not included in future analyses. The SRTM 3-arc second 90m DEM, which was used in previous successful studies (Steckler et al., 2010) was tested against the SRTM void filled 3-arc second 90m DEM. The void-filled DEM contained interpolated surfaces where there were data gaps previously. The more complete DEM, the SRTM void filled 3-arc second 90m DEM, was therefore chosen to be the primary digital elevation model used in this study. All data were projected in the WGS84 geographic coordinate system.