The Variabilityand Forecasting of
Treatment Plant Influent Water Quality
Erin Towler
Class Project - CVEN 6833
December 19, 2005
INTRODUCTION
As new drinking water regulations come into effect, utilities are faced with the challenge of meeting new regulations without jeopardizing their compliance with existing regulations. This is a formidable task, considering financial constraints coupled with the fact that water sources are becoming scarcer and often more polluted. To this end, an electronic decision tool has been proposed that will help utilities to strategize simultaneous compliance methods. The decision tool will need to consider a utility’s characteristics, including its influent water quality and treatment processes. This paper is concerned with characterizing the variability in treatment plant influent water quality and forecasting future values. This is important in deciding how to simulate influent water quality parameters that will be sent through the decision tool.
This paper compiles relevant diagnostic information on influent water quality on a national level. First, national averaged data was examined to determine variable relationships. Next, spatial variability of key influent water parameters were examined. Finally, the perspective shifts to look at nine utilities in the Colorado Front Range. A principle component analysis was conducted to try to resolve the spatial-temporal variability and to forecast future pH and alkalinity values.
DATASET
This paper utilizes the data from the United States Environmental Protection Agency’s (EPA) monitoring program, called the Information Collection Rule (ICR). The ICR’s Auxillary 1 database contains eighteen monthsof historical data (July 1997 – December 1998) from over 400 utilities from around the United States. The spatial distribution of the continental water utilities that participated in the ICR are seen in Figure 1. The database contains a large amount of data regarding influent, intermediary, and finished water quality parameters as well as information about the processes of each treatment plant.
Figure 1 Spatial distribution of water utilities that participated in the Information Collection Rule (ICR).
METHODOLOGY
The analyses completed in this paper were completed using Microsoft Access and the statistical program R. For all analyses that involved averages, only averages of available data were computed, any missing values were skipped. In all of the contour plots, annual averages were only taken on utilities that had at least 8 of 12 months of complete monthly data, unless noted. In analyses where all 18months were needed, any missing values were replaced with averages over the remaining months.
To examineparameter relationships, both the parametric linear correlation and the nonparametric mutual information calculations were employed. To look at the spatial variability of parameters, a local polynomial was fit to the data and contoured. A further discussion of local polynomial method can be seen in Loader (1999). The principle component analysis (PCA) was completed following Storch and Zwiers (1999). The forecasting aspect of the PCA analysis proceeded by fitting a locfit relationship between the first PC of alkalinity and the first PC of pH. The first PC of alkalinity was simulated using the k-nearest neighbor method. Only one simulation of 18 months was completed, and only the last 12 months were used (so as to simulate from where the original data left off). Then, based on the previously fit local polynomial relationship, the corresponding first PC of pH’s were calculated. The other PCs were bootstrapped and the whole PC matrix was multiplied by its respective eigen vector to return to the original space. One can see Regonda et. al. (2005) for more details.
RESULTS
Variable Relationships
Linear correlations and mutual information (MI) was computed for all of the variables in order to determine if there were relationships between influent parameters. If the linear correlation or the MI indicated that there was a relationship, scatterplots for those relationships were examined. Figure 2 shows the scatterplots from the relationships between alkalinity, calcium hardness, total hardness, and pH. Strong correlations were found between total hardness and calcium hardness, total hardness and alkalinity, and calcium hardness and alkalinity. This is what one would expect, since calcium hardness and total hardness are simply different measures of the same thing, and are the main contributors to alkalinity. A lower, but significant relationship was also found between pH and alkalinity, as well as with pH and total calcium and pH and total hardness. In all of the plots that include pH, a strong nonlinear relationship is observed. Of note are the very low values of alkalinity that correspond to low pH values – when providing ensembles for the decision tool, these values may need to be generated together. Figure 3shows the correlations between TOC, UV254, and temperature. TOC and UV254 are strongly correlated, as one would expect since UV254 is a measure of TOC. Also temperature and TOC and UV254 yielded some correlation, but only linear (not MI). This indicates that the relationship with temperature needs to be further examined, perhaps looking at the relationship between monthly TOC and monthly temperature (i.e. not averaged values). As a point of confirmation, the results found in this section are consistent with other findings (Zachman 2005) and these relationships will shape the way the rest of the analyses are approached. Complete correlation tables can be seen in Appendix A.
Figure 2 Scatterplots of related variables: alkalinity, calcium hardness, total hardness, and pH. Uses last 12 months of averaged data from continental US.
Figure 3 Scatterplots of related variables. UV_254 and TOC are highly related, while temperature and TOC and temperature and UV_254 are less related. Uses last 12 months of averaged data from continental US.
Spatial Variability
By creating contour plots of the average influent water quality parameters using local polynomial method, geographic trends can be seen.
Alkalinity and pH
Figure 4 shows a contour plot of the fitted average alkalinity over the last 12 months of the study. Alkalinity often comes from the leaching of calcium carbonate from rocks and soil. High levels of calcium carbonate are found in limestone, and in general, the eastern part of the United States has higher alkalinity than the western US because there is more limestone. The upper northeast has lower alkalinity because much of the limestone in that area has been scoured away by glaciers. This is consistent with what we see in Figure 4. It would be expected that a contour plot of calcium hardness and total hardness would garner the same relationship, based on the high correlation that was seen in the previous section. The alkalinity plot can also be compared by looking at a map of the interpolated raw data in Figure 5. This plot is less smooth, but allows one to focus in on local values. The other plot is better for seeing general trends. Figure 6 shows the standard errors from the fit – notice that there are higher errors on the outskirts of the original data. In addition, there are some higher errors internally.
Figure 4 Average alkalinity in ppm CaCO3 over the last 12 months of the study.
Figure 5 Contour plot of the interpolated raw data for average alkalinity.
Figure 6 Plot of the standard errors from locfit for average alkalinity.
A plot of the standard deviation of annual alkalinity can be seen in Figure 7. The highest standard deviation can be seen below the Great Lakes, in Illinois, Iowa, and Missouri. This plot shows that although there is some seasonality, in general, alkalinity is not all that different with the seasons.
Figure 7 Standard deviation of annual alkalinity.
The first eigen vector of alkalinity was also examined in order to see if the space/time decomposition was consistent with the previous findings. This can be seen in[ET1]Figure 7. One can see negative weights along much of the east coast and then positive weights in the central United States. The Western United States is mostly neutral, but with some positive weights.
Figure 8 Plot of the first eigen vector of alkalinity.
Finally, the local polynomial model was checked to see how good the fit was. The observed alkalinity is compared to the cross-validated estimate of the raw data in Figure 8. There is a fair amount of scatter, especially in the higher observed alkalinities. Figure 9 shows a 3D plot of the observed versus the cross-validated estimates, again showing that the model does not capture all of the observed alkalinities, especially in the higher range.
Figure 9 Cross-validated estimates of the raw data with a one-to-one line overlaid.
Figure 10 3D scatterplot of the observed alkalinity (filled red circles) and the cross-validated estimate of the alkalinity (empty black circles).
The contour plot of the fitted average pH over the last 12 months of the study can be seen in Figure 7. Based on the relationship that was identified in the previous section, one would expect that the trends in standard errors would be similar.
Figure 11 Average pH over the last 12 months of the study.
Total Organic Carbon (TOC)
TOC averages over the last 12 months of the study were also examined for their spatial variability. TOC concentrations differ greatly between surface water and groundwater sources, warranting a separate examination. Figure 6 shows the surface water contour plotof TOC concentrations. One can notice the higher contour values corresponding to the center of the US, which is largely agricultural and probably prone to erosion. Also, Florida has much higher TOC values than the rest of the country. This can loosely be attributed to the “swampy” nature of Florida. Figure 7 shows analyses that had been done previously on the same ICR data. This shows average TOC by state. In general, the plots show the same trends, but the local polynomial fit is not bound by arbitrary state lines, and uses interpolation. The averaged by state map does do a nice job of showing where there is no data, whereas the local polynomial map creates extrapolated concentrations in the north central part of the country, where there is no data. Figure 8 shows the groundwater TOC concentrations. Here, the values are generally lower than in the surface waters. However, Florida again shows very high TOC values. Similar information is gained from Figure 9, but it lacks interpolation capability.
Figure 12Local polynomial contour plot of the surface water sources: Last 12 months of study average TOC (ppm).Utilities contributing data are overlaid.
Figure 13 Surface water plot, last 12 months of study average TOC (ppm) from Cadmus online
Figure 14Local polynomial contour plot of the groundwater sources: last 12 months of study average TOC (ppmC). Utilities contributing data are overlaid. Utilizes all locations, regardless of the number of complete months of data. See Appendix for map of locations that use at least 8 months of data.
Figure 15Groundwater last 12 months of study average TOC (ppm) from Cadmus online
Contour plots of standard deviations for surface water and groundwater are shown in Figure 16 and Figure 17, respectively. For surface water, the central US has higher standard deviations. In addition, one can see that Florida also has the highest standard deviations[1]. There is less information gained from the groundwater countour plot, since the locations are sparse. There are higher standard deviations found in Florida, but careful examination shows that the standard deviations are lower in near all the places where there are data.
Figure 16 Surface water standard deviation contour map.
Figure 17 Groundwater standard deviation contour map.
Bromide
Bromide was examined in the context of spatial variation. In general bromide concentrations in ppm of bromide ar relatively low, with increases in the southwest and Texas, as well as in the northeast.
Figure 18 Average bromde concentration in ppm bromide over the last 12 months of the study.
Turbidity
Turbidity was also examined in the context of a spatial variation. However, this analysis did not have quite as conclusive results. The range in turbidity was great, with a number of outliers that were not geographically related. Rather, the water sources with the highest turbidity were typically large, widely used rivers such as the Missouri R., the Mississippi R., the Rio Grande R., the Ohio R., as well as others. Thus, the spatial analysis was not all that useful, and rather turbidity will likely need to be examined in a more case by case basis. Nevertheless, Figure 10 shows the average turbidity for all utilities whose average turbidity valuewas less than 25 NTU. Forty-six utilities were not included using this criteria. To further illuminate this dataset, Figure 11 shows a histogram of the data used to generate Figure 10. Again it should be noted that the majority of turbidities are extremely low, and that one may need to take a closer look at turbidity before making any generalizations.
Figure 19 Average turbidity in NTU over the last 12 months of the study (only utilities with average turbidity values less than 25 NTU were included).
Figure 20 Histogram of values used in Figure 10.
PCA Analysis
Nine of the Front Range utilities in Colorado were examined in a PCA analysis that looked at alkalinity and pH. The nine utilities were broken into two groups based on differences in pH and alkalinity, shown in Figure 10 and Figure 11. Group 1 (the left group) consists of two utilities in Aurora, one Denver utility, and one Pueblo utility. Group 2 (the right group) consists of Boulder, one Denver utility, two Colorado Springs utilities, and a Fort Collins utility. This was just done to aid in viewing the PCA and forecast results.
Figure 21 Left shows “Group 1” and right shows “Group 2” alkalinity time series.
Figure 22 Left “Group 1” and right shows “Group 2” pH time series.
Figure 12 shows the 9 utilities that were used, broken into two groups based on differences in pH and alkalinity shown in Figure 10 and Figure 11. The red triangles are Group 1, and the blue circles are Group 2.
Figure 23 Blue circles (Group 2) include Fort Collins, Boulder, 1 Denver plant, and 2 Colorado Springs plants (which are on top of one another). Red triangles symbolize 2 Aurora plants, Pueblo, and 1 Denver plant (Group 1).
A PCA was completed that used all of the data for the nine utilities (Groups 1 and 2 were combined). The Eigen spectrums were computed and can be seen in Appendix B. The first three PCs of each pH and alkalinity were scatterplotted with one another. This can be seen Figure 13. Analyzing all of the scatterplots, a similar nonlinear relationship is seenin the scatterplot of the first PCs of pH and alkalinity. Therefore, the rest of the PC analysis will be limited to the first leading PCs.
Figure 24 Scatterplots of the first three leading components of pH and alkalinity with locfit lines through the points.
The first eigen vector of alkalinity is spatially plotted in Figure 14 and zoomed in Figure 15. This shows that most of the locations contribute negative weights to the first PC, and contours changing in a northwest to northeast direction. The first eigen vector of pH is spatially plotted in Figure 16 and zoomed in Figure 17. Again, most of the locations contribute a negative weight to the first PC, in a west to east direction.
Figure 25 First eigen vector of alkalinity
Figure 26First eigen vector, zoomed in (note the “uneven” scale).
Figure 27 1st Eigen vector of pH.
Figure 28First eigen vector of pH zoomed in (note the “uneven” scale).
Figure 18 and Figure 19 show the time series of the PC values and their corresponding spectrum. As we would expect with such limited data, the only pattern being picked up is the annual cycle. This can be seen in the spectrum peak around 1 cycle per year.
Figure 29 First PC plot and smooth spectrum for alkalinity.
Figure 30 First PC plot and smooth spectrum for pH
Using the forecast method described in the methodology section, the 12 months following the historical data were forecast for all nine utilities. Next the two groups were separated (again, just for visual purposes), and the resulting forecastsare shown to the right of the vertical line in Figure 20 and Figure 21. The top plots in each figure are Group 1 and the bottom plot is Group 2. In each figure, the first 18 months of original data are shown, and then 12 more months have been forecast. One can see that each time series forecast (coded by color) seems to follow the same subtle seasonal trend as the original data.
Figure 31 Group 1 (top) and Group 2 (bottom) historical and forecast pHs. The original data is to the left of the vertical line and the forecast is to the left of the vertical line. Different point colors correspond to different utilities within the group.
Figure 32Group 1 (top) and Group 2 (bottom) historical and forecast alkalinity. The original data is to the left of the vertical line and the forecast is to the left of the vertical line. Different point colors correspond to different utilities within the group.