Comparison of two regression-based approaches for determining nutrient and sediment fluxes and trends in the Chesapeake Bay watershed

Scientific Investigations Report 2012-5244

Prepared in cooperation with the Virginia Department of Environmental Quality, Maryland Department of Natural Resources, and the U.S. Environmental Protection Agency Chesapeake Bay Program
, ,



Nutrient and sediment fluxes and changes in fluxes over time are key indicators that water resource managers can use to assess the progress being made in improving the structure and function of the Chesapeake Bay ecosystem. The U.S. Geological Survey collects annual nutrient (nitrogen and phosphorus) and sediment flux data and computes trends that describe the extent to which water-quality conditions are changing within the major Chesapeake Bay tributaries. Two regression-based approaches were compared for estimating annual nutrient and sediment fluxes and for characterizing how these annual fluxes are changing over time. The two regression models compared are the traditionally used ESTIMATOR and the newly developed Weighted Regression on Time, Discharge, and Season (WRTDS). The model comparison focused on answering three questions: (1) What are the differences between the functional form and construction of each model? (2) Which model produces estimates of flux with the greatest accuracy and least amount of bias? (3) How different would the historical estimates of annual flux be if WRTDS had been used instead of ESTIMATOR? One additional point of comparison between the two models is how each model determines trends in annual flux once the year-to-year variations in discharge have been determined. All comparisons were made using total nitrogen, nitrate, total phosphorus, orthophosphorus, and suspended-sediment concentration data collected at the nine U.S. Geological Survey River Input Monitoring stations located on the Susquehanna, Potomac, James, Rappahannock, Appomattox, Pamunkey, Mattaponi, Patuxent, and Choptank Rivers in the Chesapeake Bay watershed.

Two model characteristics that uniquely distinguish ESTIMATOR and WRTDS are the fundamental model form and the determination of model coefficients. ESTIMATOR and WRTDS both predict water-quality constituent concentration by developing a linear relation between the natural logarithm of observed constituent concentration and three explanatory variables—the natural log of discharge, time, and season. ESTIMATOR uses two additional explanatory variables—the square of the log of discharge and time-squared. Both models determine coefficients for variables for a series of estimation windows. ESTIMATOR establishes variable coefficients for a series of 9-year moving windows; all observed constituent concentration data within the 9-year window are used to establish each coefficient. Conversely, WRTDS establishes variable coefficients for each combination of discharge and time using only observed concentration data that are similar in time, season, and discharge to the day being estimated. As a result of these distinguishing characteristics, ESTIMATOR reproduces concentration-discharge relations that are closely approximated by a quadratic or linear function with respect to both the log of discharge and time. Conversely, the linear model form of WRTDS coupled with extensive model windowing for each combination of discharge and time allows WRTDS to reproduce observed concentration-discharge relations that are more sinuous in form.

Another distinction between ESTIMATOR and WRTDS is the reporting of uncertainty associated with the model estimates of flux and trend. ESTIMATOR quantifies the standard error of prediction associated with the determination of flux and trends. The standard error of prediction enables the determination of the 95-percent confidence intervals for flux and trend as well as the ability to test whether the reported trend is significantly different from zero (where zero equals no trend). Conversely, WRTDS is unable to propagate error through the many (over 5,000) models for unique combinations of flow and time to determine a total standard error. As a result, WRTDS flux estimates are not reported with confidence intervals and a level of significance is not determined for flow-normalized fluxes.

The differences between ESTIMATOR and WRTDS, with regard to model form and determination of model coefficients, have an influence on the determination of nutrient and sediment fluxes and associated changes in flux over time as a result of management activities. The comparison between the model estimates of flux and trend was made for combinations of five water-quality constituents at nine River Input Monitoring stations.

The major findings with regard to nutrient and sediment fluxes are as follows: (1)WRTDS produced estimates of flux for all combinations that were more accurate, based on reduction in root mean squared error, than flux estimates from ESTIMATOR; (2) for 67 percent of the combinations, WRTDS and ESTIMATOR both produced estimates of flux that were minimally biased compared to observed fluxes(flux bias = tendency to over or underpredict flux observations); however, for 33 percent of the combinations, WRTDS produced estimates of flux that were considerably less biased (by at least 10 percent) than flux estimates from ESTIMATOR; (3) the average percent difference in annual fluxes generated by ESTIMATOR and WRTDS was less than 10 percent at 80 percent of the combinations; and (4) the greatest differences related to flux bias and annual fluxes all occurred for combinations where the pattern in observed concentration-discharge relation was sinuous (two points of inflection) rather than linear or quadratic (zero or one point of inflection).

The major findings with regard to trends are as follows: (1) both models produce water-quality trends that have factored in the year-to-year variations in flow; (2) trends in water-quality condition are represented by ESTIMATOR as a trend in flow-adjusted concentration and by WRTDS as a flow normalized flux; (3) for 67 percent of the combinations with trend estimates, the WRTDS trends in flow-normalized flux are in the same direction and magnitude to the ESTIMATOR trends in flow-adjusted concentration, and at the remaining 33 percent the differences in trend magnitude and direction are related to fundamental differences between concentration and flux; and (4) the majority (85 percent) of the total nitrogen, nitrate, and orthophosphorus combinations exhibited long-term (1985 to 2010) trends in WRTDS flow-normalized flux that indicate improvement or reduction in associated flux and the majority (83 percent) of the total phosphorus (from 1985 to 2010) and suspended sediment (from 2001 to 2010) combinations exhibited trends in WRTDS flow-normalized flux that indicate degradation or increases in the flux delivered.

Geospatial Extents

Additional Publication Details

Publication type:
Publication Subtype:
USGS Numbered Series
Comparison of two regression-based approaches for determining nutrient and sediment fluxes and trends in the Chesapeake Bay watershed
Series title:
Scientific Investigations Report
Series number:
Year Published:
U.S. Geological Survey
Publisher location:
Reston, VA
Contributing office(s):
Virginia Water Science Center
x, 118 p.
Number of Pages:
United States
Delaware;Maryl;New York;Pennsylvania;Virginia;West Virginia
Other Geospatial:
Chesapeake Bay Watershed
North American Datum of 1983
Albers Equal-Area Conic Projection