Effects of error covariance structure on estimation of model averaging weights and predictive performance

Water Resources Research
By: , and 



When conducting model averaging for assessing groundwater conceptual model uncertainty, the averaging weights are often evaluated using model selection criteria such as AIC, AICc, BIC, and KIC (Akaike Information Criterion, Corrected Akaike Information Criterion, Bayesian Information Criterion, and Kashyap Information Criterion, respectively). However, this method often leads to an unrealistic situation in which the best model receives overwhelmingly large averaging weight (close to 100%), which cannot be justified by available data and knowledge. It was found in this study that this problem was caused by using the covariance matrix, CE, of measurement errors for estimating the negative log likelihood function common to all the model selection criteria. This problem can be resolved by using the covariance matrix, Cek, of total errors (including model errors and measurement errors) to account for the correlation between the total errors. An iterative two-stage method was developed in the context of maximum likelihood inverse modeling to iteratively infer the unknown Cek from the residuals during model calibration. The inferred Cek was then used in the evaluation of model selection criteria and model averaging weights. While this method was limited to serial data using time series techniques in this study, it can be extended to spatial data using geostatistical techniques. The method was first evaluated in a synthetic study and then applied to an experimental study, in which alternative surface complexation models were developed to simulate column experiments of uranium reactive transport. It was found that the total errors of the alternative models were temporally correlated due to the model errors. The iterative two-stage method using Cekresolved the problem that the best model receives 100% model averaging weight, and the resulting model averaging weights were supported by the calibration results and physical understanding of the alternative models. Using Cek obtained from the iterative two-stage method also improved predictive performance of the individual models and model averaging in both synthetic and experimental studies.

Additional publication details

Publication type Article
Publication Subtype Journal Article
Title Effects of error covariance structure on estimation of model averaging weights and predictive performance
Series title Water Resources Research
DOI 10.1002/wrcr.20441
Volume 49
Issue 9
Year Published 2013
Language English
Publisher Wiley
Contributing office(s) Earthquake Science Center
Description 19 p.
Larger Work Type Article
Larger Work Subtype Journal Article
Larger Work Title Water Resource Reseach
First page 6029
Last page 6047
Google Analytic Metrics Metrics page
Additional metadata about this publication, not found in other parts of the page is in this table