Assessing spatial transferability of a random forest metamodel for predicting drainage fraction

Journal of Hydrology
By: , and 



Fully distributed hydrological models are widely used in groundwater management, but model speed and data requirements impede their use for decision support purposes. Metamodels provide a simpler and faster model which emulates the underlying complex model using machine learning techniques. However, metamodel predictions beyond the ranges, in space and/or time, of training data are highly uncertain, and thus it is important to assess the predictive model performance to ranges outside the training data, i.e., model transferability. We present a novel methodology for evaluating model transferability to areas not contained in the training data set, based on various metrics that quantify the differences in covariate distributions between training and testing data. The transferability method can be employed as a screening tool to assess the suitability of a metamodel for spatial prediction beyond its training domain. We evaluated this transferability approach on a Random Forest metamodel of a 1000 km2 fully distributed coupled groundwater model for predicting drainage fraction, the partitioning of infiltrating water between drains and groundwater. We conducted spatial cross-validation on 9 holdout sub-basins to assess metamodel transferability beyond sampling locations and compared this estimate with a random split-sample validation test. Using mappable covariates only, the metamodel showed high performance (R2 = 0.79) tested on a 20% randomly sampled holdout. Conversely, metamodel performance significantly decreased for the 9 spatial holdouts (R2 ranging from 0.13 to 0.61). We document that the proposed transferability metric correlates with metamodel predictive performance, and demonstrate its use to assess model transferability to datasets outside the training data spatial domain.

Study Area

Publication type Article
Publication Subtype Journal Article
Title Assessing spatial transferability of a random forest metamodel for predicting drainage fraction
Series title Journal of Hydrology
DOI 10.1016/j.jhydrol.2022.128177
Volume 612
Issue Part B
Year Published 2022
Language English
Publisher Elsevier
Contributing office(s) New York Water Science Center, Upper Midwest Water Science Center
Description 128177, 11 p.
Country Denmark
Google Analytic Metrics Metrics page
Additional publication details