Skip to main content
U.S. flag

An official website of the United States government

Assessing spatial transferability of a random forest metamodel for predicting drainage fraction

July 16, 2022

Fully distributed hydrological models are widely used in groundwater management, but model speed and data requirements impede their use for decision support purposes. Metamodels provide a simpler and faster model which emulates the underlying complex model using machine learning techniques. However, metamodel predictions beyond the ranges, in space and/or time, of training data are highly uncertain, and thus it is important to assess the predictive model performance to ranges outside the training data, i.e., model transferability. We present a novel methodology for evaluating model transferability to areas not contained in the training data set, based on various metrics that quantify the differences in covariate distributions between training and testing data. The transferability method can be employed as a screening tool to assess the suitability of a metamodel for spatial prediction beyond its training domain. We evaluated this transferability approach on a Random Forest metamodel of a 1000 km2 fully distributed coupled groundwater model for predicting drainage fraction, the partitioning of infiltrating water between drains and groundwater. We conducted spatial cross-validation on 9 holdout sub-basins to assess metamodel transferability beyond sampling locations and compared this estimate with a random split-sample validation test. Using mappable covariates only, the metamodel showed high performance (R2 = 0.79) tested on a 20% randomly sampled holdout. Conversely, metamodel performance significantly decreased for the 9 spatial holdouts (R2 ranging from 0.13 to 0.61). We document that the proposed transferability metric correlates with metamodel predictive performance, and demonstrate its use to assess model transferability to datasets outside the training data spatial domain.

Publication Year 2022
Title Assessing spatial transferability of a random forest metamodel for predicting drainage fraction
DOI 10.1016/j.jhydrol.2022.128177
Authors Elisa Bjerre, Michael N. Fienen, Raphael Schneider, Julian Koch, Anker L. Højberg
Publication Type Article
Publication Subtype Journal Article
Series Title Journal of Hydrology
Index ID 70233571
Record Source USGS Publications Warehouse
USGS Organization New York Water Science Center; Upper Midwest Water Science Center