Skip to main content
U.S. flag

An official website of the United States government

Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland

January 31, 2013

Model adequacy is evaluated with alternative models rated using model selection criteria (AICc, BIC, and KIC) and three other statistics. Model selection criteria are tested with cross-validation experiments and insights for using alternative models to evaluate model structural adequacy are provided. The study is conducted using the computer codes UCODE_2005 and MMA (MultiModel Analysis). One recharge alternative is simulated using the TOPKAPI hydrological model. The predictions evaluated include eight heads and three flows located where ecological consequences and model precision are of concern. Cross-validation is used to obtain measures of prediction accuracy. Sixty-four models were designed deterministically and differ in representation of river, recharge, bedrock topography, and hydraulic conductivity. Results include: (1) What may seem like inconsequential choices in model construction may be important to predictions. Analysis of predictions from alternative models is advised. (2) None of the model selection criteria consistently identified models with more accurate predictions. This is a disturbing result that suggests to reconsider the utility of model selection criteria, and/or the cross-validation measures used in this work to measure model accuracy. (3) KIC displayed poor performance for the present regression problems; theoretical considerations suggest that difficulties are associated with wide variations in the sensitivity term of KIC resulting from the models being nonlinear and the problems being ill-posed due to parameter correlations and insensitivity. The other criteria performed somewhat better, and similarly to each other. (4) Quantities with high leverage are more difficult to predict. The results are expected to be generally applicable to models of environmental systems.

Citation Information

Publication Year 2013
Title Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland
DOI 10.1029/2011WR011779
Authors Mary C. Hill, L. Foglia, S. W. Mehl, P. Burlando
Publication Type Article
Publication Subtype Journal Article
Series Title Water Resources Research
Series Number
Index ID 70189181
Record Source USGS Publications Warehouse
USGS Organization National Research Program - Central Branch