Skip to main content
U.S. flag

An official website of the United States government

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

December 11, 2020

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Publication Year 2020
Title Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
DOI 10.1038/s41467-020-20142-y
Authors Alec P. Christie, David Abecasis, Mehdi Adjeroud, Juan C. Alonso, Tatsuya Amano, Alvaro Anton, Barry P. Baldigo, Rafael Barrientos, Jake E. Bicknell, Deborah A. Buhl, Just Cebrian, Ricardo S. Ceia, Luciana Cibils-Martina, Sarah Clarke, Joachim Claudet, Michael D. Craig, Dominique Davoult, Annelies De Backer, Mary K. Donovan, Tyler D. Eddy, Filipe M. França, Jonathan P.A. Gardner, Bradley P. Harris, Ari Huusko, Ian L. Jones, Brendan P. Kelaher, Janne S. Kotiaho, Adrià López-Baucells, Heather L. Major, Aki Mäki-Petäys, Beatriz Martínez-López, Carlos A. Martín, Philip A. Martin, Daniel Mateos-Molina, Robert A. McConnaughey, Michele Meroni, Christoph F. J. Meyer, Kade Mills, Monica Montefalcone, Norbertas Noreika, Carlos Palacín, Anjali Pande, C. Roland Pitcher, Carlos Ponce, Matthew J. Rinella, Ricardo Rocha, María C. Ruiz-Delgado, Juan J. Schmitter-Soto, Jill A. Shaffer, Shailesh Sharma, Anna A. Sher, Doriane Stagnol, Thomas Stanley, Kevin D.E. Stokesbury, Aurora Torres, Oliver Tully, Teppo Vehanen, Corinne Watts, Qingyuan Zhao, William J. Sutherland
Publication Type Article
Publication Subtype Journal Article
Series Title Nature Communications
Index ID 70216895
Record Source USGS Publications Warehouse
USGS Organization Northern Prairie Wildlife Research Center