TrendPowerTool v1.0: Estimate statistical power to detect a population trend

Overview and instructions

A power analysis can be used to estimate the sample sizes needed for successful study or monitoring program. Statistical power is the probability of detecting a population trend of a particular magnitude. Simple calculations are often insufficient to estimate statistical power, so a simulation-based power analysis must be used. However, running a simulation-based power analysis requires technical expertise and can require extensive computing resources.

TrendPowerTool is a web-based lookup app to provide guidance for ecological monitoring programs when resources are not available for a simulation-based power analysis. By drawing on results of 1.4 million scenarios that we simulated on a supercomputer, TrendPowerTool quickly and easily estimates the statistical power of a monitoring program to detect a population trend of a particular magnitude. TrendPowerTool provides a wide range of options for the input parameters, including the monitoring design and the spatial and temporal variation of the resource to be monitored.

Results from TrendPowerTool are based on several assumptions that were necessary to develop a generalized power analysis. If your study system does not meet these assumptions, results from TrendPowerTool might not be accurate, and it would be better to conduct a full simulation-based power analysis with the appropriate model structure.

1. We assume that you will be monitoring the same sites repeatedly over multiple time intervals. A time interval will typically be a year, and we refer to it as such; but any other interval could be used (month, 5 years, 10 years) as long as the interval is consistent when you calculate variance and determine what population trend you want to detect. If you will not be repeatedly monitoring multiple sites, this power analysis is probably not appropriate for your program. 2. We assume that a linear model is appropriate to describe the population trend. If you expect an exponential trend or some other form, this power analysis might not be appropriate. 3. We assume that you have some idea of the spatial and temporal variance in your system, and that you calculated those values the same way we did (see the "Input values" tab at the top of the page). If your values are inaccurate, the results of the power analysis will be, too. If you do not have estimates of these values, but you have an idea of what to reasonably expect, you can try a range of values to bracket the potential range of sample sizes. 4. We assume that observation error, including imperfect detection, can be accurately represented by a constant term. If you have explicitly described detection in your system, your results might be more accurate if you perform a full power analysis with a model structure that incorporates the detection term rather than relying on generalized estimates from TrendPowerTool. 5. We assume that your input values will fall within the ranges included in our simulations. If the exact values for your population are not available in TrendPowerTool, try the nearest values above and below your estimate to examine the potential range of statistical power; results might be very close to what you would get with your exact values (Figure 1). If your value is outside the range of the values we simulated, it might not be safe to assume that your statistical power will be close to the results from this tool. If the underlying assumptions are met, TrendPowerTool provides a quick, user-friendly method for estimating statistical power for programs that aim to monitor population trend. TrendPowerTool thus makes power analysis a more accessible step in the process of designing monitoring programs.

If you would like to explore a wide variety of scenarios that cannot easily be displayed on the "Run Tool" page, you might prefer to download the entire database of results from the simulations that inform TrendPowerTool. See the "Input values" tab for definitions of the columns in this file:

Download all estimates (~50 MB)

Reference for more information about the power analysis used to produce these results: Weiser, E.L., J.E. Diffendorfer, L. Lopez-Hoffman, D. Semmens, and W.E. Thogmartin. 2021. TrendPowerTool: A lookup tool for estimating the statistical power of a monitoring program to detect population trends. Conservation Science & Practice e445. https://doi.org/10.1111/csp2.445
Contact: Emily Weiser (eweiser@usgs.gov). Please report any errors in red text that show up where the plots should be; errors in bold orange text are expected and indicate a known incompatibility with the input values you've selected.

Disclaimer:
This software is in the public domain because it contains materials that originally came from the U.S. Geological Survey, an agency of the United States Department of Interior. For more information, see the official USGS copyright policy (https://www2.usgs.gov/visual-id/credit_usgs.html#copyright).
Although this software program has been used by the U.S. Geological Survey (USGS), no warranty, expressed or implied, is made by the USGS or the U.S. Government as to the accuracy and functioning of the program and related program material nor shall the fact of distribution constitute any such warranty, and no responsibility is assumed by the USGS in connection therewith.
This software is provided "AS IS."

This research used resources provided by the Core Science Analytics, Synthesis, & Libraries (CSASL) Advanced Research Computing (ARC) group at the U.S. Geological Survey.

Last updated 19 January 2021
This is alternate text
Figure 1. Estimates of statistical power obtained from simulations using exact values (Table 1) versus estimates from TrendPowerTool for four example datasets. Estimates were made for three population trends, shown as the percent change per year.

Table 1. Summary statistics for four example datasets. Values in parentheses are the nearest available in TrendPowerTool. This is alternate text

How to calculate input values from pilot data

Population parameters
1. Transform the response variable, e.g. count per unit effort, to approximately follow a normal distribution. For example, if your data are counts that follow a Poisson distribution, a log transformation might be appropriate. If your data cannot be transformed to follow a normal distribution (e.g. zero-inflated), this tool might not be appropriate.

2. Center the response variable to a mean of 1, by dividing all (transformed) observations by the global mean. Use the transformed, centered responses for all following calculations.

3. Box 1: Find the mean for each site across all years and visits to that site. Calculate the SD among sites in their mean values and choose the nearest value (SD_mu).

4. Box 2: Calculate the site-specific population trends as the proportional change per year (e.g. using linear regression with an effect of year; Trend). Calculate the SD among sites in the trend and choose the nearest value (SD_trend).

5. Boxes 3 and 4: For each site, calculate CV across years from a detrended time series to represent annual variation (Gibbs JP. 2000. Monitoring Populations. Pages 213–252 in L. Boitani and T. K. Fuller, editors. Research Technique in Animal Ecology - Controversies and Consequences. Columbia University Press, New York, New York, USA):
First, for each site: 1) calculate the mean observation for each year with pilot data (if only one survey was conducted, this is simply the observation from that survey). This will be X[s,y] for each site s and year y. 2) Fit a linear regression for each site with X[s,] as the response and the year as the explanatory variable. 3) Get the annual coefficient of variation (CV) for this site by dividing the SD of the residuals from the model by the mean observation across all years, and take the absolute value: siteCV[s] = |SD(residuals)/mean(X[s,])|. Finally, average siteCV across sites and choose the nearest value in Box 3 (CV_yr). Also calculate the SD of siteCV across sites and choose the nearest value in Box 4 (SD_CV_yr).

6. Box 5: Calculate observation error as a proportion of the mean (Obs_err). The appropriate calculation will vary depending on your sources of observation error. If you have several replicate surveys within a survey period, you could calculate the SD among surveys to estimate observer error. If you don't have replicate surveys or some other measure of error, you'll want to choose your best guess or a moderate value (not zero).


Monitoring design
7. Box 6: Choose the number of years that you plan to survey during the monitoring program. Alternatively, this can be the number of intervals between surveys. For example, if you plan to survey every 2 years over 20 years, you will have 10 intervals (year 2, 4, 6, 8... 20). Thus the value of this box would be set to 10. If you use an interval other than one year, be sure to adjust the trend as well: to monitor a 10% change over 20 years with annual surveys, the per-interval trend is 0.01 (to be selected in Box 8 below); but if you survey every 2 years, the per-interval trend is 0.02. If you're interested in annual change, the number of visits per year would not affect the survey interval. However, if you are monitoring a change within a single year (say flowering phenology), your interval could be days, weeks, or months. The number of intervals is an important component of sample size when detecting a trend over time.

8. Box 7: Choose the number of sites that you plan to survey for the monitoring program. TrendPowerTool assumes that the same sites are monitored every year.

9. Box 8: Choose the population trend that you want to detect, expressed as a proportional change per year (e.g. 0.05 is a 5% change per year), or per the interval of interest (e.g. per 2 years if surveys will be conducted every second year). If you want to detect a 50% total change over 10 years, that would be equivalent to a 5% change per year. Given the data standardization steps above, the power to detect a negative trend will be the same as the power to detect a positive trend. A smaller trend (closer to 0) will be more difficult to detect.

10. Box 9: Choose the desired power, i.e. the probability of detecting a statistically significant trend in the correct direction (positive or negative). The default, 0.80, is often used but may not always be appropriate.

If you have insufficient data to calculate any of the above, test a range of values, or select "Any" to view the results for all values that are available in TrendPowerTool. (Select "Any" for only one parameter at a time.)
This is alternate text The development of TrendPowerTool was motivated by work to design a national monitoring program for monarchs and milkweed. However, TrendPowerTool is not appropriate to apply to these monitoring targets because our observations of both monarchs and milkweed follow a zero-inflated distribution, which cannot be transformed to approximate normality. Instead, we built another power analysis that appropriately represented the distributions of observations.
Instructions: 1. See the "About" tab for an overview of this tool and when it is appropriate to use. 2. Select the values that describe your population (see the "Input values" tab to calculate the values). 3. Optionally, select "Any" for one variable that you want to explore. Selecting "Any" for multiple variables will produce an error; instead, add a new scenario where a different parameter is "Any". 4. Press "Get new results" to produce a plot and create a table. The table is not displayed, but can be downloaded in step 6. Each update will take about 10-20 seconds to plot. 5. If you change any input value(s), press "Get new results" again to replace the previous results, or press "Add to results" to show both scenarios together. Up to 6 scenarios can be plotted at a time; all will be present in the table unless you press "Get new results". 6. After running your scenario(s), save the results table with the download button. A file called "TrendPowerTool_ScenarioResults.csv" will save to your default Downloads folder. The table includes all the input values as well as the scenario number to match with the corresponding plot. To save the plot(s) shown here, right-click on the image and select "Save image as...".



Download results table

Recommended citation when reporting results from TrendPowerTool: Weiser, E.L., J.E. Diffendorfer, L. Lopez-Hoffman, D. Semmens, and W.E. Thogmartin. 2021. TrendPowerTool: A lookup tool for estimating the statistical power of a monitoring program to detect population trends. Conservation Science & Practice e445. https://doi.org/10.1111/csp2.445