Integration of Phenological Forecast Maps for Assessment of Biodiversity: An Enterprise Workflow
Recent open data policies of the Office of Science and Technology Policy (OSTP) and Office of Management and Budget (OMB), which were fully enforceable on October 1, 2016, require that federally funded information products (publications, etc.) be made freely available to the public, and that the underlying data on which the conclusions are based must be released. A key and relevant aspect of these policies is that data collected by USGS programs must be shared with the public, and that these data are subject to the review requirements of Fundamental Science Practices (FSP). These new policies add a substantial burden to USGS scientists and science centers; however, the upside of working towards compliance with top-down policies is improved discovery, accessibility (including machine readability), integration, and use of data for novel applications in support of the broader USGS mission. The purpose of this research was to exercise these new policies, as they relate to production of real-time and short-term forecasts of gridded biodiversity data, as a model for similar production and delivery of data products from other USGS projects and programs.
The objectives of this project were as follows:
- Establish and document a generalized workflow for making USGS biodiversity data and data products available for use by other programs within the USGS as well as by external partners and the public.
- Develop technical documentation and open-source code, enabling others to reuse infrastructure for data processing, validation, delivery, and reporting on data usage.
- Document this generalized workflow and lessons learned to significantly advance the delivery and translation of critical ecological information for a variety of stakeholders within the Department of Interior (DOI) and beyond in a USGS open-file report (OFR).
- Produce and release dynamic gridded maps, historical, real-time, and short-term forecasted predictions of leaf-out and flowering for several plant species, and underlying accumulated temperature data products through a case study serving as a test bed for developing a generalized workflow for real-time delivery of dynamic biodiversity data products.
Principal Investigator : Jake Weltzin
Cooperator/Partner : Alyssa Rosemartin, Lee Marsh, Sky Bristol, Tim Kern, Theresa Crimmins
Accomplishments
The accomplishments for this project are described below.
- The project team created, documented, and published two suites of national-scale gridded maps: Extended Spring Indices and Accumulated Growing Degree Days. The data, metadata, and documentation have undergone FSP review and were released in early FY 2017 (Crimmins and others, 2017).
- The team implemented OGC-compliant web services serving the gridded data and initiated collaboration with the National Biogeographic Map (NBM) under development and FSP review by Sky Bristol. The Extended Spring Index layers have been brought into the NBM as a “Bioscape,” and the Accumulated Growing Degree Day layers will also be brought into the tool soon. The project team continues to work with the NBM team to develop data summaries, called Bioscape Analysis Packages, or BAPs, that will enable NBM users to explore patterns in the Spring Indices within National Park Service and National Wildlife Refuge System units (Parks and Refuges) and also to look at daily growing degree day accumulations within a pixel over the course of a year. These data layers and BAPs will be part of the NBM when it goes live in early 2017.
- The python scripts and workflow associated with producing these suites of gridded data layers were checked into the USGS BitBucket code repository (https://my.usgs.gov/bitbucket/projects/CDI/repos/phenology-maps-workflow/browse - content no longer available).
- The team updated the USA National Phenology Network (NPN) ScienceBase entry to include information on the two suites of gridded data products (https://www.sciencebase.gov/catalog/item/52fd3728e4b0f010068e97ce).
- The team developed an API to share their data layers with the University of Arizona Library Spatial Data Explorer (https://geo.library.arizona.edu/).
- The team developed upgrades to the online USA-NPN Visualization tool to show the gridded layers as maps, which can be viewed independently or in conjunction with in-situ plant or animal phenology observation data (https://www.usanpn.org/data/visualizations).
- Map images (.png, .gif, .pdf) (figure 17) or Web Coverage Service (WCS) and WMS raster data files (GeoTiff, ArcGrid, NetCDF) are freely available and can be downloaded using the USA-NPN Geoserver Request Builder page (https://www.usanpn.org/geoserver-request-builder).
- Web services are available via the USA-NPN Geoserver (http://geoserver.usanpn.org/geoserver/ wms?request=GetCapabilities - content no longer available).
- Additional interpretive material is available at https://www.usanpn.org/data/phenology_maps.
- The project team is preparing a USGS OFR that will document a generalized workflow and lessons learned to significantly advance the delivery and translation of critical ecological information for a variety of stakeholders within the Department of the Interior and beyond. This OFR will include documentation of how leaders of a USGS project would review and validate their data products against each component of the data lifecycle and how they might prioritize efforts to reach compliance based on the OSTP and OMB open data policies. The audience for this report would be USGS project leaders seeking to publish and share their data and data products. (https://doi.org/10.3133/ofr20171003)
- The incipient publication of the USGS Survey Manual chapter 502.8 provides broad guidance for the review and approval of scientific data for release, but insufficient documentation to readily identify a workflow for review and approval of dynamic data released via APIs, or web services. The team is continuing to collaborate with members of the Office of Science Quality and Integrity, members of the Core Science Analytics, Synthesis and Library program, and the Fundamental Science Practices Advisory Committee on such a workflow.
- A Technical Information Sheet describing the gridded products was completed, reviewed, and published (USA National Phenology Network, 2016).
Note: This description is from the Community for Data Integration 2016 Annual Report.
- Source: USGS Sciencebase (id: 56d8784fe4b015c306f6cfca)
Development and release of phenological data products—A case study in compliance with federal open data policy
USA National Phenology Network gridded products documentation
Recent open data policies of the Office of Science and Technology Policy (OSTP) and Office of Management and Budget (OMB), which were fully enforceable on October 1, 2016, require that federally funded information products (publications, etc.) be made freely available to the public, and that the underlying data on which the conclusions are based must be released. A key and relevant aspect of these policies is that data collected by USGS programs must be shared with the public, and that these data are subject to the review requirements of Fundamental Science Practices (FSP). These new policies add a substantial burden to USGS scientists and science centers; however, the upside of working towards compliance with top-down policies is improved discovery, accessibility (including machine readability), integration, and use of data for novel applications in support of the broader USGS mission. The purpose of this research was to exercise these new policies, as they relate to production of real-time and short-term forecasts of gridded biodiversity data, as a model for similar production and delivery of data products from other USGS projects and programs.
The objectives of this project were as follows:
- Establish and document a generalized workflow for making USGS biodiversity data and data products available for use by other programs within the USGS as well as by external partners and the public.
- Develop technical documentation and open-source code, enabling others to reuse infrastructure for data processing, validation, delivery, and reporting on data usage.
- Document this generalized workflow and lessons learned to significantly advance the delivery and translation of critical ecological information for a variety of stakeholders within the Department of Interior (DOI) and beyond in a USGS open-file report (OFR).
- Produce and release dynamic gridded maps, historical, real-time, and short-term forecasted predictions of leaf-out and flowering for several plant species, and underlying accumulated temperature data products through a case study serving as a test bed for developing a generalized workflow for real-time delivery of dynamic biodiversity data products.
Principal Investigator : Jake Weltzin
Cooperator/Partner : Alyssa Rosemartin, Lee Marsh, Sky Bristol, Tim Kern, Theresa Crimmins
Accomplishments
The accomplishments for this project are described below.
- The project team created, documented, and published two suites of national-scale gridded maps: Extended Spring Indices and Accumulated Growing Degree Days. The data, metadata, and documentation have undergone FSP review and were released in early FY 2017 (Crimmins and others, 2017).
- The team implemented OGC-compliant web services serving the gridded data and initiated collaboration with the National Biogeographic Map (NBM) under development and FSP review by Sky Bristol. The Extended Spring Index layers have been brought into the NBM as a “Bioscape,” and the Accumulated Growing Degree Day layers will also be brought into the tool soon. The project team continues to work with the NBM team to develop data summaries, called Bioscape Analysis Packages, or BAPs, that will enable NBM users to explore patterns in the Spring Indices within National Park Service and National Wildlife Refuge System units (Parks and Refuges) and also to look at daily growing degree day accumulations within a pixel over the course of a year. These data layers and BAPs will be part of the NBM when it goes live in early 2017.
- The python scripts and workflow associated with producing these suites of gridded data layers were checked into the USGS BitBucket code repository (https://my.usgs.gov/bitbucket/projects/CDI/repos/phenology-maps-workflow/browse - content no longer available).
- The team updated the USA National Phenology Network (NPN) ScienceBase entry to include information on the two suites of gridded data products (https://www.sciencebase.gov/catalog/item/52fd3728e4b0f010068e97ce).
- The team developed an API to share their data layers with the University of Arizona Library Spatial Data Explorer (https://geo.library.arizona.edu/).
- The team developed upgrades to the online USA-NPN Visualization tool to show the gridded layers as maps, which can be viewed independently or in conjunction with in-situ plant or animal phenology observation data (https://www.usanpn.org/data/visualizations).
- Map images (.png, .gif, .pdf) (figure 17) or Web Coverage Service (WCS) and WMS raster data files (GeoTiff, ArcGrid, NetCDF) are freely available and can be downloaded using the USA-NPN Geoserver Request Builder page (https://www.usanpn.org/geoserver-request-builder).
- Web services are available via the USA-NPN Geoserver (http://geoserver.usanpn.org/geoserver/ wms?request=GetCapabilities - content no longer available).
- Additional interpretive material is available at https://www.usanpn.org/data/phenology_maps.
- The project team is preparing a USGS OFR that will document a generalized workflow and lessons learned to significantly advance the delivery and translation of critical ecological information for a variety of stakeholders within the Department of the Interior and beyond. This OFR will include documentation of how leaders of a USGS project would review and validate their data products against each component of the data lifecycle and how they might prioritize efforts to reach compliance based on the OSTP and OMB open data policies. The audience for this report would be USGS project leaders seeking to publish and share their data and data products. (https://doi.org/10.3133/ofr20171003)
- The incipient publication of the USGS Survey Manual chapter 502.8 provides broad guidance for the review and approval of scientific data for release, but insufficient documentation to readily identify a workflow for review and approval of dynamic data released via APIs, or web services. The team is continuing to collaborate with members of the Office of Science Quality and Integrity, members of the Core Science Analytics, Synthesis and Library program, and the Fundamental Science Practices Advisory Committee on such a workflow.
- A Technical Information Sheet describing the gridded products was completed, reviewed, and published (USA National Phenology Network, 2016).
Note: This description is from the Community for Data Integration 2016 Annual Report.
- Source: USGS Sciencebase (id: 56d8784fe4b015c306f6cfca)