Fundamental Science Practices (FSP) FAQ: Metadata for USGS Scientific Data
Updated August 2022
These frequently asked questions (FAQs) supplement SM 502.7 and apply only to metadata for USGS scientific data. Other guidance and information on metadata for USGS data products are available at here. Additionally, FAQs on release of scientific data are available.
Updates and additions to the FAQs will be posted as they occur (month/year). Questions about FSP policies and procedures that are not addressed here should be directed to the FSP Advisory Committee or a BAO in the OSQI.
- What are metadata?
- Why do we need metadata for data?
- What do metadata records look like?
- How do I create metadata?
- What is a metadata review, and who can perform it?
- When do I create metadata?
- I have a lot of data packaged in different datasets and databases. For what packages of data do I produce a metadata record?
- Are metadata records required for any size dataset?
- Are metadata records needed for scientific datasets and databases that are provided by non-USGS authors and are subsequently included in USGS datasets, databases, or publications?
- Are the output data generated by a model simulation also subject to the metadata requirement?
- Do summary data tables in information products such as USGS series or outside publications (for example, journals) need metadata?
- Is USGS Science Publishing Network (SPN) editing required for metadata records?
- Where do the metadata records go once we have created them?
- If my data are associated with a publication, where does the Digital Object Identifier (DOI) pointing to that publication get placed in the metadata?
- If I cannot explain how the data were created in the metadata record, where do I place the URL or DOI for the product that describes the data creation process?
- Where can I find additional guidance or information about metadata?
1. What are metadata?
The term “metadata” refers to documentation of important aspects of data that describe where, when, and why the data were collected; who collected the data; what types of data were collected; what processes were used to create the data; what quality assurance controls were used; and where the collected data are located. Metadata are provided in a human-readable form as well as in a format that is machine readable (for example, XML) for automated use.
2. Why do we need metadata for data?
Metadata enable users to find, understand, and reuse the data, thus extending the life of the data. In addition, a metadata record is required by the USGS for including data in the Science Data Catalog. Federal Government mandates (including Executive Order 12906 and OMB Circular A-16) and USGS policy SM 502.7 require metadata as an integral part of data released to the public.
3. What do metadata records look like?
Examples of metadata for data include those found at https://archive.usgs.gov/archive/sites/sofia.usgs.gov/metadata/sflwww/SOFIA_Cape_Sable.html and https://archive.usgs.gov/archive/sites/sofia.usgs.gov/metadata/sflwww/gachemca.html. Refer to these and other examples on the USGS Data Management Web page.
4. How do I create metadata?
Various tools for creating metadata are available on the USGS Data Management Metadata web page.
5. What is a metadata review, and who can perform it?
A metadata review includes both checking for compliance with metadata standards by using a recommended metadata validation tool and performing quality checks. A minimum of one metadata review by a qualified reviewer is required for all USGS scientific data approved for release. A written report of all metadata reviews (reviewer comments and how they were reconciled) must be included in the internal IPDS review package that is submitted for Bureau approval as described in SM 502.7 and Release of Scientific Data.
The role of the metadata reviewer is to evaluate the accuracy, completeness, and usability of the metadata. The metadata review can be conducted as part of the peer review (SM 502.3) or data review (SM 502.8), or it can be conducted separately as appropriate. Science Center management determines who serves as qualified metadata reviewers for the data produced by authors in their Centers. A metadata checklist that provides guidelines to reviewers is available. Additional information on metadata reviews is available on the USGS Data Management Metadata Web page.
6. When do I create metadata?
A metadata record needs to be finalized and disseminated when the data are ready to be released to others. Authors should develop an approach for compiling the metadata record at the data-management planning stage as described in SM 502.6. Metadata are collected, used, and revised as a living resource throughout the data lifecycle. Therefore, metadata creation should be started as soon as the project begins. When recorded throughout the lifecycle of data, the metadata information is likely to be more accurate and will require less effort than if it is recorded at the end of the project. The metadata information must be updated periodically to document any changes to the data, such as corrections and additions.
7. I have a lot of data packaged in different datasets and databases. For what packages of data do I produce a metadata record?
It depends on how the data will be used. You need a metadata record that describes the data package that will be cited, which is generally also the package that will be searched for in the Science Data Catalog and public search engines. Additional metadata records might be needed for separate parts of data packages that have different creation or processing details.
8. Are metadata records required for any size dataset?
There is no established size for a dataset that prescribes when a metadata record is required. A separate metadata record may not be needed if only a few sample results are presented in their entirety in a published table. However, if the table contains analytical or summary results, or a larger set of data is used from which a small number of records are extracted to create a table for a USGS series or outside publication, then it is appropriate to also have a metadata record for that larger dataset. When data are released, they must be accompanied by a metadata record. If a product uses a subset or summary of separately released data, no additional metadata record is needed.
9. Are metadata records needed for scientific datasets and databases that are provided by non-USGS authors and are subsequently included in USGS datasets, databases, or publications?
Generally, yes, because these items become part of USGS scientific information products. When incorporated into USGS information products, these datasets or databases from non-USGS sources need to comply with USGS data release requirements, including review and approval of data and documentation of the source data. This is important because metadata records establish the provenance of incorporated data and include a link to the original source data. If sufficient metadata do not exist, record(s) should be created and included as part of the new information product's data package or cited in the metadata for the new USGS data product.
10. Are the output data generated by a model simulation also subject to the metadata requirement?
Yes, model simulation data that will be made publicly available through the data release process need metadata. Source data used for the model should be well documented and cited in the metadata to allow the work to be understood and replicated by others.
11. Do summary data tables in information products such as USGS series or outside publications (for example, journals) need metadata?
No. However, the data behind the summary table, if not also presented in the body of the product, will need metadata and will need to go through the data release process.
12. Is USGS Science Publishing Network (SPN) editing required for metadata records?
No, an SPN editorial review is not required, but Science Centers have the option of obtaining such a review as described in SM 1100.2 for any product as they deem appropriate.
13. Where do the metadata records go once we have created them?
A copy of the metadata record must stay with its associated data. Upon formal release of the data, copies of metadata records for all USGS data-related products, including nongeospatial data, must also be placed in the USGS Science Data Catalog. For more information about how to include metadata in the Science Data Catalog, refer to this web page.
14. If my data are associated with a publication, where does the Digital Object Identifier (DOI) pointing to that publication get placed in the metadata?
The DOI for the associated publication should be placed in the “Cross_Reference: Citation: Online_Linkage:” element of the metadata record. To be an actionable link within the metadata record, the DOI must follow this format example: https://doi.org/10.3133/sir20155010.
15. If I cannot explain how the data were created in the metadata record, where do I place the URL or DOI for the product that describes the data creation process?
This information should be added to the “Supplemental_Information:” element of the metadata record. To be an actionable link within the metadata record, the DOI, if used, must follow this format example: https://doi.org/10.1029/2005WR004455.
16. Where can I find additional guidance or information about metadata?
Additional guidance on metadata creation, quality control and content review, tools, and best practices is available on the USGS Data Management and Metadata Web pages.
Note: Terms used in these FAQs will be referred to by their acronyms (in parentheses) as follows: U.S. Geological Survey (USGS), Survey Manual (SM), Instructional Memorandum (IM), Office of Science Quality and Integrity (OSQI), Fundamental Science Practices (FSP), Bureau Approving Officials (BAOs), Office of Science and Technology Policy (OSTP), Office of Management and Budget (OMB), Information Product Data System (IPDS), National Water Information System (NWIS), Freedom of Information Act (FOIA), Data Management Plan (DMP).