502.3 - Fundamental Science Practices: Peer Review

11/03/2016

OPR: Office of Science Quality and Integrity

Instruction:  This replaces Survey Manual (SM) chapter SM 502.3 - Fundamental Science Practices: Peer Review, dated December 16, 2011.

1.  Purpose and Scope.  Peer review, as a cornerstone of scientific practice, validates and ensures the quality of published U.S. Geological Survey (USGS) science.  The Fundamental Science Practices (FSP) requirements for peer review of USGS information products apply to all USGS scientific information, whether published by the USGS or an outside entity.

2.  References.

A. Final Information Quality Bulletin for Peer Review (Office of Management and Budget (OMB) memorandum, December 16, 2004)

B. SM 502.1 - Fundamental Science Practices:  Foundation Policy 

C. SM 502.2 - Fundamental Science Practices:  Planning and Conducting Data Collection and Research 

D. SM 502.4 - Fundamental Science Practices:  Review, Approval, and Release of Information Products 

E. SM 205.18 - Authority to Approve Information Products 

F. SM 500.25 - Scientific Integrity 

G. 305 DM 3 - Integrity of Scientific and Scholarly Activities 

H. SM Part 1100 - Publishing 

I. USGS General Records Disposition Schedule, Chapter 1300-Publishing Records 

J. USGS Peer Review Agenda (Web site) 

K. Fundamental Science Practices of the USGS (Web site)

3.  Policy.

A.  Peer review is required for virtually all science information products (SM 1100.1), regardless of media, whether published and disseminated by the USGS (SM 1100.3) or by an outside entity (SM 1100.4) if the author has full time, part time, or volunteer (including emeritus) USGS affiliation or shared affiliation (for example, between the USGS and a university). 

B.  The USGS defines peer review (also referred to as technical peer review, refereeing, or scientific peer review) as scrutiny of work or ideas by colleagues (peers) who are qualified.  In the scientific community, the implication is that education or expertise in a particular scientific field qualifies as the prerequisite when soliciting comments on the work of others in that particular field.  Qualified peer reviewers of USGS information products possess the appropriate education or expertise and additionally have no stake in the outcome of the review or publication of the work and are not directly associated with the particular work being performed.  Peer reviewers of USGS information products must not have a conflict of interest (as described in SM 502.4 and SM 500.25).

C.  A minimum of two scientific peer reviews by qualified peers is mandatory for all USGS science information products that require peer review.  Depending on the product and the intended audience, more than two peer reviews may be necessary.  Guidance on other peer review requirements is detailed in section 4.A-F.  All peer reviews must be included with the product package (that is, the revised manuscript with reviewers’ comments addressed, the peer review reconciliation, and all original peer review comments) that is entered into the internal USGS Information Product Data System (IPDS) for Bureau approval. 

D.  Involvement of non-USGS authors does not allow USGS authors to bypass the USGS peer review process.  When a non-USGS author is the lead or senior author and a USGS scientist is a co-author, the USGS scientist must comply with USGS peer review requirements in this chapter or the USGS scientist may not be listed as a coauthor.  USGS scientists with joint university affiliations must also comply with the USGS peer review requirements described herein. 

                                                                

E.  The Final Information Quality Bulletin for Peer Review describes requirements for peer review, particularly for influential scientific information and highly influential scientific assessments (as defined in the OMB Peer Review Bulletin).  While the USGS has had a long tradition of safeguarding both excellence and objectivity of its science by conducting rigorous peer review, the OMB Bulletin provides guidance aimed at enhancing the practice of peer review of all Federal Government science documents.  Information products identified by the USGS as OMB influential require additional planning and consideration with regard to peer review.  For information products that the USGS determines fall in either of the two OMB influential product categories, a minimum of two USGS-initiated peer reviews, supervisor review, and Science Center review are required followed by Bureau approval before the products are submitted to any outside entity for subsequent or additional peer reviews.  Additional reviews are highly recommended for USGS products identified as OMB influential.  Peer reviewers must be informed in writing (as part of their charge) before they begin their review that they will be reviewing influential information.  The peer review planning and summary documentation for influential USGS products is provided on the public USGS Peer Review Agenda.  

F.  To safeguard unpublished USGS scientific information that has not received Bureau approval (refer to SM 502.5), authors must include the nondisclosure disclaimer statement in manuscripts that are to be sent to outside entities, such as peer reviewed journals for review.  This statement as described in section 4.E. explicitly informs the reviewer that the manuscript is being distributed for the purpose of peer review only and may not be disclosed before it receives USGS Bureau approval.  A USGS manuscript submitted to a collaborator/partner for review as a courtesy must carry a courtesy review disclaimer statement.  Refer to SM 502.4 for more information on these disclaimer statement requirements and to Guidance on Disclaimer Statements Allowed in USGS Science Information Products for the peer review and collaborator/partner review statements to be used.

G.  Peer reviewed information products submitted for Bureau approval must include the original comments from all peer reviewers, reconciliation indicating how review comments were addressed, and the revised manuscript after reconciliation (SM 502.4 and SM 205.18).

H.  Outside (or non-USGS) information products or publications (SM 1100.4) used by the USGS include, but are not limited to, scholarly peer reviewed journals and publications by scientific societies, universities, commercial publishing houses or cooperating agencies (refer to SM 502.4, section 6.B).  Specific requirements for peer review include:

(1) USGS information products intended for release in non-USGS outlets (except peer reviewed journals as described in 3.H.(2)), must receive a minimum of two USGS-initiated peer reviews and these reviews are followed by supervisor review, Science Center review, and then Bureau approval. 

(2) USGS information products intended for release in outside peer reviewed journals also require a minimum of two peer reviews.  These peer reviews may be obtained in one of two ways: [1] both reviews are initiated and reconciled by the USGS and Bureau approval is obtained before submittal to the journal or [2] one review is initiated by the USGS, one review is initiated by the journal, and Bureau approval is obtained after all initial peer reviews are reconciled but before returning the reconciled manuscript to the journal.  In the case of the latter, the peer review process for the USGS and the peer review process for the journal may be conducted concurrently or sequentially.  Determining which approach to follow is at the discretion of Science Center management.  Additional reviews may be required depending on the Science Center’s preference (refer to SM 502.1).  In all cases, the USGS must ensure that the journal’s requirements relative to peer reviewer confidentiality are not compromised.

(3) To reiterate, as stated in 3.E, information products the USGS designates in the OMB influential categories require a minimum of two USGS-initiated peer reviews, supervisor review, and Science Center review followed by Bureau approval before the products are submitted to any outside entity for subsequent or additional peer reviews.

I.  USGS defines two categories of abstracts as described in SM 205.18.  For extended abstracts, peer review is required.  For all other abstracts, the need for peer review is at the discretion of the Science Center Director.

J.  Poster sessions and presentation materials that are presented once for scientific meetings, conferences, or hearings and are not left for dissemination or posted on a public Web site do not require peer review.  If these products are left for dissemination or posted on a public Web site, regardless of whether they contain new interpretive material, the need for peer review is at the discretion of the Science Center Director.  Presenters are cautioned that it is inappropriate to display in these materials unpublished information that may be of an OMB influential, sensitive, confidential, or proprietary nature and they must consult with their supervisors and other managers before presenting materials that potentially contain such information.

K.  Although different from peer review, specific quality-related reviews are required for data and software, and these requirements are described in SM 502.8 and IM OSQI 2016-01, respectively.

4.  Peer Review Obligations.

A.  USGS Reviewer Selection.  Peer reviewers of USGS information products may be external or internal to the Bureau.  Peer reviewers should be selected for their relevant scientific and technical expertise.  Peer reviewers will be from outside the project team in which the research was conducted or from outside the USGS as appropriate.  Peer reviewers should not be cooperators or collaborators on the product.  Manuscripts that are to be sent to an outside entity for peer review and that have not received USGS Bureau approval must carry a nondisclosure disclaimer (refer to 3.F and 4.E).  Additional guidance on selecting peer reviewers, particularly for influential USGS information products, is available in the Final Information Quality Bulletin for Peer Review.

B.  Peer Reviews Are Rigorous and Thorough.  Peer reviewers typically evaluate or critique the clarity of hypotheses, the validity of the research design, the quality of data collection procedures, the robustness of the methods employed, the appropriateness of the methods for the hypotheses being tested, the extent to which the conclusions follow from the analysis, and the strengths and limitations of the overall product.  Reviewers should check that methods used to collect data and produce results are defensible and adequately documented (refer to SM 502.2); facts and interpretations are presented straightforwardly, without apparent bias; conclusions are based on the best available data interpreted with sound scientific reasoning that avoids speculation; forecasts and predictions of natural hazards are scientifically sound; and the manuscript is clear in presentation. Peer reviewers should be provided instructions on what content is to be reviewed.  Supplemental guidance on conducting the peer review is found in the USGS Peer Review Checklist (https://www.usgs.gov/media/files/usgs-fsp-peer-review-checklist) or, if applicable, in the USGS Peer Review Checklist for Influential Products (https://www.usgs.gov/media/files/usgs-peer-review-checklist-influential-scientific-information-and), both of which may be offered to reviewers for their use.  These checklists provide a framework for reviewers to summarize their perspectives on various aspects of USGS manuscripts.  

C.  Reviewer Ethics and Conduct.  The USGS pursues vigorous and open peer review of its science and its information products.  Issues related to ensuring scientific excellence, objectivity, integrity, and conflict of interest are dealt with in accordance with the USGS code of scientific conduct (SM 500.25).

D.  Peer Reviews Are Deliberative and Predecisional.  The final peer reviewed and Bureau-approved information product represents the agency’s best scientific interpretation and factual data on the subject at that time.  Peer review of USGS manuscripts is intended to ensure the accuracy of data, the scientific validity of interpretations, and the consideration of alternative interpretations.  The rigorous quality assurance process (including peer review) embodied in the USGS FSP is deliberative because of the iterative exchange of ideas and opinions among the involved parties.  Peer reviews are considered predecisional and may be withheld or extensively redacted because they represent the collective thoughts that are being analyzed in order to arrive at that final product.  The disclosure of an unapproved manuscript and associated peer reviews could allow incorrect or incomplete information to be ascribed to the USGS, and as a result the public could be greatly harmed if such incorrect or out-of-context information were used for public policymaking or resource management (refer to SM 502.5).

E.  Nondisclosure Prior to USGS Approval for Release.  In agreeing to be a peer reviewer for a USGS information product, reviewers must agree to be bound by the strictest scientific ethics in ensuring confidentiality of the science that is being reviewed and to not disclose or divulge any results or conclusions or to make any public statements regarding the science before the science is published.  Information distributed for peer review must carry the disclaimer statement, to be added by the USGS author, concerning nondisclosure prior to the information being approved by the USGS for release.  The disclaimer statement is available at Guidance on Disclaimer Statements Allowed in USGS Science Information Products.  At a minimum, the disclaimer statement must appear on the title page of the manuscript.

F.  Peer Review Documentation and Recordkeeping.  Details about peer review documentation and recordkeeping requirements for USGS information products are described in SM 502.4, section 8. 

5.  Responsibilities.  Compliance with this policy is mandatory for all USGS employees.  Specific additional responsibilities are as follows: 

A.  Associate Directors and Regional Directors.  Associate Directors and Regional Directors set FSP policy for USGS peer review practices and collaborate with one another regarding the content and application of these practices to ensure consistency.  Regional Directors, in collaboration with the Office of Science Quality and Integrity (OSQI), execute and ensure compliance with the peer review policies and practices.  Regional Directors also work with the OSQI to mediate or assist in resolving peer review issues that cannot be resolved directly among Science Center Directors, approving officials, and others.

B.  Office of Science Quality and Integrity.  The OSQI, in collaboration with Regional Directors, executes the policies and practices governing peer review.  The OSQI collaborates with Associate Directors and Regional Directors regarding the content and application of consistent USGS peer review practices and maintains the policy documents and procedures that pertain to FSP.

C.  Science Center Directors.  Science Center Directors ensure that an accepted and consistent peer review process is in place within their unit that meets standards for scientific quality.  If necessary, they contact the Regional Director for assistance in resolving peer review issues.  They ensure that only properly peer reviewed products are forwarded for approval (SM 502.4 and SM 205.18).  Science Center Directors ensure that peer review archival records are placed in the IPDS.  They are responsible also, with concurrence from the Regional Directors and in collaboration with the OSQI, for identifying a given information product as influential scientific information or a highly influential scientific assessment pursuant to the USGS peer review requirements for OMB influential products.

D.  Approving Officials.  Approving officials include Science Center Directors (or their equivalent or designee) and Bureau Approving Officials (BAOs) in the OSQI.  They ensure that USGS standards for scientific quality are followed by confirming that peer review requirements in accordance with this chapter are met before granting Bureau approval of information products for release (SM 205.18 and SM 502.4).  They ensure that authors have adequately addressed peer review comments (that is, written reconciliation is part of the package submitted for approval).  Science Center Directors and BAOs in serving as approving officials may require that an information product receive additional peer reviews as needed to ensure the ultimate quality of the product and, if necessary, may contact the Regional Director or the OSQI Director, respectively, for assistance in resolving peer review related issues.

E.  Authors’ Supervisors.  Supervisors have the initial responsibility to ensure the quality of information products prepared by authors they supervise.  Supervisors are also responsible for the following: 

(1)  Selecting or ensuring the selection of appropriate independent and qualified peer reviewers. 

(2)  Ensuring that the author has adequately addressed peer review comments (before the product is submitted for editorial review, if applicable) and has prepared a final draft of the product that is suitable to submit for approval. 

 

(3)  Informing the Science Center Director about any product content that might be highly visible, sensitive, or controversial and identifying any internal or external groups or agencies that might have particular and/or immediate interest in the product. 

(4)  Nominating, before peer review, information products to the Science Center Director that they deem appropriate for consideration as influential USGS products (refer to 3.E) and, if products so qualify, ensuring that their office coordinates with staff in the OSQI to develop the required peer review documentation about OMB influential products for posting on the USGS Peer Review Agenda.

F.  Authors.  Authors support the peer review process by suggesting or nominating qualified peer reviewers for their own work and the work of other USGS scientists and by participating in peer reviews of the work of others (refer to 4.A-F).  They consult with their supervisor or approving official as needed with regard to the appropriateness of peer reviewer selections.  They objectively consider and appropriately address all peer review comments on information products they author by reconciling them in writing.  Authors forward the product package (revised manuscript with reviewers’ comments addressed, peer review reconciliation, and original peer review comments) to their supervisor. 

G.  Peer Reviewers.  Peer reviewers must follow the requirements in this chapter as part of consenting to serve as a reviewer.

/s/ Jose R. Aragon                                                                          11/03/2016

__________________________________________            _______________ 

Jose R. Aragon                                                                        Date 

Associate Director for Administration