Software Management

Types of Software Review

Scientific Software

Scientific Software

1. Required: Administrative Security Review

2. Required: Code Review

3. Required: Domain Review

Non-Scientific Software

Non-Scientific Software

1. Required: Administrative Security Review

2. Recommended: Code Review

3. Not Applicable: Domain Review

It is a best practice to review all software before it is made publicly available by any method. The content of this page describes general guidelines and different types of reviews that have the intent to improve the quality of your software before it is released.

Administrative Security Review

All software must have an administrative security review before it is made publicly available by any method. This type of review ensures personal, private, or otherwise sensitive information is not included in the repository. Types of sensitive information include:

  • Personally identifiable information (PII)

  • Absolute file system paths

  • Internal server host names or IP addresses

  • Usernames/passwords

Administrative security reviews may be performed by any trusted person; the reviewer does not necessarily need a strong scientific or programming background. When migrating an existing project into any non-private Git repository, it is important to remember that the entire project history needs to be reviewed if that history is to be maintained after migration.

Administrative security reviews must be applied at Scientific Software stages:

  • Preliminary software

  • Approved software

Code Review

Code reviews ensure structural code quality. This might mean different things depending on the individual project/team but some typical quality checks include:

  • Coding standards

  • Unit tests passing

  • User input cleansing

  • Memory leaks

  • Vulnerabilities

  • Optimizations

Code reviews should be performed by a trusted developer with applicable programming knowledge. This person need not have thorough subject-matter knowledge on the science topic.

Tools like Codacy and various other linters can automate parts of code review, freeing the human reviewer to focus on the substance of the code. (Refer to Test).

Code review must be applied at Scientific Software Stages:

  • Approved software

Domain Review

Domain reviews are specific to Scientific Software (Refer to Scientific Software IM) and ensure the software generates output that aligns with published or otherwise well-known expected results. Domain reviews may involve:

  • Comparing output with external data sets

  • Comparing algorithms with published, scholarly articles about the algorithm

  • Reviewing unit and integration test results

Domain reviews should be performed by a trusted scientific expert with applicable subject-matter knowledge as well as sufficient applicable programming knowledge.

Domain reviews must be applied at Scientific Software stages:

  • Approved Software