Types of Software Review

Administrative Security Review

All software must have an administrative security review before it is made publicly available by any method. This type of review ensures personal, private, or otherwise sensitive information is not included in the repository. Types of sensitive information include:

• Personally identifiable information (PII)

• Absolute file system paths

• Internal server host names or IP addresses

• Usernames/passwords

Administrative security reviews may be performed by any trusted person; the reviewer does not necessarily need a strong scientific or programming background. When migrating an existing project into any non-private Git repository, it is important to remember that the entire project history needs to be reviewed if that history is to be maintained after migration.

Administrative security reviews must be applied at Scientific Software stages:

• Preliminary software

• Approved software

Code Review

Code reviews ensure structural code quality and should be performed frequently throughout the stages of software development. This might mean different things depending on the individual project/team but some typical quality checks include:

• Coding standards

• Unit tests passing

• User input cleansing

• Memory leaks

• Vulnerabilities

• Optimizations

Code reviews should be performed by a trusted developer with applicable programming knowledge. This person need not have thorough subject-matter knowledge on the science topic.

Tools such as linters can automate parts of code review, freeing the human reviewer to focus on the substance of the code. For more information, refer to the automation section on the (Refer to Test).

Code review must be applied at Scientific Software Stages:

• Approved software

Domain Review

Domain reviews are specific to Scientific Software (Refer to Scientific Software IM) and ensure the software generates output that aligns with published or otherwise well-known expected results. Domain reviews may involve:

• Comparing output with external data sets

• Comparing algorithms with published, scholarly articles about the algorithm

• Reviewing unit and integration test results

Domain reviews should be performed by a trusted scientific expert with applicable subject-matter knowledge as well as sufficient applicable programming knowledge.

Domain reviews must be applied at Scientific Software stages:

• Approved Software

Review with Git

While there are multiple ways to perform and document the review process, one option is to use GitLab itself. Documenting the review process in GitLab can be accomplished using GitLab’s Issue feature.

• Make sure your source code is in a GitLab repository accessible by the intended reviewer.
• Create an issue in your repository project. An example title and description might be Software Release Review, requesting an administrative security review of the software.
  • Include a link to the specific source code files if multiple projects or folders exist in the repository.
  • Add a label to the issue indicating the type of review being requested. Create your own label such as “security review”.
• Assign the issue to the reviewer.
  • The reviewer will reply to the issue with review comments and notification that the review has been completed. This may involve iteratively responding to the reviewer through the Issue feature and updating the repository to resolve review comments.
• Optionally, reviewers may also create a branch of the project for review and make suggested changes directly in the source code using a merge request.