Software Management

Develop Software

Good code writing practices are the foundation to well documented and reusable code of any type. However, the intent of this website is not to teach code writing skills, rather the content is focused on tools and best practices supported by the USGS. For community discussions on code writing best practices consider joining these supported groups.

USGS DevOps Working Group

USGS DevOps Working Group

The USGS DevOps Working Group shares new techniques and lessons learned using DevOps tools and methods.

Learn More

USGS Software Development Cluster

USGS Software Development Cluster

The USGS Software Development Cluster is a community for USGS software developers and other interested parties to discuss software development best practices and policies.

Learn More

Source Control

Source control provides a logical way to organize and control collaboration, revision, and release of source code. As a best practice, private or sensitive information (including but not limited to: personally identifiable information (PII), absolute file system paths, internal server host names or IP addresses, and usernames/passwords) should always be extracted from the source code and maintained separately.

Note: Git maintains a repository history and any commits containing sensitive information must be purged from the repository history. Refer to the Git Help on Rewriting History.

Where should I put my source code?

The USGS only supports Git for source code control. There are three unique hosting platforms for Git available to USGS employees. For more information contact GS Help Git (gs_help_git@usgs.gov).

Each platform component provides similar but distinct functionality. When deciding which component to use, please see the following information:

USGS GitHub: USGS Organization on GitHub [EXTERNAL]

This component is a commercial off-the-shelf hosting solution provided and hosted by GitHub. GitHub supports integrated issue tracking, project boards, and third-party integrations among other features. This component is publicly accessible. Private repositories are not available in this component. Any person – regardless of USGS affiliation – can create a GitHub account and collaborate on USGS projects by submitting pull requests.

This component is a commercial resource governed by its terms of service as amended for government use.

USGS OpenSource GitLab

This component is a commercial off-the-shelf installation of GitLab Community Edition (CE) provided and hosted in the cloud by the USGS Cloud Hosting Solutions (CHS) team. This component supports integrated issue tracking and project boards among other features. Some listed features may not be enabled in the “OpenSource GitLab” component.

This component is publicly accessible from anywhere on the internet. Projects identified as “Public” within this component are viewable by all users regardless of authentication. Projects identified as “Internal” within this component are viewable by any authenticated user. Projects identified as “Private” are viewable only by authenticated and authorized users. Default project visibility is “Internal”.

All USGS personnel automatically have accounts on this system. Simply log in with your Active Directory credentials or smart card. External collaborator accounts may be requested by a USGS sponsor and require the collaborator use two-factor authentication through a free third-party application (or open source alternative). Authorization is managed through project, user, and group permissions assigned by system and project maintainers.

This component is hosted in the Amazon Web Services (AWS) Cloud and leverages AWS managed services to provide the highest available up-time and recovery. Data stored on this component are backed up nightly. System and application patches and updates are applied monthly.

Note: USGS Scientific Software Policy - All authoritative copies of Scientific Software intended for approved release must reside here.

USGS InnerSource GitLab

This component is a commercial off-the-shelf installation of GitLab CE provided by GitLab and hosted in the cloud by the USGS CHS team. This component supports integrated issue tracking, project boards, container image registry, and shared task runners among other features. Some listed features may not be enabled in the “InnerSource GitLab” component.

This component is only accessible from machines bound to Active Directory (AD), and while on the DOI network or using the Pulse VPN. Projects identified as “Public” within this component are viewable by all users regardless of authentication. Projects identified as “Internal” within this component are viewable by any authenticated user. Projects identified as “Private” are viewable only by authenticated and authorized users. Default project visibility is “Private”.

All USGS personnel automatically have accounts on this system. Simply log in with your Active Directory credentials or smart card. Authorization is managed through project, user, and group permissions assigned by system and project maintainers. External collaborator accounts are not supported.

This component is hosted in the AWS Cloud and leverages AWS managed services to provide the highest available up-time and recovery. Data stored on this component are backed up nightly. System and application patches and updates are applied monthly.

 

Start Developing

While actively developing a project, you should follow an "Integration Manager Workflow." Using this technique works especially well when collaboratively developing a project and it works for individual developers as well. This technique encourages using pull requests to integrate code changes. Pull requests facilitate team review and change history tracking as you actively develop software. Recommended references:

Continuous Integration (CI) refers to advanced individual or team methods for integration management. When collaboratively developing a project, each member pulls from and pushes to a remote repository. During this time, not only can the existing code base change, but new code can be added as well as new libraries, and other resources that create dependencies and potential conflicts in the main line (or master branch) of code in the repository. CI ensures when developers submit code to the repository they must first update their code (i.e. rebase it) to reflect the new changes in the repository since they checked out their copy before integration into the main line of code can occur.

There are many possible tools to implement CI. The most popular is Jenkins, however,Others like Travis and CircleCI have gained a wide following for teams who do not want to administer their own CI environment.