Landsat Collection 1 Multispectral Scanner Data Processing
Landsat 1-5 Multispectral Scanner (MSS) data have historically proved challenging to Landsat Level-1 processing systems.
Return to Landsat Collection 1 Overview
A significant amount of the MSS data have missing scan lines, and others are not full-data nominal scenes. The effort to process Landsat MSS data into Collection 1 required data processing changes to influence the geometric and radiometric accuracy of the Level-1 data products.
Geometric Accuracy Details
Changes made within the Landsat Processing Systems produce data with better geometric accuracy than in Pre-Collection MSS data. Missing scan lines are now treated more consistently, and more robust outlier rejection will be used when applying the measured offsets found between the systematic imagery and the ground control library, will improve the geometric accuracy for all the MSS formats within the USGS archive.
Missing Scan Lines
An issue with how missing scans were being handled in the creation of Level-0 files was discovered. There are instances where a scanline of data is missing in the raw data stream, or there is a loss of data such that the imagery of a scan cannot be successfully assembled. In Pre-Collection data processing, rather than inserting a fill scan of no data, the scan following the missing scan was inserted in its place – producing a discontinuity because all scans acquired after the missing scan were out of order by one scan.
When the creation of a precision-terrain corrected Level-1 product was attempted, this discontinuity would cause failures in full-scene data registration, as the resulting time discontinuity with respect to what would be a nominal systematic image created two separate image segments with different along-track/time offsets within the Level-0 file. This discontinuity would produce a conflict between the two image segments when measured against the ground control library and cause: 1) a complete failure of the generation of a precision-terrain product, 2) partial-scene data registration, or 3) a “splitting of the difference”, where only one portion of the scene would be registered to the ground control.
In MSS Collection 1 processing, a full scanline of no data is created. While this causes a scan line of no-data across the scene, this change improves the geometric integrity of the data and allows for better ground registration. The Figure 1 displays the differences between the Pre-Collection and Collection 1 processing.
Image Registration Improvements
An example of having only a portion of the image registering can be seen in in the image below. The colors displayed show the residuals for the Pre-Collection data having a strong variable geometric change in the along track direction for the precision-terrain image. At times the Pre-Collection image shows a registration accuracy of less than half a pixel, while at other locations the registration error is greater than 2 pixels. This is demonstrated by the green-cyan-blue colors within the image.
The Collection 1 data residuals contain only the green and cyan colors indicating a post-fit registration of better than one pixel to the GLS image.
The Gverify Root Mean Squared Error (RMSE) results split into four equal quadrants of the images are shown below each image. These results show an improvement of almost 50 percent with the Collection 1 Level-0 file. The results were derived from the Level-0 files shown in the Figure 1.
The Geodetic Accuracy reports of the RMSE and standard deviations of the precision models’ fit to the ground control is shown in Figure 3. These results belong to the Level-0 and GVerify results shown in Figures 1 and 2. Although the difference between the Pre-Collection and Collection 1 results differ by about only 2 meters (with respect to the post-fit RMSE and standard deviation), the fact that the Collection 1 report shows that almost 3 times as many GCPs are used in registration indicates a much larger area of the image is well registered to the ground control for the Collection 1 image.
The frequency of the missing scan issue within the MSS archive is unknown. During investigations and analysis of the issue there were not any clear patterns with respect to either mission, acquisition time, or geographic location. The issue occurs in some fraction of the MSS-R Level-0 format data, so for some of these products a fill scan(s) may be present and the registration results may significantly improve for Collection 1.
MSS Precision Outlier Rejection
The processing system for Collection 1 MSS data requires changes to the outlier rejection logic in the precision correction step that alters the way it handles the measured offsets between the systematic image and the ground control library.
Prior to Collection 1 processing, measured offsets between the systematic image and the ground control library were identified as outliers and removed based upon a simple standard deviation check which had a very large threshold setting. This very conservative approach allowed large offsets to be included in the precision correction solution.
The Collection 1 outlier rejection will be done using a student-t check that is similar to the method used with the other Landsat sensor precision correction processes (OLI, TM, ETM+).
Analysis comparing the differences in the geometric accuracy using this new outlier logic indicates that more MSS data will successfully complete the precision correction step and will improve the post-fit residuals of the precision-terrain products in the majority of data products generated.
Figure 4 displays Gverify results for a Pre-Collection precision-terrain image compared to the Collection 1 image. The Gverify RMSE results, split into four equal quadrants of the images, show an improvement of almost 36 percent in the Collection 1 image.
The Geodetic Accuracy report in Figure 5 displays the RMSE and Standard Deviations of the precision models’ fit to the ground control is the results of Figure 4.
The differences in the post-fit results between the Pre-Collection and Collection 1 precision corrections shown in Figure 5 demonstrate how the larger number of points kept for Pre-Collection comes at a cost to the post-fit accuracy of the image.
For the Gverify results displayed in Figure 4, a similar number of validation points is displayed, however the colors for the Collection 1 validation points show a much better fit for the Collection 1 image. The exception to this would be the south-west quadrant of the images, where the Collection 1 results show fewer points kept.
Visual inspection indicates that the Collection 1 image is better registered to the GLS image than the Pre-Collection image; however, the landscape changes within this region for the MSS and validation scenes made even a visual inspection difficult.
To further investigate the changes brought on by the new student-t outlier rejection step and the missing scan issue, 152 scenes from a variety of MSS formats were processed using the Pre-Collection Level-0 file outlier rejection and the Collection 1 Level-0 (student-t) outlier rejection. The Geodetic Accuracy results for these data sets are shown in Figure 6.
Within the Geodetic Accuracy-Radial plot, the Pre-Collection scenes seem to fall into three distinct areas which represent:
- Scenes that barely met a 120-meter threshold (values above 120-meters will cause the precision solution to be declared failed and processing will fall back to generate a systematic product).
- Scenes that were considered as good products with 60-meter or less post-fit residuals.
-
Scenes that failed to produce a precision solution by falling above the 120-meter threshold mentioned above.
The premise behind this split was to see how the changes would affect the scenes that were considered to be border-line of being well registered, those that were well registered, and those that failed to register at all.
In the GVerify-Radial plot, Collection 1 processing shows that the new outlier rejection improves the post-fit results, with the exception of a few scenes, while creating a precision-terrain corrected product for all but three scenes that otherwise would have fallen back to a systematic product.
The last of these items is shown by the post-fit residuals having values greater than 120-meters Pre-Collection but less than 120-meters for Collection 1.
It is important to note that the frequency of the missing scan issue was not determined for the scenes in Figure 6, and that data sets were chosen as images that would register: cloud-free and no historical registration issues, with respect to the WRS-2 path/row location or other geographic-type dependencies.
At the time of analysis, among the 152 scenes chosen to represent the percentage of the data sets within the archive as a whole, the number of scenes for each MSS format type was chosen:
MSS-R: 111
MSS-A: 23
MSS-X (Orphan) ORF: 6
MSS-P: 5
CCT-X: 4
MSS-X (Wideband Video) WBV: 3
The missing scan issue would only affect the MSS-R format, as it is the most numerous format within the Landsat archive and has continued to grow since the study.
The results shown in Figure 6 demonstrate that there is expected to be improvement in all the MSS formats and the three classes of registration defined (border line registered, well registered, and those that failed to register).
As a final analysis for geometric changes associated with MSS and Collection 1 processing, the results from 197 random scenes that registered using the Collection 1 processes were compared against those using the Pre-Collection processes.
Probability density plots for the Geodetic Accuracy radial RMSE for Pre-Collection and Collection 1 are shown in Figure 7. The overall mean and standard deviation of the radial RMSE for the Collection 1 and Pre-Collection results were 21.41 / 7.01 and 45.28 / 41.04 meters respectively. These results demonstrate that not only is the Collection 1 better registered overall and more consistent than the results for Pre-Collection. While the 197 random scenes are a very small percentage of the MSS data archive, it is indicitive that the number of scenes adversely affected by the Collection 1 changes will be very small while the benefit with these changes in the increase in the number of precision-terrain data sets generated and the improved post-fit residuals will be significant.
Radiometric Accuracy Details
Changes made within the Landsat Processing Systems produce data with better radiometric accuracy than in Pre-Collection MSS data. The radiometric updates applied
Radiometric Recalibration
MSS data initally downloaded to U.S. ground stations were radiometrically processed on various systems and archived in different formats. As radiometric algorithms and processing parameters evolved, differences in radiometric calibration among the different formats of MSS data were found.Since 2010, the Landsat MSS archive has been largely increased by raw, unprocessed MSS data (MSS-R format) received from International Cooperators through the Landsat Global Acquisition Consolidation (LGAC) effort.
To ensure consistent calibration among different data formats, a recalibration approach has been formulated. First, lifetime look-up-tables (LUTs) of per-day gains and biases were estimated, based on the information extracted from calibration wedges available with the raw MSS data. Scene average gains and biases of any scene processed through historical processing system, if available, were then recalibrated using these LUTs to account for calibration variability over time. Similarly, the scene average gains and biases of any raw scene were also adjusted to the LUT gains and biases.
In addition, cross calibration between any MSS data and MSS-R data was performed to remove any residual differences and to bring all MSS data formats to the same radiometric scale.
Radiance Calibration
MSS radiance calibration was updated to incorporate the recalibration approach described in the previous section. The calibration of Landsat 5 Thematic Mapper (TM5) was also updated for Collection 1, thus requiring a revised TM-to-MSS cross-calibration. A sequential cross calibration, using the newly radiometrically recalibrated MSS data, was performed in radiance space from TM5 to MSS5, from MSS5 to MSS4, and so on through MSS1 to maintain consistency throughout the Landsat Collection 1 archive.
To get the benefit of this updated calibration, users can convert Collection 1 Level-1 data to updated calibrated radiance by applying the "RADIANCE_MULT" and "RADIANCE_ADD" parameters provided in the metadata file (MTL.txt) that is delivered with the Level-1 product.
Figure 8 shows temporally-trended red band (Band 6 for MSS 1-3; Band 3 for MSS 4-5) top of atmosphere (TOA) radiance (normalized by Earth-Sun distance and solar zenith angle) of a region of interest (ROI) in the Sonora Desert. TM5 and MSS1 through MSS5 radiances are plotted with adjustments applied for spectral band differences between respective sensors.
The table below lists the average percent changes in TOA radiance between Pre-Collection and Collection 1 MSS data per sensor. The average percent change in TOA radiance is estimated from a lifetime trend of the Sonora desert ROI. A positive change indicates that Collection 1 data will be brighter than Pre-Collection data.
Reflectance Calibration
The MSS reflectance calibration has been updated to take the advantage of recalibration approach and to tie MSS reflectance to the superior Landsat 8 Operational Land Imager (OLI) reflectance calibration.
The process involved a sequential cross-calibration approach in reflectance space to transfer OLI reflectance calibration to MSS. The reflectance based cross-calibration gains were then used to derive MSS band specific effective exo-atmospheric solar irradiance (ESUN) values. The table below lists effective ESUN values used in calbration parameter files (CPFs) to implement the reflectance calibration update.
Like Landsat TM and Landast ETM+, Landsat MSS Collection 1 Level-1 data can be converted to updated calibrated reflectance by applying the "REFLECTANCE_MULT" and "REFLECTANCE_ADD" parameters provided in the MTL.txt file. (The reflectance calibration of TM and ETM+ data was updated with the release of Collection 1 products in April 2017.)
Figure 9 shows temporally-trended red band TOA reflectance of a ROI in the Sonora Desert. TM5 and MSS1 through MSS5 data are plotted with adjustments applied for spectral band differences between respective sensors and Landsat 8 OLI.
The table below lists the average percent changes in TOA reflectance between Pre-Collection and Collection 1 MSS data per sensor. The average percent change in TOA reflectance is estimated from a lifetime trend of the Sonora desert ROI. A positive change indicates that Collection 1 data will be brighter than Pre-Collection data.
Documentation
Geometric
Landsat 1-5 Multispectral Scanner (MSS) Geometric Algorithm Description Document (ADD)
Landsat Multispectral Scanner (MSS) Level 1 (L1) Data Format Control Book (DFCB)
“Assessment of the NASA-USGS Global Land Survey (GLS) datasets”, Remote Sensing of Environment, Gutman, G., Huang, C., Chander, G. Volume 134, July 2013, Pages 249-265.