Sources/Usage: Public Domain. View Media Details

But with observations collected every eight to 16 days, there are limits to what can be tracked. On the most recent episode of the USGS EROS podcast Eyes on Earth, listeners had the opportunity to hear about a provisional and soon to be science quality data product that merges Landsat data with data from the European Space Agency’s Sentinel-2 satellites.

Harmonized Landsat-Sentinel (HLS) data will be available through the Land Processes Distributed Active Archive Center (LP DAAC), which is located at EROS. The new data set will offer more opportunities to monitor rapid change and be available in a cloud-friendly format at no cost to users.

The guests on Episode 45 were Landsat Science Team member and Landsat Project Scientist Jeff Masek, who wrote the algorithm that produced the HLS prototype, and Brian Freitag. His team at the Interagency Implementation and Advanced Concepts Team (IMPACT) project at NASA Marshall Space Flight Center worked to scale up HLS and make it friendly for use in the cloud.

Visit the Episode 45 show page to download the audio file and access links on how to get started with HLS. Visit the Eyes on Earth main page to access every episode.

Here are the highlights from the conversation with Masek and Freitag, edited for length and clarity.

EYES ON EARTH (EoE): Let’s talk a little bit about Landsat and Sentinel. How are these satellite systems similar, and how are they different?

JEFF MASEK (JM): From an applications perspective, both Sentinel and Landsat serve a lot of the same needs. They’re used to look at land characteristics and land change worldwide—things like global agriculture, natural disasters, changing icecaps, urbanization—basically any time you need to get down to a spatial resolution fine enough to see human activities and the consequences of human activities.

That said, there are some technical differences between the systems. Landsat sees objects on the ground as fine as about 30 meters. Sentinel-2 is a little bit finer resolution, about 10 meters. They have a lot of spectral bands in common, but the band placement is a little different. They’re also in different orbits, so they’re observing the same target with slightly different viewing geometry, and again, that affects the brightness of the imagery that comes out for a particular target.

EoE:  So this would be like the difference between, say, two people taking a picture of a stop sign from 50 yards away. If one person is standing directly in front of it and the other person is sort of standing to the left, there’s going to be a slight difference there?

JM: Exactly. If you’ve ever looked at a lawn with the sun behind you, it’s really bright. Then if you turn around and look at it with the sun in front of you, it’s really dark because you are looking at shadows. So the angle really affects the overall brightness, and what we call the spectral response.

EoE: So those are the differences we are looking at. Different angles, different spectral bands. With Landsat we have a thermal band as well, right?

JM: Correct. That’s another difference between the two systems. Landsat has the thermal infrared bands, where we can look at surface temperature. Sentinel-2 does not have those bands. That’s a unique aspect of Landsat that we only get from that system.

EoE: Can we talk a little bit about what Landsat and Sentinel imagery is used for today?

JM: Landsat is our longest-lived remote sensing system. It first launched in 1972, so we have going on 50 years now of a long-term record of the Earth’s land environment. So we can look at deforestation, changes in agriculture and irrigation, and changes in glaciers and ice caps. In addition, both Sentinel and Landsat are used for land management. If you want to understand what crops are growing where, what areas need more irrigation or what areas need more fertilizer, both Landsat and Sentinel data are used for that, as well.

EoE: What is Harmonized Landsat Sentinel Data?

JM: We’re trying to bring the two data sets together and do the necessary geometric and radiometric adjustments so they can be used completely interchangeably. What that means in practice is that we take the input data, map them to the same grid, and map them to the same 30-meter spatial resolution so they physically overlay each other. Then, we adjust the spectral values—the radiometry—so they look like each other. We do the angular correction and the band pass corrections to produce products that are easy for people to use.

EoE: What would that allow people to do, and why can’t we do those things now? What stands in the way?

JM: In the case of Landsat, we only get an image every 16 days, or eight days if you have two satellites in orbit. There’s a lot of phenomena that vary on time scales that are shorter than 16 days or eight days. You could look at an inundation event, the progression of a fire, crop harvesting, issues of water quality. What you really want is imagery that captures that rapid variability. We don’t have that from Landsat alone. So the idea of Harmonized Landsat Sentinel is you bring both data sets together and you end up with an observation at that high resolution every two to three days, which is phenomenal.

EoE: So instead of seeing something every five days with Sentinel or every eight to 16 days with Landsat, we are looking at a three-day window. Are there other operations or applications where that would be especially useful?

JM: The big one is agriculture. For a bunch of reasons, you want to look at croplands much more frequently than we can look at with Landsat throughout a growing season. If you can characterize the phenology curve really well, that tells you a lot about what’s growing, but it also tells you how well it is growing. That leads right into information that’s relevant for agriculture management. Should I water? Should I fertilize? Should I plant a particular crop here this year?

EoE: Brian, what have you been working on to make this happen?

BRIAN FREITAG (BF): Jeff started developing HLS in 2014 as a beta production in a targeted area. They were doing most production over North America, with some other regions around the globe, smaller areas that were of interest to different project scientists. As part of the Satellite Needs Working Group, NASA goes back to survey a number of federal agencies for what they needed to perform their operational tasks. HLS was identified from the 2016 cycle as something that would be really useful, not only over North America and the United States, but as a global product.

Starting in 2017-2018, NASA’s perspective shifted from using this targeted domain that Jeff had used previously—they wanted to expand that out to global. Global production really kind of started in 2019. And that is where my group, the IMPACT (Interagency Implementation and Advanced Concepts Team) Project, started to assist in taking the algorithm and scaling it up.

EoE: Tell us a little bit more about the IMPACT Project. Is there a simple way to explain what you’re doing there?

BF: IMPACT is located at Marshall Space Flight Center, and our role is to support NASA’s Earth Science Data Systems. (NASA) tasked us with taking the existing algorithm that Jeff and Junchang Ju had developed at Goddard, moving it into the cloud, and then making it more cloud friendly so we could get an idea of what that would look like. Typically, when you have an algorithm and you’re running it on a high-performance computing (system), it’s a block algorithm. You kind of just shove it through, you get input/output, and you have status: success or fail. What we wanted to do was basically benchmark the different steps of the algorithm so we could track its performance through time by separating out the different components. It is almost like you are taking a full cake and breaking it down into individual ingredients. Then you’re able to check each individual ingredient as you develop the cake, so in the end, the cake that you have from the new process matches the cake that you had before.

EoE: So Harmonized Landsat and Sentinel will be a situation where (a user) would search for a particular spot on the globe and see Landsat and Sentinel images that are interchangeable?

BF: Right now, we have two data products. We have the Landsat component and the Sentinel component. NASA has a nice visualization platform (NASA Worldview) where you can have these data layers that give you “quick look” browse imagery to see what a scene may look like. But then also you have the general full constellation of what the two data products look like combined. As you focus on a particular area as a function of time, you will see that two- to four-day revisit period.

JM: People often ask what we mean by harmonized. What we are talking about is that to the user, it doesn’t matter to them which satellite the observation comes from. They can use the observations interchangeably for the common bands.

Sources/Usage: Public Domain. View Media Details

EoE: What do you hope to see done with this HLS data? What does success look like five years from now, from your perspective? Daily deforestation alerts in the Amazon? Weekly algae bloom updates? Urban heat island tracking?

JM: Yeah, all of that. I’d like to see HLS or spin-offs from it incorporated into real land management activities. Like having the U.S. Department of Agriculture downloading this on a regular basis and doing crop forecasts and crop type analysis. To see the U.S. Forest Service look at the daily scale of recovery from a forest fire. That to me would be the ultimate payback in the work. Because then people are really using it, and using it to its fullest extent.

BF: I’d like to see the number of users increase, and to see users come in and learn how to access the data through the cloud and shift their workflows. As we send out these data access surveys, seeing people shift from the https-based downloads to cloud-based analysis. That shift over the next five years is something we really hope to pioneer with HLS.

JM: I would also mention the international aspect, a little bit of a “kumbaya” moment. We have the opportunity to bring together data from multiple international programs. You’re going to be seeing more and more of that, because international remote sensing programs are becoming more and more diverse and robust. There’s that opportunity to combine data from these programs, to increase the frequency of observations, or to do new types of observations. So, HLS is very much in that mode, and I think that’s pretty exciting.