The U.S. Geological Survey’s Landsat 7 Flight Operations Team achieved nothing less than a major milestone in mid-July of 2020. For eight straight years, they had not one operator error. Not one.
Granted, there’s a lot of automation involved with this multi-million-dollar piece of machinery. But operators have to ensure the satellite avoids space debris. They’ve had to adjust orbits. Every day, they are uploading countless commands directing the satellite on what to do while also checking on its health.
Now, add to that the challenge of moving their entire operations from one building to another on NASA’s Goddard Space Flight Center campus in Greenbelt, Maryland, in the last few years. And then there’s the ongoing coronavirus pandemic that is forcing most of them to work from home.
In the most recent Eyes on Earth podcast, Guy Thayer with The Aerospace Corporation, who is the Landsat 7 Flight Systems Manager, discussed those challenges, and why eight years of error-free operations is worth celebrating. Here are a few highlights from that conversation.
Tell us why straight years of no operator errors flying Landsat 7 is a big deal.
“Well, it’s a really big deal because the operator errors are pretty easy to make. So, having this amount of time go by and this many activities happen and not have an error is a truly large achievement.”
How do you operate a satellite?
“It’s mostly done through files and commands that we send. But those files have to be created and we’re talking about errors. That’s a place we could easily make an error.”
When you talk about files, do you send up hundreds or thousands of commands every day?
“The long and short of it is, we set up two different loads, we call them, of command loads every day, 365 days a year. One tells the satellite what to do and when to do it. And the other one is the ephemerous load that tells it where it’s going to be during that period of time. The operators have to go in and create that load every morning and then throughout the day they observe the satellite and make sure that it is operating from that command load.”
Does operating Landsat mean simply keeping it in its orbit, or what else do these operators have to do to keep it performing the way it should?
“Obviously, keeping it in its orbit is a big part of the job. Typically, we would have to send commands to raise the orbit of the spacecraft as it encounters drag, but lately we haven’t had to do very much of that in the last couple of years. The bigger work is the everyday work of creating those loads and putting them up on the spacecraft. The main part of the job is reviewing the information we get from the spacecraft and making sure that it is healthy.”
What do you mean by healthy?
“Power is a big deal to us, obviously. We have to generate and store power so we can operate the spacecraft. So, we need to closely monitor the output of the solar array, and the health of the batteries, that we can monitor by watching the temperature and the pressure of the batteries. We look at the attitude of the spacecraft to make sure that it is working within specifications. And we look at, for example, temperatures of gyros and how much power they are drawing to make sure that they are working properly. Unlike your car for instance, you can hear squeals and knocks and what have you, right? But with the spacecraft, we have to ascertain what’s going on by looking at telemetry, by looking at data points.”
What kind of challenges with Landsat 7 really acquire your attention?
“It is an older satellite. Most things that break tend to break early. We’ve kind of settled into a rhythm. So, what are the actual challenges? The regular challenges that every satellite operator faces. I don’t want to call it monotony, but it is maintaining focus on keeping the thing running, doing many similar things every day. So, complacency is actually one of the challenges that you have. We’ve been very, very lucky, knocking on wood, that we haven’t had very many major challenges in the last several years. I mean the solar array slowly degrades. And in our case, our angle to the sun is slipping based on the fact we are no longer doing inclination maneuvers to maintain our inclination because we don’t have enough fuel to do so. So, that is a challenge, trying to figure out going forward if we’re going to be able to generate enough power.”
How often do you have to worry about something hitting the spacecraft?
“Not very often right now. Remember, I mentioned a little while ago about the fact that the drag environment is fairly benign right now?”
If you get an alarm, and a light goes off or something that would suggest that there’s an issue going on, how long do you have to figure that out before it becomes a major problem?
“That is a big “it depends” question. What the issue is will determine how long we have to react. For example, let’s say the satellite went into safe hold, which, knocking on wood again, we’ve never done. It is safe, and it is stable. We can sit there almost indefinitely and not have to work too quickly, which would make us happy because we don’t like to do anything quickly. But, for example, if the solar array were caught in the wrong angle and were stuck, we would have to act very quickly within a couple of hours to start getting that thing fixed.”
Do you have moments that get your blood pumping?
“We absolutely do. In the last eight years, we’ve had a couple of different things happen. For example, we had a clock generator that is an esoteric piece of equipment that meters the clock output. We never think of it; it’s buried in there. It’s not something that we can command. One day, it just kind of switched from the primary piece of equipment to the redundant piece of equipment. For the life of us, we never could figure out. That is jarring, when something like that happens, and you don’t know why it happened. That gets your blood going because then you start worrying about the cascading effect of “What if this? What if that?” If you don’t know what caused something to happen, it makes you very nervous about what’s happening on the spacecraft. But, other than that, we’ve been pretty darn lucky.”
Have you ever heard of operator errors that have occurred with other satellite systems?
“Well, let me tell you about one. It was easy for me to remember because it was literally my first day representing USGS with Landsat 5, actually. The command load for that was a two-part command load. You send up what you want the spacecraft to do and then you would have to send up another file that told the spacecraft when to perform those things. The dilemma was the operator, somehow, mistakenly sent up two of the command files without the time stamp file. Now, normally that wouldn’t cause any problem at all, right? It would be gobbledygook. It wouldn’t make any sense to the spacecraft. But, it just so happens that the satellite gods were not smiling on us that day and one of the commands that correlated with the command load, the satellite read as an actual command. The next time the spacecraft was supposed to come into contact with the ground system, we didn’t get contact. If you want to know what gets your blood pumping, that’s what gets you going. When it comes over the hill and you don’t see it. We figured it out quickly, thank goodness. We got things right within a day but that’s how easy it can go downhill quick.”