ESRI’s ArcGIS Pro’s Trace Networks & NHDPlus HR Map Service

Video Transcript
Download Video
Right-click and save to download

Detailed Description

National Hydrography Advisory Call:

This episode features speakers Remy Myers (ESRI’s Product Manager for Trace Network) and Kevin McNinch (USGS National Hydrography Team). Remy runs through a demonstration of ArcGIS Pro’s Trace Network. Additionally, from a higher level, this presents an opportunity for those geometric network users within ESRI’s Desktop world to move into the ArcGIS Pro platform. Kevin introduces the NHDPlus HR Map service, discusses a few best practices for using it and where to access the service.

Hydrography for the Nation:

High-quality hydrography data are critical to a broad range of government and private applications. Resource management, infrastructure planning, environmental monitoring, fisheries management, and disaster mitigation all depend on up-to-date, accurate, and high-quality hydrographic data. The U.S. Geological Survey National Geospatial Program National Hydrography Advisory Call has initiated a series of virtual seminars to highlight the uses of hydrographic data. These presentations are intended to share success stories from users who have solved real world problems using hydrography data, provide information about the National Hydrography Dataset and related products. The USGS manages surface water and hydrologic unit mapping for the Nation as geospatial datasets. These include the National Hydrography Dataset (NHD), Watershed Boundary Dataset (WBD), and NHDPlus High Resolution (NHDPlus HR). Hydrography data are integral to a myriad of mission critical activities undertaken and managed by government entities (Federal, State, regional, county, local, Tribal), nonprofit organizations, and private companies.

For more detailed information on national hydrography products visit https://usgs.gov/NHD
 

Details

Date Taken:

Length: 00:42:30

Location Taken: Reston, VA, US

Transcript

Al Rea: Hi, everybody. This is Al Rea, and this is the USGS Hydrography Advisory Call. Hang on, I'm getting some background noise. I'm going to mute everyone. So we'll go ahead and get started. Today, we have two topics, and the first that we'll talk about is we have a couple of folks from Esri here who are going to talk about new capabilities in ArcGIS Pro, sorry, called the Trace Network. So we have Remy Myers, who is going to give the first presentation. The second presentation, which will be about half an hour from now, is Kevin McNinch from USGS, and he'll be talking about the NHDPlus High Resolution web services that we're rolling out. So let's go ahead. Remy, go ahead and take it away. You might have to unmute yourself.
Stephen Zahniser: Yeah, just real quick here. This is Stephen Zahniser. Just wanted to do a real quick introduction. I want to thank everybody for giving us the opportunity here to present the Trace Network today. Remy is the product manager here for Trace Network, so he's going to run through a quick demonstration of what the Trace Network is, and really, from a higher level, this presents an opportunity for those geometric network users within our desktop world to move into the ArcPro world. So it really is a transitional tool as we start to migrate to that next generation of ArcGIS. So with that, I will kick it over to Remy, and we want to make sure we leave a little bit of time at the end for some questions and answers just in case any pop up. And we're definitely available afterwards if any follow-up questions occur. Thank you.
Remy MyersCool. Thank you so much, Stephen. So it's really exciting for me. The trace network is certainly one of many packages that are rapidly evolving along with many information management systems. And as you know, I've ... Well, let me introduce myself a moment. I'm the product manager for the trace network and the Utility Network, and I've had a fairly long relationship with USGS, going back to my failed college days a geology major, through working with USGS in a career with the US military and working with numerous utilities over the last 20 years. And what's exciting for me is seeing this translation, or transition if you will, from a single user or siloed environment of information management, information collection and information distribution to this enterprise architecture. And having the trace network capable in the ArcGIS Pro environment is the first step toward really expanding those capabilities and integrating not only the trace capabilities but the whole ArcGIS hydro capabilities into this new enterprise architecture. So one, we really appreciate the experience over the last six months beta-testing with USGS for the pre-release of the trace network. It has gone live to the public as of today, I believe, is the release date for ArcGIS 2.6. So the trace network is available to the general public as a single user platform on file geodatabase. As I mentioned, it was beta-tested and primarily or initially configured for hydrologic models, but it's also applicable to a whole range of trace networks. We're anticipating in addition to hydrologic studies that rail corporations might be interested in it, social network models and other non-utility disciplines. But certainly, most of the features and functions were designed and configured with that hydrologic stream centerline approach in mind. And we built it to give users such as yourself a path forward from the traditional geometric network model that you've used for so many years, to take advantage of the new functionality and capabilities in ArcGIS Pro. And one of the things that's exciting for me is that the tools and the capabilities alongside the geometric network are substantially increased as well. So it really gives you a much bigger platform not only to view your networks, to edit your networks but to analyze your networks. And then as we release this product to the public, we provided capabilities or tools to enable networks to migrate directly from a geometric network into a trace network, to give you full editing capabilities, such as rubberbanding connected features. So for example, if you've got a station that is part of the network and it's monitoring a stream location, if that stream location needs to be edited as a user would move one of the nodes or vertices of the stream, that metering station would move and remain connected to the stream with one simple edit. Additionally, you've got the ability to set flow direction based upon the digitized direction of the lines. That's absolutely feedback that we've received from you and it was designed and configured based upon the properties of some of the most common hydrologic model sets. In addition to that flow direction, we've also given you trace capabilities based upon those flow directions. Whether you're looking at upstream or downstream or you want to do more advanced analytics using weights or other configurable metrics in your stream properties. We've also given you the ability to create network diagrams from the stream features from a selected trace or selected features. And as of right now, it's available on file geodatabase, which means a single user can rapidly stand up this product and use it without any additional overhead. Now, moving forward, we're looking at adding more capabilities into the first quarter 2021 release of ArcGIS Pro that'll be named 2.7. It'll also be ArcGIS Enterprise 10.9, and one of the first capabilities that we're looking at including in the network is the ability to have multi-user editing support through feature service deployments. So that means that you can now leverage your server capability to have much wider teams of editors working off the same networks. You can expand capabilities through services architecture to provide deeper and greater analytics and provide your product as a view to a much, much larger audience through web instances or mobile platforms. We also want to be able to give you the ability to create connectivity between non-coincident points and contain features inside of other features. These are feature functions that are in our existing utility network that we just want to make available for you. So for example, you may have your station, and let's say that you want to have some sort of communication device that's located at or near the station but not at the same location. You could offset the two features, and if you run a trace to include the station, it will also pick up that communication device that supports that station. By containment, we're also looking at giving you the ability to perhaps create survey station facilities, and you could ... either metering devices or communication devices inside of these facilities to give you a higher resolution of your models at all scales of your network. We're also working ... we talked to development yesterday, and we were excited to hear that they're working on including trace results, so that they can be used for greater depth of Python analytics after trace is completed. And to give you more software development kit tools for the trace network, so that you can expand or enhance functionality as needed. So let's go ahead and take a look at the trace network and see some of these things in action. So I have a small hydrologic region and in this region, I have a starting point. So really quickly from that starting point, I can execute a trace and determine the centerlines of all the streams that are downstream from that trace. So what's really interesting about this type of capability is I could quickly see my path from my starting point to the reservoir or to the body of water that is downstream of it and be able to identify all of the other elements of the stream that are included. If I were to trace from the same starting point upstream, I could not only quickly see the upstream elements, but we could take those elements and explore in the context of a network diagram. So here we have the stream selection and if I were to just populate it in a diagram, those features will be returned in a whitespace drawing that's very geographically similar. Now, I may want to take those geographic features and build it out into some sort of tree-like diagram. And from that tree-like diagram, I can establish the direction I want my tree to flow. In this case, I'm going to build it out from left to right or I can include a number of other properties. I can expand or reduce the lines based upon parameters that I would set in this diagram. But it just gives me the capability of taking that stream feature and representing those elements, both the junctions of the stream, any stations it may have but branches of it and see it in a clear whitespace diagram. Now, let's talk just for a moment about how we can build a new network. So I showed you the capabilities of an existing network. This is another portion of the hydrologic system, but it's not in a trace network. So what I'd like to do is using the tools that are available, I'd like to go ahead and transform these lines in points into a new trace network. So let me go ahead and bring in my catalog, and I will add the geodatabase folder. And in that folder, I establish the name of my new network, and I will populate the stations and the stream centerlines and even give it some form of connective policy. So are these simple line features or are they complex edges that may have more CAD-like capabilities such as multipatch lines or things like that. But I'll just go ahead and load these as simple points and line features and will create the trace network. And as soon as that network is built, we will go ahead and enable that network capability. And as soon as that happens, we have a fully functional trace network. So one of the features that I was very excited about as a former user and consumer of the geometric network, it was a challenge for me to take new CAD designs or new networks like this and change the directional flow. So as this network becomes enabled, we'll go ahead and pull it into a map and we'll see the directional flow of these network systems. There we go. So I'll go ahead and just add this network to my current map, and it'll bring in all the network features. So I've actually got some challenges in this network. I've got some dirty areas or some problems that have occurred. Looking at these dirty areas, what has actually happened is I've got areas of my network that have the potential of overlapping. So what I'll do is I'll go ahead and validate that portion of my network. So it's telling me we still have some overlaps. Let's see if we can get those corrected real quick or what I'd like to do is see if we can identify one that we can fix. So what an overlap feature means is that during the cadastral drawing or during the drawing, that one or many parts of the line self-intersects. And as a general rule of the trace network, we try to prevent that from happening. Let's see. We don't have a self-intersect yet. It could be here. Let's see what we've got here, yeah. So let's say that I just take these vertices right here, and we pull them off. So this is an example of data challenges that a user could face as they're building or loading features from a CAD repository. Now, some of the things that we're working on with the ArcGIS Enterprise team is the ability to rapidly detect and isolate these features. So we're still good here. I don't want to take too much more time, but I do want to just run through one case study of this. Let's see. I don't ... I don't know. Let me go ahead ... I don't want to ... I know we're limited on time. Let me just go ahead and move ahead on the data set. So the next thing I wanted to demonstrate was how we can change flow direction. So in this case, our linear assets are flowing upstream instead of downstream. We can quickly select these features, and using some of the tools in the network, we can rapidly change those network features or the network directional features. So I'll go ahead and redraw, and now my networks are now flowing downstream. So I'll go ahead and stop for a moment for questions if anybody has any questions at this time. Can you guys hear me?
Stephen ZahniserIt looks like Gerald has got a question. I thought I saw him throw his hand up.
GeraldSure. My question is in tracing the networking, is it possible to get a distance, get linear distance between two points on the network from a trace? Or does it only get the whole features?
Remy MyersNo, absolutely. So let me go back to the original trace demo. One of the features that you would include in the trace would be a network attribute that would be the distance of the line segment. So the trace could sum that network length. Now, it looks like this is in decimal degrees, so you would want to reconfigure the data source so it's measuring in feet or in meters. But you can aggregate the sum of the trace length using the length tool or the M-value of the features.
Stephen ZahniserIt looks like we have another question from Mike Tinker. In the case of say, a canal, can the flow line be bi-directional?
Remy MyersSo that's a really good point. There are areas where you can have indeterminate flow direction. So I think the best way I could show that is to just go ahead and run the display flow direction here. So yeah, if there are regions where you've got flow in multiple directions, you could have an indeterminate or a loop in the network. So yeah, that's absolutely an appropriate case.
Stephen ZahniserIt looks like Amanda has a question as well. Amanda Lowe.
Amanda LoweI do not have a question. I don't know why my hand is raised. Sorry about that.
Stephen ZahniserIt's just pure excitement, right?
Amanda LoweYes. That must be it.
Remy MyersAny other questions?
Stephen ZahniserCurtis Price has a question about Hatcher symbology support.
Remy MyersCan you read to it to me, I'm sorry?
Stephen ZahniserIt's just is Hatcher symbology supported?
Remy MyersOh, absolutely. Absolutely
Stephen Zahniserputting mark lines on ... putting mark lines by river mile.
Remy MyersAbsolutely. Yeah. So that's an interesting use case. You can even not only have Hatch symbol displays, but you can integrate the river line miles with system annotation and show those hatches as distances downstream or distances from station.
Stephen ZahniserAnd thanks, Curtis, for the question. And John Becker has a question. Can we have Remy's e-mail address in case I need to do a follow-up? Absolutely.
Remy MyersAbsolutely.
Stephen ZahniserI'll post both mine as well as Remy's e-mail in the chat window here, so it'll be available.
Remy MyersAny other questions?
Stephen ZahniserI think we're ...
Al ReaThis is Al. I just wanted to kind of confirm, you said this is ... what you [Indistinct] pretty much all in 2.6 which is rolling out today?
Remy MyersYes, sir.
Al ReaThen, you did show a few additional functions that were going to come in 2.7.
Remy MyersYes, in first quarter of 2021.
Al ReaGreat. All right. Well, thanks so much, Remy and Stephan, for showing us what's happening with the trace network. This has been high on our list for quite some time. Ever since Pro first came out, we were like, "Okay, we need networks." So this is great to see.
Stephen ZahniserYeah, I think we're all quite excited to kind of push some of this functionality forward. So thanks for all your participation in kind of helping lead that charge forward.
Al ReaAll right. Great. Okay, well, next up, we have Kevin McNinch, who is going to talk about some web services that we are rolling out for the NHDPlus High-Res. Yeah, Kevin, yeah. We're seeing your ... Yeah, I can hear you.
Kevin McNinchLet me get this in presentation mode. All right. So yeah, as Al said, we're going to talk about ...
Al ReaCan you switch? Can you switch your display screen?
Kevin McNinchI have it on the wrong one. There we go. Is that better?
Al ReaYes. Perfect. Thank you.
Kevin McNinch: Yeah, so we do want to introduce the NHDPlus High Resolution map service. We are announcing it today. It's actually been available for a little bit, kind of hidden, but we are rolling this out today and it is available. I'm going to go over some details about how you access it and how ... some best practices for using it, so I'll go through some details as we go forward. As a little bit of background, we do have several hydrography-themed map services available by The National Map. Here's the URL at the bottom here that shows all the National Map services beyond just the hydrography services. It's a useful one to have to be able to get to a bunch of different map services that we have, and they're all pretty useful. All the hydrography ones are in there. And I'll go through those really quickly and individually. These are the four hydrography themed map services that we have. We have a USGS Hydro Cached, and that's a tile map service, so it draws pretty fast, and it's good to really quickly explore the data. We have an NHD service and WBD service. Those have been around for a while. Those are dynamic map services. They draw a little bit slower because we're creating the images on the fly, but you can zoom in a lot further than you can on the cached map service. So those are pretty useful for exploring the data, but now we have an NHDPlus High-Res one that we're going to talk about today. And that's also a dynamic map service and it does have some limitations, but we have it rolled out there with the actual data. So just got to look at more details of that as we go forward. So here's a little more detail. We do have a URL. I have this up three or four times during these slides. So if you want to copy that down or go to that URL I put up earlier. We do encourage you to go and look at this service. We do it have it as an ESRI endpoint and a WMS endpoint as well. As I mentioned, it is a dynamic map service. We have some pretty fresh stuff going on. The goal is to have this service as the most up-to-date version of the NHDPlus High-Res, as we still have the NHDPlus High-Res in what we're calling beta. There are going to be some non-100 percent up-to-date as we transition from beta to non-beta product. So right now, the last update was in March. We actually have an update ready to go. We were hoping to get it ready for today, but it'll probably be later this week. It'll be updated to July 2020 data. And as you can see here, we do have some scale dependencies on it down at the bottom. So that's some of the challenges that I'll talk about a little bit later of how to use the services. Some of the scale dependencies make it a little bit challenging to view, but we have some other ideas about how you can use it with some of our other services, so I'll show that in just a little bit. So really, what is this service used for? It's a map service, so it's not a feature service. So you can't actually get to the underlying data besides to do some really simply identifies and those kind of things, but we really see this as an easy way to go in and visualize this data as a natural, seamless extent. So you can kind of see all the data together as you pan and scroll. You can get to the data without downloading because the download for NHDPlus High-Res can be a lot, especially if you want to look at a lot of different areas. And it is a really good way to go into this and quickly look at some VAAs, value added attributes that the NHDPlus High-Res does and what the data behind the NHDPlus High-Res is. It is those value added attributes. So its a really quick way to go in and see those and kind of explore those. And if you use it in conjunction with some of our other services, there's some easy to help exploring the data and finding what you want to see. So the best place to go in and find more details about the service is the service details page. If you go to the URL right there, it'll take you to here. In this, you can get to the layers over there and you can see the attributes with the layers, and you can get some description and some more details about the service itself. Some of the fundamentals of the service. I would encourage you, if you want to know what is in here, I would encourage you to go here for the details because based on the next slide, there are some differences in the stage product, the NHDPlus High-Res stage product and this map service. And the biggest difference is we've simplified the data model to make it more easily consumable as a service, so we have a new data model that's a subset and part of the VPU distribution data model that we have for the stage product. We are working on a poster. We don't have it quite ready yet, so the best way to go in and see exactly what is in that data schema, as I mentioned before, is to go and explore in this service details page. I am going to highlight some of the biggest differences in the data models from the stage product that you would download and then service that you would use on as a map service. The stuff on the right of this is actually all of this included in this map service. So we have these dozen or so feature classes. And there are no tables included. No raster data is included and if it's not listed on the right here of this slide, it's not included. So we do have some differences. So how do we account for the VAAs, the value added attributes that I mentioned earlier? So we basically created this new feature class on the right. I'm going to go to the next slide, I can come back here if we need to. We have a new feature class in here called NetworkNHDFlowline. So this includes a selection of the VAAs that have been joined to this new feature class of the flowline features that participate in the NHDPlus High-Res network. So if they're on the flow network of NHDPlus High-Res, they're included here, and we have the value added attributes included as attribution on that feature classes. As a corollary to that, there's a NonNetworkNHDFlowline that includes NHDFlowline features that were not part of the NHDPlus High-Res network. And this kind of shows a summary on the right of the slide, kind of the attributes that are included. I'm not going to go through all of them. The majority of the attributes are included, but it is not 100 percent of the attributes. As we go forward, we would like feedback if there's an attribution not included that you would find useful in this map service, please let us know, and we will consider those. We tried to take a good shot at what would be useful to most people, and that's kind of where we start with this map service for now. So I'm going to go back to this previous slide again. There's two other feature classes here that are circled in this kind of mustard color. So we have two feature classes that we're going to add to this, NHDPlusBoundaryUnit and NHDPlusGage. These are not in there yet. They will be in the next update which will probably be before ... the end of the summer, first of the Fall. Sometime around there. We don't have this data quite calculated yet, so it wouldn't be super useful for some of it anyway, but we are going to be adding this in the future. So this'll be a way to get to the NHDPlusBoundaryUnit, which would be like the vector processing units and the raster processing units, and get some data behind those things. And then, we are working to get the NHDPlusGage, which would be the gages that are used in the EROM calculations. Those are going to be updates coming soon. So how much time do I have? So I'm going to show just real quick how to use it. But I guess maybe before I do that, are there any questions about the service itself, before I kind of show what it looks like?
Al ReaKevin ... Curtis Price asked in the chat, "Are the raster data in the plan at all for services?" In particular, he says the flow direction and flow accumulation, you could look at catchments ... if you're looking at issues and catchments or something, trying to figure out what went on?
Kevin McNinchYeah. So we currently do not have plans to do the rasters, but we have identified that as something we need to discuss going forward is how we can account for this data and this product and if we do a national NHDPlus service product under of the same idea.
Al ReaSo are there other questions?
Al Rea[Indistinct] Not hearing or seeing any other questions. Go ahead, Kevin.
>> All right. So i was just going to show real quick kind of using the service. So I have ArcGIS Pro here. I just added connections to the ArcGIS servers that we have. So we have the one that this service is on. It's hydro.nationalmap.gov. It's the URL that I put up earlier, and we have the WBD and NHDPlus High-Res service. So if we add that to the map here, going to turn this base map off and just display that layer. So we do have this, with all the data layers in NHDPlus High-Res that we have, and it is not drawing for some reason. There we go. So you can see that we have a lot of data. You can go through, and you can turn the layers on and off. That probably wasn't the best one to turn it off, and my connection seems to be really slow right now. Huh. Well, that's not good. I can go back to the slide. I'm going to go back to the slide. I kind of have the same thing. So the idea is you can add this map service in, and then we can turn layers on and off. I don't do it, but then I show an example of identifying features. So if you click on a feature, I'm not ... Fairchild Creek, I had clicked on. So it will be this one right here in the middle. You can get to a lot of the ... You can get to all the attributes. So this will be a really good way to go in and explore the VAAs that are on the data. So as I mentioned earlier, we do have scale dependencies on this layer on this map service. So they are pretty tight right now, and full disclosure, they're kind of frustrating to work with sometimes. So a good way to use the service is if we add in the USGS Hydro Cached map service I mentioned earlier, which is a tiled map service. It draws pretty quick. We can draw this data layer, which isn't maybe not quite up-to-date but it's close enough that you can use it to explore the data and find the area you're looking for or any other of the USGS services or any other service out there that you want to use, to kind of explore the data, where you can find the data and then zoom into the correct appropriate level and be able to see the NHDPlus High-Res map service. So I kind of shared that there in those slides, as you're using the hydro cached service to kind of find where you want to go, and then you can zoom in and then turn on the NHDPlus High-Res layer, and you can turn on and off any of the layers you don't want to see. That's kind of it in a nutshell. My demo didn't quite work there. So I guess we can ask more questions or if anybody else has any other thoughts about tips or anything for using or access points or if you want to see the URL again.
Al ReaKevin, it might be good if we post the URL in the chat.
Kevin McNinchYeah.
Al ReaOne of us will do that here pretty soon.
Kevin McNinchOkay.
Al ReaI would also mention that ArcGIS Online is a nice, low impact ... low weight, lightweight interface for things. You can open up the service in ArcGIS Online quite easily and use all the different base maps in there. So if you either don't have ArcGIS Pro access to you right now or actually just want to see something really quick, that may be a good way to go, too.
Kevin McNinchYeah. Absolutely. Yes.
Al ReaYou may be working on ... Well, I've got a low priority or hasn't been a high priority until now, a little project to try to put together an ArcGIS Online web map that would use these services and sort of give you a good starting point. When we get that in a little bit better shape, we can let you all know about that and put it in the newsletter and so on. Any questions for Kevin? Not hearing much. Anyone have questions just overall, just comments to share with us? Nope. Not hearing anything, so we'll go ahead and call it a day then. Thank you, everybody, for joining us. We'll give you 15 minutes.