NGP Standards and Specifications

Lidar Base Specification: Deliverables

Lidar Base Specification 2021 rev. A

Delivery is required for all ancillary products that support the processing of the lidar dataset including, but not limited to, imagery and all metadata associated with those data. 

Return to LBS Table of Contents

Metadata

Swath Separation Images

Classified Point Data

Bare-Earth Surface (Raster Digital Elevation Model)

Breaklines

 

Metadata

  • Product metadata files shall comply with the Federal Geographic Data Committee (FGDC) "Content Standard for Digital Geospatial Metadata" (CSDGM) (FGDC, 1998).
  • Metadata deliverables shall include the following:
    • A survey report detailing the collection of all ground survey data including the following: 
      • Control points used to calibrate and process the lidar and derivative data.
      • Check points used to validate the lidar point data or any derivative product.
    • Lidar Mapping Report (LMR). The LMR shall include a graphic showing flightline vectors by lift, delivery block tiles, and DPA. It will convey, at a glance, the different collection flights covering the delivery block tiles. 
    • The LMR shall also include the following sections:
      • Data Acquisition and Processing including 
        • A brief write-up of the project to include the USGS 3DEP Lidar QL and any additional products and services required as part of the task order.
        • A brief write-up to include any circumstances encountered during acquisition or processing that resulted in data anomalies, such as
          • adverse environmental conditions (for example, heavy smoke, standing snow, flooded land, full leaf-on conditions);
          • information regarding tidal constraints including tide-gauges used, tidal datum, and tidal windows;
          • issues with data collection (for example, lidar sensor problem, ABGNSS interference); 
          • reference to any ancillary files, such as low confidence polygons that are used to delineate data anomalies. 
      • ABGNSS-inertial Processing including:
        • A write-up of the process including processing software used.
        • Comprehensive processing reports with graphics from POSPac, Inertial Explorer, or similar software (these can be provided as appendices and are satisfactory replacements for any of the following items if requested content is contained in those reports).
        • List or graphics of any reference stations used for differential correction.
        • Confirmation of processing reference frame.
        • Graphics of processing interface to show trajectory data and labeled reference stations for each lift. If precise point position used, a graphic of trajectory only. 
        • Graphics of processed plots for each mission/flight/lift to include, at a minimum:
          • forward/reverse separation of trajectory,
          • estimated accuracy of trajectory,
          • and any additional plots that contractor used in the analyses of trajectory quality and believed to be of value. 
      • Point cloud creation including:
        • Summary of original point cloud (LAS) creation software and processes used, to include:
          • version—this can be represented with a screenshot embedded in the document;
          • coordinate reference system (reference frame, projection, units, vertical datum, geoid height model) selected for point output—this can be represented with a screenshot embedded in the document; 
          • and any supporting graphics the contractor believes are of value.
      • Geometric quality including:
        • Write-up of point density/spacing assessment and results for the dataset including which points were tested and the methodology for assessing density/spacing. 
        • Write-up of geometric adjustment to include
          • a summary of sensor model calibration, boresight process, and subsequent swath adjustment process, and
          • a summary of smooth surface precision and interswath accuracy analyses.
        • Graphics and statistics to support smooth surface precision analyses.
        • Graphics and statistics to support interswath accuracy assessment to include:
          • Swath separation image graphics and write-up of processes and settings used to create these images. Settings to be documented include image GSD, color ramp values, points used, surface algorithm (TIN, point-in-cell, other). This can be documented with a screenshot embedded in the document. 
          • Reporting of other methodologies for assessing interswath and any supporting statistics. 
        • Any additional graphics and statistics the contractor believes may be of value in conveying relative accuracy of the point cloud data.
      • Production including:
        • Point-cloud flagging and classification routines to include:
          • software used,
          • process for assigning withheld flag,
          • process for assigning additional classification codes (for example, vegetation by height, segmentation, building classification).
        • Breakline placement methodology and elevation conflation method including:
          • Description of hydrologic features treated with breaklines (for example, area of water body, stream width).
          • Breakline compilation methodology including:
            • software used and whether compilation was manual, semi-automated, or fully automated,
            • elevation conflation method,
            • QA/QC procedures employed to confirm breaklines meet 3DEP requirements,
            • list of any ancillary imagery or data used for breakline compilation including imagery sources and acquisition dates. 
          • Raster production including interpolation method used for bare earth DEM creation and the version of GDAL used to define the raster CRS. 
    • A georeferenced, polygonal representation of the detailed extents of each lidar swath collected, as a GIS layer. The goal is a set of polygons that define the area actually covered by the swaths, not merely the points collected in the swaths.
      • The extents shall be those of the actual coverage of the collected swath, exclusive of peripheral TIN artifacts:
        • Minimum bounding rectangles or simplified rectangles are not acceptable.
        • The boundary will generally follow the overall shape of the swath. 
      • Each swath polygon shall be attributed with the following:
        • The lift’s unique ID (string format).  
        • The unique Point Source ID of the swath (string format).  
        • The type of swath (string format): 
          • "Project"
          • "Cross-tie” 
          • “Fill-in" 
          • “Calibration” or
          • “Other” 
        • Start time in adjusted GPS seconds to the nearest integer.
        • End time in adjusted GPS seconds to the nearest integer.
      • Esri polygon shapefile or geodatabase or geopackage is required. Geopackage is preferred.
    • A georeferenced, digital spatial representation of the detailed extents of each delivered dataset.
      • The extents shall be those of the actual lidar source or derived product data, exclusive of peripheral TIN artifacts or raster NODATA areas. 
      • A union of tile boundaries or minimum bounding rectangles is not acceptable. 
      • For the point data, no line segment in the boundary will be longer than the four times the ANPS from the nearest lidar point. 
      • Esri polygon shapefile or geodatabase is required. 
    • Product metadata (FGDC compliant, XML format metadata).
      • One XML file is required for each of the following deliverable product groups:
        • Classified point data. 
        • Bare-earth DEMs. 
        • Breaklines. 
        • Any other datasets delivered (digital surface models [DSM], intensity images, height above ground surfaces, and others). 
      • Metadata files for individual data files within a deliverable product group are acceptable but are not required.
      • FGDC-compliant metadata shall pass the USGS Metadata Parser (MP) without errors.
  • A block of lidar-related metadata tags specified by the USGS shall be included in the CSDGM (FGDC, 1998) metadata files for all lidar data deliverables. All tags are required.
  • Tags requiring a numeric value shall not contain text (that is, units) because the required reporting units are defined in the appendix 4. The descriptive template of this lidar metadata block is provided in appendix 4 and a completed example is provided in appendix 3.

 

Swath Separation Images

Description: Swath separation images (figure 4) use color-coding to illustrate differences in elevation (z-) values where swaths overlap. The color-coded images are semi-transparent and overlay the lidar intensity image. They are ancillary metadata used as visual aids to more easily identify regions within point cloud datasets that may have suspect interswath alignment or other geometric issues. These images may be produced by a variety of methods; however, their usefulness will be ensured by following the specification requirements and documenting the processes used to create the images in the lidar acquisition and processing report.  

  • Image creation:
    • All returns, single returns, or last returns shall be used to create the images.
    • All point classes and flags shall be enabled when creating the images and points flagged as withheld or classified as noise shall be excluded.  
    • Elevation values and differences shall not be subjected to a threshold or otherwise clipped so all differences are represented. 
    • The images will be derived from TINs to reduce the number of false difference values on slopes; however, other algorithms are acceptable. 
    • The images shall consist of a 50 percent transparent RGB layer overlaying the lidar intensity image.  
    • The images shall use at least three color levels wherever two or more swaths overlap within a pixel. 
    • Where two or more swaths overlap within a pixel (based on point source ID),  
      • pixel color shall be based on vertical difference of swaths using the following breaks (based on multiples of the Swath Overlap Difference for the QL, table 2). 
        • For QL1 or QL2 data the breaks are:
          • 0-8 cm: GREEN; 
          • 8-16 cm: YELLOW; 
          • > 16 cm or > last additional color ramp bin value: RED (for example, addition of ORANGE pixels for the range of 16-24 cm would require red pixels to represent > 24 cm).
        • For QL0 data the breaks are:
          • 0-4 cm: GREEN; 
          • 4-8 cm: YELLOW; 
          • > 8 cm or > last additional color ramp bin value: RED (for example, addition of ORANGE pixels for the range of 8-12 cm would require red pixels to represent > 12 cm).
      • color choice of GREEN, YELLOW, and RED is suggested but not required.
      • no pixel shall remain uncolored (transparent) in the overlap areas.
    • Where swaths do not overlap, pixel values shall be intensity alone.
  • Image file formats and version control: 
    • Swath difference image format shall be delivered as GeoTIFF or JPEG (with world file) by tile. 
    • The point cloud geometry and intensity data delivered shall be identical to the point cloud geometry and intensity data used to create the difference images. Changes in the point cloud geometry or intensity requires recreation of the difference images.  
  • Spatial extent and coordinate reference system: 
    • Spatial resolution (pixel dimension) of the images shall be no greater than 4 times the Nominal Pulse Spacing (4 x NPS) in the project’s linear unit (meters or feet).
    • The difference images must be representative of the associated data delivery.
    • The images shall be in the same CRS as the point cloud data to ensure alignment with the point cloud.

 

Classified Point Data

  • Classified point data deliverables shall include or conform to the following procedures and specifications:
  • All project swaths, returns, and collected points shall be fully calibrated, adjusted to ground, classified, and segmented into tiles. Project swaths exclude calibration swaths, crossties, and other swaths not used, and not intended to be used, for product generation. 
  • LAS Specification version 1.4, PDRF 6, 7, 8, 9, or 10.
  • Withheld bit flag set as appropriate. 
  • If collected, waveform data in external auxiliary files with the extension .wdp. See LAS specification version 1.4–R15 (ASPRS, 2011) for additional information. 
  • Correct and properly formatted georeference informa­tion as WKT (OGC, 2001) included in all LAS file headers. 
  • GPS times recorded as Adjusted GPS Time at a precision sufficient to allow unique timestamps for each pulse. 
  • Intensity values, normalized to 16-bit. See LAS specification version 1.4–R15 (ASPRS, 2011) for additional information. 
  • Tiled delivery, without overlap, using the project tiling scheme. 
  • Classification, as defined in table 5, at a minimum. 

 

Bare-Earth Surface (Raster Digital Elevation Model)

Delivery of a hydro-flattened bare-earth topographic DEM is a requirement for all USGS–NGP lidar projects. Specific research projects may be exempt from some or all these requirements.

  • Bare-earth surface deliverables shall include or conform to the following procedures and specifications:
    • Bare-earth DEM, generated to the limits of the DPA. 
    • All ground-classified points not flagged as withheld shall be used in the creation of the bare-earth DEM.
    • DEM resolution as shown in the table 6
    • 32-bit floating-point GeoTIFF raster format.
    • The NODATA value of '-999999' shall be defined in GDAL_NODATA tag #42113.
    • GDAL version 2.4.1 or later or as otherwise agreed to in advance and specified in the Task Order, shall be used to populate GeoTIFF keys and tags.
    • Additional requirements for GeoTIFF tiling, compression, and internal overviews may be referenced in Task Orders.
    • DEM data shall be in the same CRS as the lidar data. 
    • Georeference information in or accompanying each raster file, as appropriate for the file format. This information shall include both horizontal and vertical systems; the vertical system name shall include the geoid model used to convert from ellipsoid heights to orthometric heights. 
    • Tiled delivery without overlap. 
    • DEM tiles with no edge artifacts or mismatch. A quilted appearance in the overall DEM surface will be cause for rejection of the entire DEM deliverable, whether the variations are caused by differences in processing quality or character among tiles, swaths, lifts, or other artificial divisions. 
    • Void areas coded using a NODATA value of ‘-999999’ and shall be defined in GDAL_NODATA tag #42113.
    • Hydro-flattening as outlined in the “Hydro-Flattening” section. Depressions (sinks), whether natural or man-made, are not to be filled (as in hydro-conditioning). The methodology used for hydro-flattening is at the discretion of the data producer (refer to appendix 2 for more information on hydro-flattening). 
    • Bridges removed from the surface (refer to the “Glossary” section for the definition of “bridge”). 
    • Road or other travel ways over culverts remain intact in the surface (refer to the “Glossary” section for the definition of a culvert). 
    • A report on the assessed absolute vertical accuracy of the bare-earth surface in accordance with the guidelines set forth in ASPRS (2014). Absolute vertical accuracy requirements using the ASPRS methodology for the bare-earth DEM are listed in table 4
    • QA/QC analysis materials used in the assessment of absolute accuracy. 

 

Breaklines 

  • Delivery of all breaklines collected on or used in support of the project is required for USGS–NGP lidar projects. This includes breaklines used for bridge and saddle treatments and any additional breaklines required by project cooperators.
  • Breaklines representing all hydro-flattened features in a project, regardless of the method used for hydro-flattening, are required for USGS–NGP lidar projects. Specific research projects may be exempt from these requirements with prior approval of the USGS–NGP.
  • Breakline deliverables shall include or conform to the following procedures and specifications:
  • Breaklines developed to the limit of the DPA. 
  • Breaklines delivered in shapefile or file geodatabase formats, as PolylineZ and PolygonZ feature classes, as appropriate to the type of feature represented and the methodology used by the data producer. 
  • Breakline data shall be in the same CRS as the lidar data. 
  • Each breakline feature class shall have properly formatted, accurate, and complete georeferenced information stored in the format’s standard file system location. Each shapefile shall include a correct and properly formatted .prj file. All CRS information for 3-dimensional (3D) data shall include the vertical reference and identify the geoid model used to convert from the ellipsoid to orthometric heights. 
  • Breakline delivery may be in a single layer or in tiles, at the discretion of the data producer. In the case of tiled deliveries, all features shall edge-match exactly across tile boundaries in both the horizontal (x, y) and vertical (z) spatial dimensions. Delivered data shall be sufficient for the USGS to effectively recreate the delivered DEMs using the lidar points and breaklines without substantial editing.