Lidar Base Specification: Data Processing and Handling Requirements

ASPRS LAS File Format

Time of Global Positioning System Data

Datums

Coordinate Reference System

Well-Known Text

Units of Reference

File and Point Source Identification

Positional Accuracy Validation

Absolute Horizontal Accuracy

Relative Vertical Accuracy

Check Points

Absolute Vertical Accuracy

Use of the LAS Withheld Bit Flag

Use of the LAS Overlap Bit Flag

Point Classification

Classification Consistency

Tiles

Point Duplication

ASPRS LAS File Format

• All point deliverables shall be in LAS format, version 1.4-R15, using Point Data Record Format 6, 7, 8, 9, or 10. Data producers are encouraged to review the LAS specification version 1.4–R15 in detail (ASPRS, 2011).

System Identifier

• The LAS System Identifier (SystemID) shall be encoded with the sensor model(s) used to collect data using the standardized encoding method provided by the LAS wiki, https://github.com/ASPRSorg/LAS/wiki/Standard-System-Identifiers. 
• If a hardware system does not have a corresponding code in the wiki, the wiki instructions for adding a new code will be followed by the data producer. 

Time of Global Positioning System Data

• GPS data shall be recorded as Adjusted GPS Time (Standard [satellite] GPS time minus 1*10⁹) at a precision sufficient to allow unique timestamps for each pulse.
• The encoding tag in the LAS header shall be properly set. See LAS specification version 1.4–R15 (ASPRS, 2011).

Datums

• All data collected shall be tied to the datums listed below:
  •  For the CONUS, unless otherwise specified by the user and agreed to in advance by the USGS–NGP:
    • The horizontal datum for latitude and longitude and ellipsoid heights will be the North American Datum of 1983 (NAD 83) using the most recent NGS-published adjustment (currently NAD 83, epoch 2010.00, realization of 2011).
    • The vertical datum for orthometric heights will be the North American Vertical Datum of 1988 (NAVD 88). 
    • The geoid model used to convert between ellipsoid heights and orthometric heights will be the lat­est hybrid geoid model of NGS, supporting the latest realization of NAD 83 (currently [2020] GEOID18 model).
  • For Alaska, American Samoa, Commonwealth of the Northern Mariana Islands, Guam, Hawaii, Puerto Rico, U.S. Virgin Islands, and other areas:
    • USGS–NGP and all collection partners shall agree to and specify horizontal and vertical datums, ellipsoids, and geoids in advance of data collection.

Coordinate Reference System

• Lidar data and all related or derived data and products shall be processed and delivered in a single CRS agreed upon in advance of data collection by the USGS–NGP and all project partners and cooperators.
• The complete CRS definition and its WKT representation, both horizontal and vertical, shall be documented as part of the agreement.
• In all cases, the CRS used shall be recognized and published by the European Petroleum Survey Group (EPSG).
• Each project shall be processed and delivered in a single CRS, except in cases where a project area covers multiple CRSs such that processing in a single CRS would introduce unacceptable distortions in part of the project area. In such cases, the project area is to be split into subareas appropriate for each CRS. The following requirements apply to the subareas:
  •  Each subarea shall be processed and delivered as a separate subproject with its own CRS.
  •  All requirements for a single project will apply to each subproject. 
  •  The DPA boundaries of adjacent subareas shall have topologically coincident boundaries along their common borders.
  •  For each project or subarea, all spatial data within the area shall be in the same CRS.
  •  An additional CRS delivery, arranged in advance, may also be required on specific projects.

Well-Known Text

• CRS information in LAS files shall use WKT as defined in OGC (2001). All other WKT specifications, including Esri, ISO, and OGC (2015) are expressly forbidden.
• The CRS information may be recorded in either a variable length record (VLR) or an extended variable length record (EVLR) at the discretion of the data producer.
• The CRS record shall contain no whitespace unless enclosed within double quotation marks.
• The CRS record shall contain no carriage returns (CRs), line feeds (LFs), or new lines (NLs), or any other special, control, or nonprintable characters.
• For verification or generation of properly formatted WKT, the USGS recommends the use of the gdalsrsinfo (http://www.gdal.org/gdalsrsinfo.html) tool. gdalsrsinfo is a command line tool that can be downloaded and installed using the OSGeo4W installer (https://trac.osgeo.org/osgeo4w/). The following command will produce WKT that the USGS considers to have valid form: $ gdalsrsinfo -o wkt “EPSG:<code>” . However, the USGS recommends four exceptions to the gdalsrsinfo output:
  • gdalsrsinfo adds an EXTENSION[] tag to capture geoid information in the VERT_DATUM[] section that is not defined in the WKT specification. Data providers shall remove the EXTENSION[] tag if it is shown.
  •  In cases where the datum name output from gdalsrsinfo differs from that listed in the EPSG Registry database (https://epsg.org/home.html), the USGS would prefer that the name be changed to match the EPSG Registry; however, the GDAL output will be accepted. For example, EPSG:1116 is named “NAD83_National_Spatial_Reference_System_2011” in the output from GDAL but the name on EPSG Registry is “NAD83 (National Spatial Reference System 2011)” and the only listed alias is “NAD83(2011)”
  •  For all projected coordinate systems, the USGS recommends WKT (OGC, 2001) default values: AXIS[“X”,EAST], AXIS[“Y”,NORTH]; however, the GDAL output (“Easting” and “Northing” rather than “X” and “Y”) will be accepted.
  •  gdalsrsinfo and EPSG outputs use “metre” instead of the U.S. convention “meter.” Either spelling is acceptable to the USGS.
• The USGS recognizes that the GDAL tool is not a rigorous standards-based solution, but it is a mutually convenient, open-source tool suitable for 3DEP purposes at this time. Following are the USGS directions for specific WKT format and content:
  • The vertical CRS shall be included in the CRS.
  • The geoid name shall be appended to the VERT_CS[] name field.For example:
  • VERT_CS[“NAVD88 height (ftUS) - GEOID18”]
  • Horizontal and vertical CRS shall be wrapped within a COMPD_CS. 
  • The EPSG AUTHORITY[] tag shall not be included for the compound coordinate system.
  • User-defined entities will not be allowed for capturing geoid information in the WKT (for example, GEOID_MODEL[]). These nonstandard entity entries are not consistently machine readable.
  • All elements of the CRS record shall include the EPSG AUTHORITY[] entry and a valid EPSG code, except where no EPSG code exists for the element or where otherwise excluded from this requirement within this specification.
  • A given LAS file may contain any number of CRS entries, as VLRs and (or) EVLRs in any combination, as WKT and (or) GeoTIFF in any combination, regardless of the PDRF, provided that:
    • ALL entries shall be tagged as “Superseded”—EXCEPT for the single valid entry to be used. See LAS specification version 1.4–R15 (ASPRS, 2011) for further details.
    • The single valid entry shall be compliant WKT (OGC, 2001).
    • The global encoding bit for CRS shall be set to 1.
• The geoid model used to convert elevations from the ellipsoid to orthometric heights shall also be identified in the <lidar><ldrinfo><ldrgeoid> tag within the Federal Geographic Data Committee (FGDC) metadata files.
• The NGS model filename shall be recorded (for example, <ldrgeoid>g2018u0.bin</ldrgeoid>).

Units of Reference

• Horizontal and vertical units of measure will be in the same units (meters preferred). Mixed units (such as horizontal units of meters and vertical units of feet) are not recommended. Projects delivered with mixed units shall be considered as varying from the specification and will be noted as such in the spatial metadata.  
• All references to the units of measure “Feet” and “Foot” shall specify “International,” “Intl,” “U.S. Survey,” or “US.” 
• For linear data: 
  • Airborne acquired data shall be reported with a level of precision of 1/100 (0.01).
  • Survey point data shall be reported with a level of precision of 1/1000 (0.001).

File and Point Source Identification

• At the time of its creation and prior to any further processing each swath shall be assigned a unique file source ID.
• Each point within the swath shall be assigned a point source ID equal to the file source ID.
• The point source ID on each point shall be persisted unchanged throughout all processing and delivery.
• The file source ID for tiled LAS files shall be set to '0' (see LAS specification version 1.4–R15 [ASPRS, 2011]).

Positional Accuracy Validation

• Prior to classification and development of derivative products from the point data, the absolute and relative vertical accuracy of the point data shall be verified and a detailed report of the validation processes used shall be delivered.

Absolute Horizontal Accuracy

• The horizontal accuracy of each lidar project shall be reported using the form specified by ASPRS (2023): “This data set was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data (2023) for a ___ (cm) RMSEx / RMSEy Horizontal Accuracy Class which equates to Positional Horizontal Accuracy = +/- ___ cm at a 95% confidence level.”

Relative Vertical Accuracy

• Relative vertical accuracy refers to the internal geometric quality of a lidar dataset without regard to surveyed ground control. Two primary factors need to be considered in lidar data vertical accuracy:
• Intraswath Precision (Smooth Surface Precision)
  • Precision will be calculated as:
    • Precision = Range - (Slope x CellSize x 1.414) where:
      • Precision, Range, and Slope are rasters (square cells assumed);
      • Range is the difference between the highest and lowest lidar points in each pixel;
      • Slope is the maximum slope of the cell to its 8 neighbors, expressed as a decimal value, calculated from the minimum elevation in each cell; and
      • CellSize is the edge dimension of the cell. 1.414 is the factor to compute the diagonal dimension of the pixel.
      • CellSize is set to the ANPS, rounded up to the next integer, and then doubled:
      • CellSize ꞊ CEILING(ANPS) × 2 
      • where:
        • CEILING is a function to round ANPS up to the next integer.
  • Assessment of precision will be made on hard surfaced areas (for example, parking lots or large rooftops) containing only single return lidar points.
  • Sample areas for assessment of precision will be approximately 100 pixels.
  • To the degree allowed by the data and the project environment, multiple sample areas representing the full width of the swath(s) (left, center, and right) will be examined.
  • Multiple single swaths from a single lift may be used if needed to sample the full swath width.
  • At a minimum, precision shall be assessed against for each lift of each aircraft/instrument combination used on the project. Additional areas may be checked at the discretion of the USGS–NGP.
  • Each test area will be evaluated using a signed difference raster with a cell size equal to the ANPS, rounded up to the next integer, then doubled (CellSize=CEILING(ANPS)×2).
  • The difference rasters will be statistically summarized to verify that root mean square difference in the z -direction (RMSD_z) values do not exceed the limits set forth in table 2 for the QL of information that is being collected.
  • Precision shall be reported by way of a polygon shapefile delineating the sample areas checked and using the cells within each polygon as sample values, attributed with:
    • minimum slope-corrected range (numeric),
    • maximum slope-corrected range (numeric), and
    • RMSD_z of the slope-corrected range (numeric).
• Interswath (Overlap) Consistency
  • Overlap consistency will be assessed at multiple locations within overlap in nonvegetated areas of only single returns and with slopes of less than 10 degrees.
  • To the degree that the data allow, test areas should be located such that the full width of the overlap is represented.
  • The overlap areas that will be tested are those between the following:
    • adjacent, overlapping parallel swaths within a project;
    • cross-tie swaths and a sample of intersecting project swaths in both flight directions; and
    • adjacent, overlapping lifts.
  • Each overlap area will be evaluated using a signed difference raster with a cell size equal to the ANPS, rounded up to the next integer, then doubled (CellSize=CEILING(ANPS)×2).
  • The difference rasters will be statistically summarized to verify that RMSDz values do not exceed the limits set forth in table 2 for the QL of information that is being collected.
  • The interswath consistency shall be reported by way of a polygon shapefile delineating the sample areas checked and using the cells within each polygon as sample values, attributed with:
    • minimum difference in the sample area (numeric),
    • maximum difference in the sample area (numeric), and
    • RMSD_z of the sample area (numeric).

Checkpoints

Data producers are encouraged to carefully review the requirements in the “Positional Accuracy Standards for Digital Geospatial Data” (ASPRS, 2023).

• All checkpoints shall be located within the DPA.
• Checkpoints for NVA assessments shall be surveyed in clear, open areas (which typically produce only single lidar returns) devoid of vegetation and other vertical artifacts (such as boulders, large riser pipes, and vehicles). 
• Checkpoints shall not be located on ground that has been plowed or otherwise disturbed.
• The same checkpoints may be used for NVA assessment of the point data and DEM.
• Checkpoints for VVA assessments shall be surveyed in vegetated areas (typically characterized by multiple return lidar).
• Checkpoints will be located in areas having a minimum homogeneous area of (ANPS*5)², with less than one-third of the required RMSE_z deviation from a low-slope (<10 degree) plane. 
• In land covers other than forested and dense urban, the tested check point will have no obstructions above 15 degrees over the horizon.
• All tested locations will be photographed showing the position of the survey tripod and the ground condition of the surrounding area.
• Control points used in the calibration process for data acquisition shall not be used as checkpoints.
• Checkpoints shall be an independent set of points used for the sole purpose of assessing the vertical accuracy of the project.
• Every checkpoint used to assess absolute vertical accuracy shall have a corresponding ground photograph with the following requirements: 
  • Photographs shall be captured at the time of the checkpoint survey.
  • Photographs shall be taken from each of the cardinal points (North, South, East, and West).
  • GPS survey equipment shall be in view so that the surrounding environment is recorded with respect to the point location being collected.
  • Photos shall be taken during daylight hours.
  • Photographs shall be of sufficient spatial resolution to enable interpretation of terrain undulations and vegetative cover surrounding the checkpoint for a minimum of 10 feet in all directions surrounding checkpoint.
  • All photographs shall be delivered
    • embedded in a single PDF document, preferably in the ground survey report, or
    • as individual images in sub-directories
  • All images shall be labeled with, or image file names shall include, the checkpoint ID.
• The quantity and location of check points shall meet the following requirements, unless alternative criteria are approved by the 3DEP in advance (see ASPRS 2023 for additional information.):
  1. The ASPRS-recommended total number of check points for a given project size shall be met.
  2. The ASPRS-recommended distribution of the total number of check points between NVA and VVA assessments shall be met.
  3. Checkpoints within each assessment type (NVA and VVA) will be well-distributed across the entire project area.  See “Glossary” section at the end of this specification for a definition of “well-distributed.”
  4. Within each assessment type, check points will be distributed among all constituent land cover types in approximate proportion to the areas of those land cover types (ASPRS, 2023). 

Absolute Vertical Accuracy

• Absolute vertical accuracy of the lidar data and the derived DEM will be assessed and reported in accordance with ASPRS (2023) .
• Vegetated and nonvegetated land cover types shall be assessed for absolute vertical accuracy. 
• Federal Emergency Management Agency (2003) identifies seven land cover types; National Digital Elevation Program (2004) and ASPRS (2004) reiterate the first five of those types. The way in which each of the seven classes was reported under the previous standards and how they are reported under the new ASPRS standards and by this specification are shown in table 3.
• Four absolute accuracy values shall be assessed and reported:
  1. NVA for the point data
  2. VVA for the point data 
  3. NVA for the DEM
  4. VVA for the DEM
• RMSE_v for NVA shall be calculated including survey error as described in ASPRS (2023). See the Glossary entry for RMSE_v for detailed equations. 
• The minimum NVA and VVA requirements for all data, using the ASPRS methodology, are listed in table 4. NVA values shall be met, VVA values shall be reported.
• NVA for the point data shall be assessed by comparing checkpoints surveyed for NVA assessment (see Checkpoints) to a triangulated irregular network (TIN) constructed from ground-classified lidar points in those areas.
• VVA for the point data shall be assessed by comparing checkpoints surveyed for VVA assessment (see Checkpoints) to a triangulated irregular network (TIN) constructed from ground-classified lidar points in those areas.
• NVA and VVA for the DEM are assessed by comparing checkpoints to the final bare-earth surface.
• The minimum required thresholds for absolute accuracy and data internal precision may be relaxed by the USGS-NGP when any of the following conditions are met: 
  • A demonstrable, substantial, and prohibitive increase in cost is needed to obtain this accuracy, which is often the case in heavily vegetated project areas.
  • An alternate specification is needed to conform to previously contracted phases of a single larger overall collection effort such as for multiyear statewide collections.
  • The USGS–NGP agrees that the use of an alternate specification is reasonable and in the best interest of all stakeholders.

Use of the LAS Withheld Bit Flag

• The withheld bit flag, as defined in LAS specification version 1.4–R15 (ASPRS, 2011), shall only be used to identify points that cannot be reasonably interpreted as valid surface returns. Examples include outliers, blunders, geometrically unreliable points, aerosol back-scatter, laser multi-path, airborne objects, and sensor anomalies.
• The withheld flag may be used in conjunction with other classification codes (low/high noise for example), but it should be used in all cases where the previously mentioned criteria are met.
• The usage of the LAS withheld bit flag is of such importance that proof of performance is required. This proof shall be provided as
  • preferred: Maximum Surface Height Rasters as detailed in Appendix 1, or 
  • other test or metadata as agreed to by the USGS in advance and documented in the project Task Order.
• The proof of performance shall be derived from the lidar point cloud as delivered.
• The proof of performance shall be recreated upon any changes to the point cloud.

Use of the LAS Overlap Bit Flag

• The overlap flag shall not be used for lidar point clouds intended for inclusion in 3DEP data holdings.

Point Classification

• The minimum, required classification scheme for lidar data is found in table 5. 
• The following requirements apply to point classification:
  • All points that fall within the minimum classification scheme (table 5) and not flagged as withheld shall be properly classified.
  • Additional classes may be used on specific projects.
  • Accuracy of point classification into classes beyond the minimum scheme (table 5) will not be assessed by the USGS, as documented in metadata.
  • Assessing and verifying accuracy of point classification into classes beyond the minimum scheme will be the responsibility of the partner requesting the additional classes.
  • No points in the classified LAS deliverable may remain assigned to Class 0, unless these points are flagged as withheld.
  • Points classified as water will only be checked when associated with a breakline.
  • Classification codes shall not be used in lieu of the overlap bit flag to identify overlap points. 
  • Model key points, if calculated, shall be identified using the key point bit flag as defined in LAS specification version 1.4–R15 (ASPRS, 2011). Model key points may, in addition, be identified using class 8 at the discretion of the data producer.

Classification Consistency

• Point classification is to be consistent across the entire project. Noticeable variations in the character, texture, or quality of the classification between tiles, swaths, lifts, or other non-natural divisions will be cause for rejection of the entire deliverable.

Tiles

• A single nonoverlapping project tiling scheme will be established and agreed upon by the data producer and the USGS–NGP before collection.
  • The tiling scheme will be used for all tiled deliverables.
  • The tiling scheme shall be generated in and use the same coordinate reference system and units as the lidar point cloud and raster data in order to align orthogonally to the raster pixels.  
  • The tile size shall be an integer multiple of the cell size for raster deliverables.
  • The tiles shall be indexed in x and y to an integer multiple of the x and y dimensions of the tile.
  • The tiled deliverables shall edge-match seamlessly and without gaps.
  • The tiled deliverables shall conform to the project tiling scheme without added overlap.

Point Duplication

• Duplication of lidar points (x, y, z, and timestamp) within the project is not acceptable. LAS files containing duplicated points will be rejected. Near duplication (that is, a group of points duplicated but with a slight but consistent spatial offset) will be regarded as duplication.