Seeing the World with Data: 3D LiDAR with MapD and Uber’s deck.gl

LiDAR stands for Light Detection and Ranging, a surveying technique which scans a surface using laser light and returns a densely populated 3D point cloud. LiDAR lets you answer questions which were previously impractical or impossible to answer. For example, you can calculate the precise tree coverage around every building on an island. Trees are natural windbreaks, reducing the damage caused by hurricanes. FEMA could use LiDAR for analysis as they plan for the upcoming hurricane season.

In this blog post, we will be visualizing LiDAR in 3D with the assumption that the data is already processed and loaded into MapD.

LiDAR Data

The biggest challenge with visualizing LiDAR data is volume; LiDAR data from the Netherlands alone has 640 billion points adding up to terabytes of data. Also, due to its widespread application in various domains from physical geography, to drones and autonomous cars, this data is generated more and more quickly every day.

Presently, the most common way to use LiDAR data is via tiling, a time-consuming process. Now MapD users can just plug in LiDAR data and visualize it instantly, with MapD in combination with deck.gl, Uber’s GPU-based rendering engine.

Creating a Pointmap Using MapD and deck.gl

To create a pointmap in MapD Immerse, we can use the X and Y coordinates from our LiDAR dataset. But the real essence of LiDAR data is in the 3rd dimension, the ability to visualize the Z axis. To accomplish this, we will be linking MapD Connector with Deck.gl.

There are two design aspects that have been considered while building this application. First, instead of plotting the entire LiDAR dataset on the map, I have limited the points to current bounding box. This is made possible by MapD’s blazing fast GPU-based SQL engine, which lets you query points within a given bounding box on the fly. Second, within every bounding box I have limited the number of points to 50,000 and the sampling method used for this is Knuth Multiplicative Hashing.

Use Case: Modeling Storm Impacts on Coastal Areas

After storms like Hurricane Irma, it is critical to understand the impact of trees in minimizing building damage during storms. For our example, we use LiDAR data from Jekyll Island, one of the barrier islands off the coast of the state of Georgia.  Microsoft Building Footprint data is used for the buildings (as polygons) on Jekyll Island, joined with parcels tax information from Glynn County to estimate building value. This data helps understand the impact of the rise in sea level on buildings, providing valuable insights for both city planners and insurance companies.

Variety in the data requires a fair amount of pre-processing, but for your convenience, I have provided pre-processed data ready to be loaded into MapD.

Let’s take a quick peek into the application itself:

Color Legend

Blue Points: Submersed Points
Green Points: Vegetation, i.e., Trees and shrubs
Light Brown Points: Surface, i.e., Rooftop and Ground
Brick Red Polygons: Buildings/Houses
Blue Polygons: Flood Level Indicator

Filters

Asset Loss: Displays the total property loss due to flooding (does not include land value)
Height Above Mean High High Water (MHHW): Slider lets you increase the sea level, reference datum is MHHW  
Number of Returns: An attribute specific to LiDAR data which helps in differentiating Surface from Vegetation
Point Radius: Size of Point
Building Type: Type of buildings on Jekyll Island
Ground Points: Displays only Ground Points
Surface Points: Displays Surface Points including Ground Points
Vegetation Points: Displays only Vegetation
High Precision: This switch is activated at the highest zoom level and when turned on it displays 500,000 points.
Freeze Screen: Locks the current display, after which no more requests are sent and the user can pan/zoom using the same points

Visualizing tree coverage around a building on Jekyll Island

There are three factors which are worth considering during a storm:tree coverage, tree height, and distance of trees from buildings. Tree coverage around a building helps mitigate the damage caused by incoming heavy winds, as the wind now hits the tree rather than the building. Height of the tree is useful in estimating how far away from the house the treetops deflect the wind. It’s also worthwhile to know the distance between the house and the tree, in the case the tree is blown over.

To visualize these three factors using the application, I have increased the point radius to 11px and freezed the screen. As seen below, we can visualize tree coverage, tree height and  the distance of trees from buildings.  

Visualizing flood water rise and asset loss

At Jekyll Island, rising sea levels due to global warming and the risk of damage from stronger storms have become important factors to understand. Using this application, planners can stimulate the impact on buildings of a rise in sea levels from 0 - 6 feet above MHHW. They can also estimate property losses. To do so, I have switched off Ground, Surface and Vegetation Points.

Visualizing the shape and height of dunes around Jekyll Coast

Dunes are very useful for Jekyll Island to prepare for rising sea levels. This application lets you visualize dunes by switching off Vegetation Points. When turned on, the vegetation cover over the dunes can be analyzed, which would be helpful in finding dune patches prone to soil erosion.

Code Implementation

I begin with establishing a connection with MapD server using MapD Connector to access MapD’s SQL engine. Once the connection is established, I query the database for points and polygons within the current bounding box, i.e., poly in the below queries:

Query for LiDar Data:

var rows_passing_filters="(SELECT COUNT(rowid) FROM jekyll_points WHERE ST_Contains(ST_GeomFromText('POLYGON(("+poly[0][0]+" "+poly[0][1]+","+poly[1][0]+" "+poly[1][1]+","+poly[2][0]+" "+poly[2][1]+","+poly[3][0]+" "+poly[3][1]+"))’), point) "+filterPoints+")";

var pointQuery="SELECT X,Y,Z, HeightAboveGround,NumberofReturns,Classification,rowid FROM jekyll_points WHERE MOD(jekyll_point.rowid * 2654435761, 4294967296) < "+"((50000*4294967296)/"+rows_passing_filters+") AND ST_Contains(ST_GeomFromText('POLYGON(("+poly[0][0]+" "+poly[0][1]+","+poly[1][0]+" "+poly[1][1]+","+poly[2][0]+" "+poly[2][1]+","+poly[3][0]+" "+poly[3][1]+"))’),point) "+filterPoints+" LIMIT 50000";

Query for Polygon Data:

var queryIntersect = “SELECT DISTINCT(FID_1) FROM jekyll_polygonEdges WHERE ST_Contains(ST_GeomFromText('POLYGON(("+poly[0][0]+” ” + poly[0][1] + ",” + poly[1][0] + " ” + poly[1][1] + ",” + poly[2][0] + " "+poly[2][1]+","+poly[3][0]+" "+poly[3][1]+"))'),mapd_geo)”

var polyQuery = "SELECT mapd_geo,avg_z,avg_ground,BLDGVAL17 FROM jekyll_polygons WHERE SHAPE_area<12291479 AND FID_1 IN (“+ queryIntersect +”) "+filterPoly;

Using the query results I created three layers, one point cloud layer and two polygon layers using deck.gl. Finally, every time the bounding box is modified, the layers are updated with the new data.

The application is open source and can be accessed here. With all the resources at your fingertips, what LiDAR data do you plan on visualizing today? Dr. Michael Flaxman has some great suggestions for you to get you started, and if you face any issues with ingesting the data or deploying this application, leave us a message on our community page.

Code Repository: https://github.com/mapd/lidar-visualization

‍