A New Closed-Source File Format Threatens the Open-Source Mapping Community

You knew—you just knew—that lasers on airplanes would lead straight to war. What's surprising, however, is that it's a war over open-source versus proprietary software formats for LiDAR data and not, you know, lasers.

LiDAR stands for "light detection and ranging," which really undersells just how cool it is. Basically, arrays of lasers are mounted on aircraft, and in flight, they fire 125,000 pulses per second at the ground and measure the speed at which they bounce back.

This data is used to create three-dimensional maps of the Earth's surface, manmade structures, or even the floor of a body of water. And it does so in such detail that LiDAR scans are able to detect changes in elevation as small as four inches. LiDAR systems have already been used to search for lost cities in the Amazon jungle and to create maps to help people better prepare for mudslides or the next storm surge.

Needless to say, mapping 125,000 laser pulses per second creates massive files. The de facto file formats for storing LiDAR "point clouds" were LAS and the compressed LASzip, which are both open source, developed by the German software firm Rapidlasso.

The LAS exchange format was developed with LiDAR point-cloud data in mind, but it can be used for any three-dimensional x,y,z combinations. It's not a perfect system—it stores the points in scan order, which means that accessing a single data point requires loading up the entire huge file no matter what, as geospatial programmer Paul Ramsey points out—but boosters claim it's easier to take data from one system to another when it's stored in a file format that is open-source. LAS has pretty much been the rule since 2003, with LASzip being introduced to make archiving the big files easier.

But then someone had to go and get fancy. The mapping software company Esri introduced "Optimized LAS," in January 2014, as a closed-source compression format to compete with the open source LAS. The company is already a market leader in geographic information software, but it said it was releasing the product for free.

The open-source geospatial community was not pleased. A group of 89 researchers, professors, and developers just released an open-letter, claiming that after more than a year of trying to work with Esri, they have no choice left but to go to the public.

"We, the undersigned, are concerned that the current interoperability between LiDAR applications, through use of the open 'LAS' format, is being threatened by ESRI's introduction and promotion of an alternative 'Optimized LAS' proprietary format," they wrote. "This is of grave concern given that fragmentation of the LAS format will reduce interoperability between applications and organisations, and introduce vendor lock-in."

The sentiment is mostly that another proprietary format is just going to slow down companies, government agencies, and universities that want to tap into this growing resource.

Martin Isenburg at Rapidlasso has been following the ESRI intrusion since the beginning, and wrote on his blog, "I have received (and continue to receive) a fair number of 'off-the-record' emails from people across the industry expressing feelings that range between 'disappointment', 'anger', and 'disgust' over what is seen as an attempt to sabotage our multi-year effort of creating an open, free, and efficient compressed LiDAR exchange format."

For whatever it's worth, this isn't the first time that ESRI has taken on an open-source community. Spatial Source reported that in May 2013, another open letter got ESRI to withdraw an application to make its "Geoservices REST API" the standard way for web clients to communicate with geospatial technologies at the Open Geospatial Consortium. For all the jokes that are made about them—and the unfortunate fact that they have nothing to do with lasers and airplanes—it seems open letters have effected change in the past.