Site point clouds
25 Jan 2021
We describe here how the Jura Mountains mapping website uses LIDAR point clouds. SwissTopo releases all its digital products with no charge starting on 1 March 2021, at which point we shall load a full set of point-clouds for the Jura Mountains.
But first, some guidance for users. The website uses the Potree open-source point-cloud viewer which was originally developed with three navigation modes, designated OrbitControls, FirstPersonControls and EarthControls. A WebGL enabled browser/system is required (check browser). The mouse movements are:
- Double Click: sets centre of rotation and zooms into that point.
- Left Click, hold and move mouse: rotates the view
- Middle Scroll Wheel: zoom in and out
- Right Click, hold and move mouse: pans in the scene
The viewing experience can be adjusted via the side bar (click on the top-left menu icon) with respect to:
- objects displayed (under "Filter", select the classified object)
- point budget and field of view (under "Appearance")
- eye-dome lighting (under "Appearance")
- colouring scheme (for instance, select "Elevation" and set the minimum and maximum range).
We are currently using 1 km square preclassified digital surface model point-cloud tiles supplied by Swisstopo (see swissSURFACE3D).
Tiles are organised by commune. The Jura Mountains' mapping covers several communes. We start with the commune of Saint-Cergue and to be able to position users there is a layer on the main map that marks the point-cloud tiles that are available.
Using Ubuntu 18.04 LTS, PDAL was set up in a Miniconda3 virtual environment. Miniconda3 for Python 3.8 was downloaded:
and installed according to the user guide, basically:
- bash Miniconda3-latest-Linux-x86_64.sh
- conda list
- conda create --yes --name myenv --channel conda-forge pdal
- conda update pdal
A recent guide to using PDAL details many useful commands, especially those needed to convert swissSURFACE3D point-cloud tiles in the Swiss cooordinate system to the EPT-formatted files that can be displayed by the Potree viewer.
After starting the conda environment in /home/user/pdal (for user "user"):
- conda activate envpdal
it is useful to run some PDAL commands to check that PDAL is correctly set up. For example, the "information" and "first point" commands for the swissSURFACE3D tile 2494_1146.las in /home/user/pdal/data:
- pdal info data/2494_1146.las
- pdal info data/2494_1146.las -p 0
Some tiles have only partial coverage so merging to a merged las was sometimes necessary using:
- pdal merge data/2494_1146.las data/2495_1146.las data/merged.las
for the merging of two las file. The swissSURFACE3D tiles that were either merged or not merged need to be re-projected to EPSG3857 using:
- pdal translate data/2494_1146.las data/2494_1146_3857.las -f filters.reprojection --filters.reprojection.in_srs="EPSG:2056" --filters.reprojection.out_srs="EPSG:3857"
Entwine was installed in the envpdal virtual environment using:
- conda install --name envpdal --channel conda-forge entwine
EPT files are then generated in a directory using:
- entwine build -i data/2494_1146_3857.las -o data/2494_1146_3857_ept --srs "EPSG:3857"
A simple way to run the Entwine-Potree platform is to serve EPT files. Several http servers can be used, including http-server, SimpleHTTPServer, live server, and the Apache XAMPP. For http-server, in the envpdal virtual environment:
- conda install nodejs -y
- npm install http-server -g
Making sure that the EPT files are stored in a subdirectory to the working folder, run in /home/user/pdal:
- http-server -p 8066 --cors
using --cors to allow serving data to the remote Entwine-Potree platform on port 8066.
The Entwine-Potree plaform is cloned to an Apache2 web directory, say /var/www/html/poent using:
- git clone https://github.com/connormanning/potree.entwine.io.git
To update the Potree point-cloud viewer to the latest version (1.7.2), Potree needs to be cloned to a working directory, say /home/user/potree using:
- git clone https://github.com/potree/potree
- cd potree
- npm install
to create a "build" directory that contains directories called "potree" and "shader".
These directories together with the directory "libs" in the directory "potree" must then be copied to replace directories with the same names in the Entwine-Potree platform at /var/www/html/poent. Generally speaking, updating the Potree viewer version 1.6 to the latest Potree viewer version 1.7.2 only requires the "libs/three.js/extra" directory to be copied to Entwine-Potree.
Serving point clouds
Various parts of the Entwine-Potree installation at /var/www/html/poent need adjustment.
Potree uses html files to configure the display of point clouds (see examples) . Entwine-Potree takes a similar approach, except that the files, which are stored in our /var/www/html/poent/data directory, are somewhat different (see the Entwine-Potree data directory) . Morover, Potree 1.7 requires some changes compared with the files needed for the Potree 1.6 currently supplied with Entwine-Potree.
The Potree viewer in Entwine-Potree can of course use configuration files of the type used by the Potree installation. However, if many point clouds need to be made available via a user-friendly interface, Entwine-Potree's React application is adequate (see demo). We in fact use a geojson layer on the main map to access the ept files.
Somewhat surprisingly, one also finds that none of the Potree 1.7 sample configuration files allow the Properties -> Attribute section of Potree's sidebar panel to be displayed. This means that users cannot choose between displaying images based say on elevation and point-cloud classes (ground, vegetation, water, buildings, etc).
In maintaining the Entwine-Potree approach, the main adjustment needed on updating Potree to version 1.7 is that for a configuration file such as denmark.html in the Entwine-Potree data folder:
- materal: 'xxx'
needs to replace
where xxx can be 'elevation', 'classification', 'rgba', etc
Setting the material configuration to rgba causes a point cloud to appear in the Potree viewer as a grey-scale image provided the brightness is increased (otherwise the image may be black depending on the intensity range). The following works for us:
- material: 'rgba',
- rgbGamma: 0.50,
- rgbBrightness: 0.50,
- rgbContrast: 0.40,
- intensityRange: [0,2500],
We have been unable to display rgba images in colour, possibly because our point clouds do not have colour information (this can be checked apparently). The issue can be avoided by removing the rgba display option in the Properties -> Attribute section of Potree's sidebar panel.
An Entwine-Potree confuration file links to the external EPT source files using a resource link of the form;
- resource: 'http://localhost:8066/2494_1146_3857_ept'
where the link refers to the http-server address and port described above.
A standard Potree installation's configuration file can also use an external EPT source as well as EPT files stored locally in say the directory "/var/www/html/potree/data/pointclouds". Local storage is not available for the Entwine-Potree.
For the Potree 1.7.x examples for viewing EPT source files (e.g., ept.html), in our case via the main map Apache2 webserver or via a React application, it was necessary to add:
- activeAttributeName = 'xxxx';
where xxxx could be height, elevation, classification, etc (i.e., the Attributes that can be selected in the Entwine-Potree Properties sidebar section but are not displayed in a standard Potree 1.7.x installation's sidebar).
The Attributes section is presumable not displayed when using the ept.html file because a THREE window viewer is set up:
- import * as THREE from "../libs/three.js/build/three.module.js";
- window.viewer = new Potree.Viewer(docu .....
that allows the Attributes section to be displayed.
The Entwine-Potree configuration files specify the position and target and allow for annotations to label the point-cloud display.
A simply approach is to set the tile's position as the centre of a tile. For this the tile's EPSF 4326 coordinates need to be converted to EPSG 3857 coordinates, using for example:
giving in this case, the configuration file's position as:
- position: [676030,5854900,1321],
where the third parameter is the maximum elevation from the elevation range that is set as:
- elevationRange = [1137, 1321],
for one of our tiles.
The elevationRange is is given by the minz and maxz parameters of the bbox variable displayed by running the pdal info command (see above).
A suitable target that at least leads to a reasonable display is to set the target equal to the position, with the height set as minz, which for our case gives:
- target: [676030,5854900,1137],
Increasing the latitude (y) improves the perspective.
For an annotation label, start by setting pos, cpos and ctgt to the adjusted target and then adjust the three annotation variables to label a specific feature.
Potree's OpenLayers mini-map (toggled using the icon at the top-left) is very useful. To change the map source it is necessary to edit ol.js in the libs/Openlayers3 directory. It may also be necessary to change the webserver CORS configuration. For our Apache2 server combined with a nginx web proxy, an Access-Control-Allow-Origin header needed to be added for the /var/lib/mod_tile directory that serves OpenStreetMap tiles. The header also needed to be added to the nginx configuration file that specifies the location for the tile server.
Changing the mini-map to use Baremaps vector tiles would be useful.
By way of a summary, the latest version (Version 2.2.0 of August 2020) of PDAL (the Point Data Abstraction Library) simplifies considerably the generation of point-cloud files formatted using the latest version (Version 2.1.0 of July 2019) of EPT (Entwine Point Tile) that can be viewed using the latest version (Version 1.7.2 of October 2020) of the well-established Potree point-cloud viewer.
Future work concerning PDAL will focus on advanced applications that use PDAL pipelines for tasks such as colourising LIDAR point clouds with imagery (see the "Analysis" -> "Colorizing points with images" section of PDAL.io exercises).
25 January 2021