JuraMap routing

JuraMap ¦ JuraMap routing ¦ Jura Mountains mapping

In spite of there being many providers of routing services that incorporate elevation data, both commercial and to a limited extent at no cost (e.g., OpenRouteService; Open-Elevation, Azure Maps Elevation API, Elevation API), there is still a need for self-hosted services. This is because elevation datasets, especially those with high resolutions, are large and difficult to work with using cloud platforms (see: GPXZ; ViewfinderPanoramas). Jura Mountains mapping makes use of the open-source SwissTopo swissALTI3D high-precision, 2-metre resolution, Digital Elevation Model (DEM) which does not appear to be available via any open-source routing service. The routing service also needs to be incorporated into a routing application which is capable of displaying the elevation profile of a route on a map.

Routing service

In self-hosting an open-source routing service one is aware that most established open-source routing engines have an elevation API out-of-the-box (e.g., Graphopper, Valhalla, pgRouting, OpenRoute Service, Open-Elevation).

OpenRoute Service (ORS) is well-supported and well-documented. Installation is straightforward. For Ubutu 18.04 LTS, as explained by ORS, ensure that Maven is installed and that Java 1.11 is the default Java environment (e.g., in a terminal, submit "sudo update-alternatives --config java" and set the Java version). Then download the zip of the ORS repository at:

https://github.com/GIScience/openrouteservice

(the ORS git terminal command "git clone https://github.com/user/openrouteservice.git" does not work).

Then unzip the zip contents to say "/home/user/ors", where the user is "user", and:

cd /home/user/ors/openrouteservice

Then copy "/home/user/ors/openrouteservice/src/main/resources/ors-config-sample.json" to "/home/user/ors/openrouteservice/src/main/resources/ors-config.json".

Then edit "ors-config.json", setting mainly:

base_url (set to something like "ors.mydomain.net");
profiles.active (set to "hiking" in the case of JuraMap);
graphs_root_path (set to "/home/user/ors/openrouteservice/src/main/files/graph" in the case of JuraMap);
elevation_cache_path (set to "/home/user/ors/openrouteservice/src/main/files/cgiar_provider" in the case of JuraMap).

Then set permissions:

sudo chmod -R 777 /home/user/ors/openrouteservice/src/main/files

Then run Maven:

mvn -DskipTests clean package

In the case of JuraMap, copy the ors.war file in " /home/user/ors/openrouteservice/target" to Apache Tomcat (version 7 - apache-tomcat-7.0.103 - in the case of JuraMap seems to work fine, although Tomcat 8 is recommended) :

sudo cp /home/user/ors/openrouteservice/target/ors.war /usr/local/apache-tomcat-7.0.103/webapps/ors.war

Then reverse proxy the ORS service that is running on port 8080 of a server IP of say 192.168.1.111. JuraMap uses Apache2 where the sites-available/000-default-le-ssl.conf file created by certbot HTTPS certificate creation defines a virtual host following the processing of a "/etc/apache2/sites-available/000-default.conf" file with the reverse proxy virtual host entry:

ServerName ors.mydomain.net

ErrorLog ${APACHE_LOG_DIR}/error.log

CustomLog ${APACHE_LOG_DIR}/access.log combined

ProxyRequests Off

Order deny,allow

Allow from all

</Proxy>

ProxyPass / http://192.168.1.111:8080/

ProxyPassReverse / http://192.168.1.111:8080/

Order allow,deny

Allow from all

</Location>

RewriteEngine on

RewriteCond %{SERVER_NAME} =ors.mydomain.net

RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

</VirtualHost>

Setting up HTTPS certificates using EEF's certbot is a bit beyond the scope of this note.

It is probably best to stop Apache Tomcat before copying the ors.war file and to then start Tomcat once the file is copied. Graphs should then be built automatically in the "graphs" folder and .dem files loaded into the "cgiar_provider" folder. At first, these are for the "heidelberg.osm.gz" file stored by default in the "files" folder. To test, the following link (for coordinates in Heidelberg) should work:

http://localhost:8080/ors/routes?profile=foot-walking&coordinates=8.676,49.418%7C8.683,49.409

For JuraMap, we in fact change the ors-config.json elevation_provider from "cgiar" to srtm", and the files loaded into the "cgiar_provider" folder are "dem136185" and "dem136186" together with "N46E005.hgt.zip" and "N46E005.hgt.zip". We have not yet tested using SwissTopo ALTI2D DEM files with ORS (they work fine with our self-hosted OpenElevationService service - discussed below - that is also maintained by the ORS developers).

For JuraMap, maps are edited using a self-hosted OpenStreetmap website (the so-called OSM Rails Port). Osmosis is used to output the website's Postgresql database on port 5432 as an .osm file.to a folder called "/home/ertert/osm-site/database". The terminal command is:

/home/usr/osm/osmosis/bin/osmosis --read-apidb host="localhost:5432" database="openstreetmap" user="user" password="xxx" validateSchemaVersion="no" --write-xml file="/home/ertert/osm-site/databases/layer.osm"

The "layer.osm" file is compressed to a "layer.pbf" file using osmconvert. For Ubuntu 18.04, osmcovert is installed using:

sudo apt-get install oscmtools

and the osmconvert terminal command is:

osmconvert /home/ertert/osm-site/database/layer.osm --out-pbf > /home/ertert/ors/openrouteservice/src/main/files/layer.pbf

If Maven is used to modify the application, the "graph" and "cgiar_provider" folders need to be emptied or recreated, the ORS application undeployed from Tomcat (one can use the Tomcat Manager) and Tomcat stopped before Maven is run again, the new ors.war file copied to Tomcat and Tomcat is restarted to rebuild graphs and load the cgiar_provider files. Our Tomcat start and stop commands are:

/usr/local/apache-tomcat-7.0.103/bin/startup.sh
/usr/local/apache-tomcat-7.0.103/bin/shutdown.sh

Detailing how Tomcat is installed and set up is beyond the scope of this note.

If "layer.osm" is changed, the "graph" folder needs to be emptied, deleted or renamed and the ORS application stopped and restarted using the Tomcat Manager to rebuild graphs.

Routing application

The pioneer for routing applications that use OpenRouteService (and other routing providers) is the Leaflet Routing Machine (LRM) plugin. There has been some discussion about incorporating elevation into the LRM along the lines of what has been done for the SCASB website for which the source is available.

For basic routing (i.e., clicking on a map to set waypoints for a route), the SCASB extends LRM's leaflet-routing-machine.js file with support for the OpenRouteService (see the extended LRM .js file). The file's "L.Class.extend" needs to be edited at around line 20030 for a self-hosted OpenRouteMap service by changing:

"host" to "ors.mydomain.net/ors"
"profile" to "foot-hiking" (in the case of the JuraMap).

The OpenRouteService API key is no longer needed so the SCASB index.js file should be edited by for example adding:

ors_token = '';

A basic routing service obviously needs other simplications. The JuraMap routing application that uses its own OpenRouteService provides an example of a basic routing application that is adequate and indeed extremely useful for JuraMap purposes.

A more powerful application such as SCASB would need a geocoding service (SCASB uses the OpenRouteService Geocoding service). But geocoding is not needed and is probably impractible for JuraMap's limited geographic scope.

OpenElevationService

It was unclear initially whether version 2 of the open-source OpenRouteService incorporated elevation data out-of-the box. Evidently it does (JuraMap uses version 6.7.0 released on 4 January 2022) so the SCASB d3-based elevation profile graph is generated without needing any adjustment to code.

Nonetheless, we are still unsure if SwissTopo ALTI3D DEM geotifs will work with the OpenRouteService. So an OpenElevationService (OES) has been set up and is in fact used for displaying the elevation profiles of each section of a JuraMap trail.

For example, a trail such as the "Geology" trail can be selected and users can click on the map trace to see the elevation profile for a section of the trail.

It is perhaps useful to explain how OES was installed and set up.

The OES repository was cloned:

git glone https://github.com/GIScience/openelevationservice.git

and renamed to "oes".

A Python vitual environment was created at /home/user/oes:

cd /home/user/oes
python3.6 -m venv .
source bin/activate

To prevent a "gevent" error, the latest pip was installed:

python3 -m pip installl --upgrade pip

Then have the application's "manage.py" file recognised:

echo "export FLASK_APP=manage" >> bin/activate

The file "/home/user/oes/openelevationservice/server/ops_settings.sample.yml" needed to be renamed to "ops_settings.yml" and the srtm_parameters changed to the user name and password and the provider_parameters changed for an OES Postresql-PostGIS database called "oes", for example.

Flask can be run to create the database using the terminal command:

flask create

But to be sure it is probably best to create a database in the conventional way. As is cusomary for the OpenStreetMap rendering servers and the OSM Rails Port:

sudo -u postgresql -i
createdb -p 5432 -T UTF8 oes
psql -p 5432
\c oes
CREATE EXTENSION postgis;
ALTER TABLE geometry_columns OWNER to user;
ALTER TABLE spatial_ref_system OWNER to user;
SELECT Postgis_full_version();
\q
exit

The SELECT terminal command should give the Postgis vesion as POSTGIS ="2.4.4" .... for example, as a check.

Requirements are then installed:

pip install -r requirements --default-timeout=1000

Flask can be used to import a geotif DEM into the oes database. It is probably preferable to use QGIS to do this since geotifs may need to be reprojected to EPSG 4326. Using say QGIS Desktop for Windows, open a geotiff DEM and reproject the file (Raster -> Projections -> Warp). ALTI3D DEM's also need to be transformed from Float32 to Int16 (Raster -> Conversion -> Translate -> Advanced parameters -> Output data type -> Int16).

Then create a connection to the oes database (Browser -> PostGIS -> New Connection and submit details). Then install the PostGIS Raster Import plugin and using Database -> PostGIS Raster Import submit the oes connection name and table name (say "oes_cigar" as per the "ops_settings.yml" file) before clicking the "Upload" button. Uploading may take a while.

The standard DEM geotifs for OES are downloaded from http://srtm.cgiar-org/q by selecting a tile area followed by "Search" and "Download". Geotifs should be downloaded to /home/user/oes/tiles (for Switzerland, the SRTM DEM geotiff is called "srtm_38_03.tif").

The terminal command:

flask importdata

can be used for the data import into the oes database, but the QGIS approach described above is preferred.

The flask command in fact uses the raster2pgsql executable. In principle, raster2pgsql should be installed when PostGIS is installed. But this is often not the case. Sometimes a raster2pgsql.exe executable can be found and copied to /usr/bin. So instead of using flask or QGIS one can run:

raster2pgsql -s 4326 -a -C -M -P -t 50x50 /home/usr/oes/openelevationservice/tile/*.tif oes_cgiar | psql -q -h localhost -p 5432 -U user -d oes

With data uploaded, the OES service is started with:

flask run

which listens on "http://localhost:5000". The terminal command:

curl -XGET http://localhost:5000/elevation/point?geometry=6.0,46.0

should give a response for a point in Switzerland using the srtm_38_03.tif.

Changing the port number may be necessary. The flask run command runs the Werkzeug run_simple web server for which the "port" can be changed in /home/user/oes/lib/python3.6/site-packages/werkzeug/serving.py .

However, it is best to run Gunicorn as the production web server using the terminal command:

gunicorn --bind 0.0.0.0:5000 manage:app

Setting up OES to run securely with an https address can once again be carried out using certbot.

As has been explained, Wuerkzeug supports ad-hoc and self-signed certificates once OpenSSL has been installed:

pip install pyopenssl

And as explained, if you use Apache as a reverse proxy, then you can configure the certificate with Apache, and then Apache can "terminate" an encrypted connection. This means that Apache will accept encrypted connections from the outside, but then use regular unencrypted connections to talk to the Gunicorn backend. This is a very useful set up, as it frees an OES application from having to deal with certificates and encryption.

For a production Gunicorn server, using Apache as a reverse proxy, Apache's "/etc/apache2/sites-available/000-default.conf" file needs the entry:

ServerName certbot.mydomain.net

ServerAlias certbot.mydomain.net

DocumentRoot /var/www/le

Satisfy Any

Allow from all

Require all granted

</Location>

</VirtualHost>

where "certbot.mydomain.net" or a similar address is a server address for a certificate.

After running certbot to create the Apache conguration file "000-default-le-ssl.conf" for encrypted https connections at "/etc/apache2/sites-available", "000-default-le-ssl.conf" will need to be edited to look like this:

ServerName certbot.mydomain.net

ServerAlias certbot.mydomain.net

ProxyRequests Off

Order deny,allow

Allow from all

</Proxy>

ProxyPass / http://192.168.1.111:5000/

ProxyPassReverse / http://192.168.1.111:5000/

DocumentRoot /var/www/le

Satisfy Any

Allow from all

Require all granted

</Location>

SSLCertificateFile /etc/letsencrypt/live/certbot.mydomain.net/fullchain.pem

SSLCertificateKeyFile /etc/letsencrypt/live/certbot.mydomain.net/privkey.pem

Include /etc/letsencrypt/options-ssl-apache.conf

</VirtualHost>

</IfModule>

certbot will have created the full chain (fullchain.pem) and private (privkey.pem) keys at "/etc/letsencrypt/live/certbot.myhost.net"

Copying these two key files to "/home/user/oes" allows OES to run as a flask application using the terminal command:

flask run --cert=./cert.pem --key=./privkey.pem

or using Gunicorn:

gunicorn --bind 0.0.0.0:5000 manage:app

But first the file permissions need to be changed:

sudo chown user:user /home/user/oes/cert.perm
sudo chown user:user /home/user/oes/privkey.perm

Regarding the routing application (JuraMap, for example) that uses OES, the result from an Overpass API query can be parsed and manipulated to create a string of coordinates for a trace. The string is used as the input to a XMLHttpRequest and the response to the XMLHttpRequest is loaded into the Leaflet elevation control:

var el = L.control.elevation(elevation_options);

var url = "https://certbot.myhost.net/elevation/line";

var xhr = new XMLHttpRequest();

xhr.open("POST",url);

xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function (xhrresponse) {

if (xhr.readyState === 4) {

xhrresponse = '{"name": "route.geojson","type": "FeatureCollection","features": [{"type": "Feature","geometry": {"type": "LineString","coordinates": [' + xhr.response + ',"properties": null}]}'

xhrresponse = xhrresponse.replace('[{"attribution":"service by https://openrouteservice.org |
data by http://srtm.csi.cgiar.org","geometry":','')

xhrresponse = xhrresponse.replace(',"timestamp"','},"timestamp"');

xhrresponse = xhrresponse.replace('"0.2.1"}','"0.2.1"');

map.removeControl(el);

el.clear();

el.addTo(map);

el.load(xhrresponse);

var data = '{"format_in":"polyline","format_out":"polyline","geometry": [' + strCoordinates + ']}';

xhr.send(data);

}};

22 July 2022