If you would like to check metric- or time- distance between 2 groups of points, but there are too many of them to click it on google maps and in addition googlemap key to such tasks is now paid, I strongly encourage you to read this post and try make it with tips wroted here, because it is also an opportunity to learn more about construction and operation of QGiS together with it's plugin QNEAT3 and OpenStreetMaps. Of course there are available some tools to do it, by all of them either can't connect Many to Many points(M:N) - eventually only one to many, or they need paid access key. Solution that i describe in this post gives you actual data ready to conduct on them so many analysis as you want and from/to number of points limited only by your computer resources.
Availability of free and wonderful products like:
OSM maps, which are ready to download in formats compatible with GIS software like shp,
GIS software like QGIS,
QNEAT3 plugin which is easy to install on QGIS,
creates the opportunity for conducting distance analyzes yourself. My example applies the area of Poland, but OSM maps currently have sufficient quality to do it for almost all of the world. It means it doesn't matter if you are interested in data from East Asia or South America, their quality for this places are - probably - also enough to use it to our goal.
Exact task, which I've got was to make the fastest connections between two groups of points (first had about 200 points and second - about 2000), with connections' time threshold of 30 minutes. All points were located within Polish borders.
To do tasks described below you will need:
QGiS program - best last stable version (actually 3.10) available to download from here: https://qgis.org/pl/site/forusers/download.html
QNEAT3 - QGIS plugin which you can simply install from tab Plugins -> Manage plugins
My workflow, developed during making this task, is presented below:
1. Getting optimal road network
Maxspeed field in roads OSM files(for Poland) is in 2020 completed for about 50% road sections. To get reliable time distance matrix result we need this attribut completed for 100% roads. Furthermore, even this 50% attribute values are maxspeed while we need annual driving speed - it's huge difference. Therefore this section is the biggest on this post. After many trials carried out based on small piece of analysed area and comparysions my results with another tools making similar things like Google Maps, i've specified methodology best considering 3 aspects:
a) similarity of the results to the real travel times taken mostly from Google Maps,
b) load on computer resources,
c) ease and free availability of data. First thing I've had to think of is road database, preferably with average travel speed's attribute. Only one among of available such databases proved to be sufficient to my task. It was, already mentioned, Open Street Maps. The main advantage of this map source is complexity and equal coverage of almost all the world. I've also couldn't complain about topology, whereas that data is free. Download and prepare this data is easy and acceptable even for beginners, because for many countries is available in shp format from the site https://download.geofabrik.de/.
Download road data (.shp) for your region(s): https://download.geofabrik.de/
When you have your road database downloaded yet, you can use my QGIS model to automatically prepare this data to analysis. Yellow elements are source spatial files, white ones are QGIS' algorithms and green one is the result - ready road network.
Open Qgis and set your project's Coordinate System to your local metric coordinate system
Download and open model to automatically prepare your road database to distance matrix analysis: https://github.com/abialasik94/Qgis-models
unpack zip, find layers with ending "roads_free_1" and "traffic_free_1" and copy all such files to another folder with name "data"
probably the area of your analysis is bigger than range of 1 file from OSM - in this case you'll have to find shp with borders suitable for OSM files. For example - OSM files for Poland are divided by voivodships and i've used shp file with polish voivodships available from official government site http://www.gugik.gov.pl/pzgik/dane-bez-oplat/dane-z-panstwowego-rejestru-granic-i-powierzchni-jednostek-podzialow-terytorialnych-kraju-prg
Now is the moment to describe methodology used in the model above. So:
- OSM shp with roads which you've added to folder "data" before the moment,
- OSM shp with traffic data, which contain traffic lights.
- spatial file with polygons suitable with data parts from OSM for you region.
- spatial file with built up areas taken from Corine Land Cover 2018. It's available to download from CLC website https://land.copernicus.eu/pan-european/corine-land-cover/clc2018?tab=download and in many european countries, from government sites.
After downloading, cut range to boundaries of your analysis area. Then you should choose built-up areas by numbers: 111, 112, 121, 141 and 142, according to the film below:
Data similar to CLC are also available from OSM, but the level of detail of data from the clc database is closer to the needs of this analysis. This kind of OSM data is in my opinion too detailed. But if your analysis area is not covered by CLC, you should find land cover data suitable to your analysis area - for example OSM or USGS.
- set of points from- and to- which we're making time-distance matrix.
The general principle of this model is to add annual speed attribute to the roads layer and determine its value based on road class, buildings and traffic lights near the roads. All calculations are divided by provinces in order to optimize the use of computer resources. Below the prepare-part of model's workflow is presented:
- transform(...) - transforming all input layers to project's coordinate system. Therefore it's important to choose metrical project's CS,
- filter(...) - filtering 3 input OSM layers (only some kinds of road or only traffic lights from general traffic layer)
- clipped(...) - all input layers clipped by provinces.
After above steps, our layers are prepared to main analysis.
Now from left: builtUpAreas are aggregated(internal boundaries in polygons are deleted), and buffered by 1cm (to be sure of correctly selecting only parts of roads all inside built-up areas). Parallel, the lines are extracted from built-up areas and clipped by province.
On the other side, points from and to which we'll make a distance matrix are buffered by 2 kilometers. Residential roads are cutted to this buffers and connected with main roads. --Thanks to this action road network near from start/end points is denser than far from this points, what makes network possibly precise on the one hand and possibly not excessively heavy on the other hand.--
Our connected road network's attribute maxspeed is coppied as backup to another column and cutted in places of intersection with built-up lines. That makes us possible to select road sections inside of build-up areas. Thanks to this selection field calculator creates new attribute "isbuiltup"(nam in model: "czyzabudow"), assigns to it value = 1 to roads sections inside such areas and value=0 outside such areas.
Now, based on road class, maxspeed OSM attribute added before the moment and attribute "isbuiltup"-("czyzabudow"), field calculator complete attribute maxspeed(which from this moment really will mean annualspeed) by values obtained based on code below:
--This code is available in model, but for imparient readers is txt file below :)--
OK! Now it's time to explain right side of the model. Clipped traffic lights are buffered too, but this time only by 3 meters. --The reason of such small buffer distance is to possibly most avoid including another road sections on multi-level crossroads.--This buffers are also buffered once again by 1cm to ensure that only road sections located max 3m from traffic lights will be selected. Parallelly, 3m buffers are converted to lines.
Our result road layer with calculated annual driving speed is cutted in places of intersection with 3meters traffic lights-buffer. Next all road sections inside this lights buffers are selected. Similarly as earlier with "isbuiltup"("czyzabudow"), now field calculator makes new attribute "istrafficlights"("czyswiatla") and fill it by value=1 for selected road sections and value=0 for unselected. --At this point all parallel model operations are finally connected in one workflow :)--. Road sections with "istrafficlights" value=1 are selected and "maxspeed" value filled earlier is overwritten by value=2. It means big slowdown of driving speed very near(3m from traffic lights). There is only snapping of road network in the places of intersections with built-up and traffic lights and repairing of geometry left and the result is road network with filled annual driving speed parameter.
This action needs much computer resources, therefore maximally reduce your computer usage(switch off all unnecessary programs). Then run the model in accordance with film below
If calculating lasts really too long - divide your provinces and run model as showed below
2. Generation of time distance matrices based on made road network
Thanks to QNEAT3 Qgis plugin, distance matrix generation when data is ready is very simple. Here is verified tutorial about QNEAT3, writed in polish: http://www.dts.put.poznan.pl/wp-content/uploads/QGIS/QGIS_analizy_sieciowe.pdf
Using this tool is described also in many another languages, but just in case, i'll show it on below film too.
Σχόλια