-
Notifications
You must be signed in to change notification settings - Fork 6
/
performance.txt
40 lines (30 loc) · 1.04 KB
/
performance.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# c with no concurrency
$ time ./pbf
total node references 93796649
real 0m17.815s
user 0m16.597s
sys 0m1.208s
# python with no concurrency
$ time ../street.py
done. total node refs: 93796649
real 2m10.270s
user 2m57.327s
sys 0m8.181s
(7.2x speedup for C)
In Python version using imposm.parser.OSMParser concurrency does not seem to help.
IDF:
concurrency 4 25.5s
concurrency 2 36.5s
Benelux:
concurrency 2 1m56s
concurrency 1 2m12s (~500MB resident)
# C with 8 parallel processes on Benelux feed
$ for i in {1..8}; do time ./pbf /var/otp/graphs/benelux/planet-benelux.osm.pbf& done
real 0m35s
# C with 4 parallel processes on Benelux feed
real 0m21s
Memory use around 2-5MB per process, probably mostly for zlib buffer.
# C with 2 parallel processes on Benelux feed
real 0m18.5s
So it is entirely possible to fork and count nodes, find intersections, load nodes simultaneously.
Or, each process could touch only the blocks mod its pid == 0. The problem with the latter is that you have to merge the results, whereas the former works by itself.