+ GeoJSON stats for exports

[emi420]

What type of PR is this?

• 🍕 Feature

What does this PR do ?

This PR adds a new file stats.json to the exported .zip when "include_stats": true is in the request.

For make this work, a new parameter plugin_fn was added to the query2geojson function.

This function will be executed over every line of the resulting GeoJSON and can be used later to perform other actions like adding transliterations for names.

How to test ?

Send a request with "include_stats": true like this one:

curl -X 'POST' \
  'http://localhost:8000/v1/snapshot/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "include_stats": true,
  "geometry": {
    "type": "Polygon",
    "coordinates": [[[83.98115146067687,28.211311261108108],[83.98115146067687,28.200566215460043],[83.99977233635042,28.200566215460043],[83.99977233635042,28.211311261108108],[83.98115146067687,28.211311261108108]]]
  },
  "filters": {
    "tags": {
      "line": {
        "join_or": {
          "building": ["yes"]
        }
      }
    }
  }
}'

The resulting .zip file will include a stats.json file with statistics for tags.

[kshitijrajsharma]

Question : How do you think this gonna go to HDX for shapefiles and geopackage ?

[emi420]

I have to check @kshitijrajsharma it depends on how that works. I think it's by exporting shapefile/geopackage from PostGIS using ogr2ogr right? in that case I think the strategy for those formats should be different.

[kshitijrajsharma]

Yes correct , we are fetching directly from database ! Geojson is not popular for large files , shapefiles and geopackage is most downloaded one ! Yes we need to think a way for this ! May be do calculation in db level

[emi420]

But for HDX we're creating exports for all formats together right @kshitijrajsharma? In that case we could generate the stats file once (when creating the GeoJSON export) and then attaching it to the other formats. Also, we plan to add part of these stats to the dataset's description in markdown format.

[kshitijrajsharma]

Yes but not for all of the exports include geojson though ! We can further discuss on this !

[kshitijrajsharma]

Request for source code of geojson-stats @emi420

[emi420]

Ok, so yes, a different strategy will be needed for other formats @kshitijrajsharma

Source code for geojson-stats:

https://github.com/emi420/geojson-stats

[emi420]

@kshitijrajsharma do you think we're ready to go with this one? it will provide optional metadata stats for GeoJSON exports, that's what we need now. Then I'll add the HTML version of the stats (#281) and transliterations (#280)

[kshitijrajsharma]

@kshitijrajsharma do you think we're ready to go with this one? it will provide optional metadata stats for GeoJSON exports, that's what we need now. Then I'll add the HTML version of the stats (#281) and transliterations (#280)

One more : HDX exports are using custom snapshot endpoint , can you add test case for that one too ? With one sample hdx export

[emi420]

I close this PR because I'm creating a new one that includes also HTML stats and transliterations.