attribute_function: how to go from geojson -> shp -> mbtiles

[BenOnTrack]

Hi Richard,

I'm working on a specific mbtiles built from an initial geojson that has a few properties such as name, id, bbox.
As I'd like those data to be rendered with the exact same pattern as my "basemap" built with TileMaker, I'm currently:

• converting the .geojson -> .shp using ogr2ogr
• using the .shp as a source to build the mbtiles

I've checked that my .shp is correctly converted and contains the exact same information as the .geojson, by using an online tool such as mapshaper. I also confirm that the .dbf has the expected column "name", "id" and "bbox" (via https://www.dbfopener.com/)

My tilemaker_process file has the following attribute_function

function attribute_function(attr,layer)
	return { class="way",  name=attr["name"], id=attr["id"],bbox= attr["bbox"]}
end

and my tilemaker_config is:

"layers": {
		"ways":{
			"minzoom"       : 4     ,
			"maxzoom"       : 14    ,
			"simplify_below": 12    ,
			"simplify_level": 0.0003,
			"simplify_ratio": 2,
			"source": "../data/shp/ways.shp"		}
	},

I can confirm that the mbtiles is successfully generated, but when I query data I only see the "class" properties, and none of my other properties. So I suspect there's something wrong in my declaration of my data, such as name=attr["name"]. If this piece of code is not accurate, would you be able to troubleshoot what's wrong ?

The definition states: Limited Lua transformations are available for shapefiles. You can supply an attribute_function(attr,layer) which takes a Lua table (hash) of shapefile attributes, as already filtered by source_columns, and the layer name. It must return a table (hash) of the vector tile attributes to set.

I believe I'm not sure to fully understand this part of this sentence ...as already filtered by source_columns, and the layer name....

[systemed]

I believe I'm not sure to fully understand this part of this sentence ...as already filtered by source_columns, and the layer name....

See https://github.com/systemed/tilemaker/blob/master/docs/CONFIGURATION.md#shapefiles, in particular:

You can specify attribute columns to import using the source_columns parameter, and they'll be available within your vector tiles just as any OSM tags that you import would be. To import all columns, use "source_columns": true.

In other words, the source_columns parameter in the JSON controls which attributes from the shapefile are "let through". If you want attribute_function to be able to access all attributes, you need to set "source_columns": true.

[BenOnTrack]

That was an easy one! thanks for the quick feedback - I confirm it works great, ...and also that I can't read.

[systemed]

No problem. In due course I'd like to rewrite the whole of attribute_function so it works more like way_function (i.e. you can put stuff in different layers using Lua) but that's not an immediate priority!

[BenOnTrack]

Also today it seems that the attribute_function output all its values into the properties field of the mbtiles object, such as when you parse a feature from a mbtiles generated with the attribute_function, you would end up with something like (if we use my example above):

{
"properties":{
                        "name" : "x",
                        "bbox" : "y", 
                        "id" : "z"}
}

If you set "include_ids": true in the config file, it manages to generate some idand the object becomes

{
"id" : "w",
"properties":{
                        "name" : "x",
                        "bbox" : "y", 
                        "id" : "z"}
}

I'd need to dig into the code to understand how it generates an ID (w in the example above) without any OSM object provided as input, and if those ID are unique and if the code is indempotent.

If that's the case, then I would not need my .shp ID (z in the example above) as my objective was to re-create a similar mbtiles object whatever the source:

• .pbf -> mbtiles object with id and properties
• .pbf -> geojson -> .shp -> mbtiles object with id and properties

Today with my current method, and include_ids = false, I would have:

• .pbf -> mbtiles object with id and properties
• .pbf -> geojson -> .shp -> mbtiles object with properties (and id being one of the properties)

hence my query function would need to be different depending on the source