Regexes and zooms parsed wrongly

[stevage]

I'm not sure how to diagnose further, and am open to suggestions. Here's my code:

Map {
  background-color: #b8dee6;
}

#countries {
  ::outline {
    line-color: #85c5d3;
    line-width: 2;
    line-join: round;
  }
  polygon-fill: #fff;
}

@zoomlevel: 12;
#amenities[zoom >= @zoomlevel] {  
  [tourism="attraction"] { 
    marker-type:ellipse;
    marker-fill:red;
    marker-width:20;
    //marker-opacity: 0.5;
  }
}

#amenities[zoom >= 11] {
  [tourism=~"anythingelse"] { 
    marker-type:ellipse;
    marker-fill:blue;
    marker-width:6;
  }
}

So here's what goes wrong.

1. With @Zoomlevel: 12, any tourism=attraction points are rendered as blue dots from zoom 11 upwards. (Should be red)
2. With @Zoomlevel: 11, they show correctly as red from zoom 11 upwards.
3. If I insert "marker-opacity: 0.5;" in the first rule, then I get small blue translucent circles in both cases (ie, even case 2 which worked before.)

Replacing the regex with [tourism="anythingelse"] works perfectly in all cases.

Here's the mapnik.xml corresponding to the third case:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Map[]>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over" background-color="#b8dee6" maximum-extent="-20037508.34,-20037508.34,20037508.34,20037508.34">

<Parameters>
  <Parameter name="bounds">-180,-85.05112877980659,180,85.05112877980659</Parameter>
  <Parameter name="center">0,0,2</Parameter>
  <Parameter name="format">png8</Parameter>
  <Parameter name="minzoom">0</Parameter>
  <Parameter name="maxzoom">22</Parameter>
  <Parameter name="scale">1</Parameter>
  <Parameter name="metatile">2</Parameter>
  <Parameter name="id"><![CDATA[amenitiesbug]]></Parameter>
  <Parameter name="_updated">1399594269000</Parameter>
  <Parameter name="tilejson"><![CDATA[2.0.0]]></Parameter>
  <Parameter name="scheme"><![CDATA[xyz]]></Parameter>
</Parameters>


<Style name="countries" filter-mode="first">
  <Rule>
    <PolygonSymbolizer fill="#ffffff" />
  </Rule>
</Style>
<Style name="countries-outline" filter-mode="first">
  <Rule>
    <LineSymbolizer stroke="#85c5d3" stroke-width="2" stroke-linejoin="round" />
  </Rule>
</Style>
<Layer name="countries"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>countries-outline</StyleName>
    <StyleName>countries</StyleName>
    <Datasource>
       <Parameter name="file"><![CDATA[/usr/share/mapbox/project/amenitiesbug/layers/countries/82945364-10m-admin-0-countries.shp]]></Parameter>
       <Parameter name="type"><![CDATA[shape]]></Parameter>
    </Datasource>
  </Layer>

<Style name="amenities" filter-mode="first">
  <Rule>
    <MaxScaleDenominator>400000</MaxScaleDenominator>
    <Filter>([tourism] = 'attraction')</Filter>
    <MarkersSymbolizer marker-type="ellipse" fill="#0000ff" width="6" opacity="0.5" />
  </Rule>
  <Rule>
    <MaxScaleDenominator>400000</MaxScaleDenominator>
    <Filter>([tourism].match('anythingelse'))</Filter>
    <MarkersSymbolizer marker-type="ellipse" fill="#0000ff" width="6" />
  </Rule>
</Style>
<Layer name="amenities"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>amenities</StyleName>
    <Datasource>
       <Parameter name="type"><![CDATA[postgis]]></Parameter>
       <Parameter name="table"><![CDATA[(select *
from planet_osm_point
where tourism is not null and name like 'Wairewa%') p]]></Parameter>
       <Parameter name="key_field"><![CDATA[]]></Parameter>
       <Parameter name="geometry_field"><![CDATA[]]></Parameter>
       <Parameter name="extent_cache"><![CDATA[auto]]></Parameter>
       <Parameter name="extent"><![CDATA[5584815.01761261,-7317823.57245253,20037259.7301813,2849863.9668471]]></Parameter>
       <Parameter name="host"><![CDATA[gis.researchmaps.net]]></Parameter>
       <Parameter name="user"><![CDATA[gis]]></Parameter>
       <Parameter name="dbname"><![CDATA[gis]]></Parameter>
    </Datasource>
  </Layer>

</Map>

[stevage]

Oh, here's the project.mml for completeness:

{
  "bounds": [
    -180,
    -85.05112877980659,
    180,
    85.05112877980659
  ],
  "center": [
    0,
    0,
    2
  ],
  "format": "png8",
  "interactivity": false,
  "minzoom": 0,
  "maxzoom": 22,
  "srs": "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over",
  "Stylesheet": [
    "style.mss"
  ],
  "Layer": [
    {
      "id": "countries",
      "name": "countries",
      "srs": "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over",
      "geometry": "polygon",
      "Datasource": {
        "file": "http://mapbox-geodata.s3.amazonaws.com/natural-earth-1.4.0/cultural/10m-admin-0-countries.zip",
        "type": "shape"
      }
    },
    {
      "geometry": "point",
      "extent": [
        50.16924689301642,
        -54.77242729472166,
        179.9977666749446,
        24.789112296764717
      ],
      "Datasource": {
        "type": "postgis",
        "table": "(select *\nfrom planet_osm_point\nwhere tourism is not null and name like 'Wairewa%') p",
        "key_field": "",
        "geometry_field": "",
        "extent_cache": "auto",
        "extent": "5584815.01761261,-7317823.57245253,20037259.7301813,2849863.9668471",
        "host": "gis.researchmaps.net",
        "user": "gis",
        "dbname": "gis"
      },
      "id": "amenities",
      "class": "",
      "srs-name": "900913",
      "srs": "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over",
      "advanced": {},
      "name": "amenities"
    }
  ],
  "scale": 1,
  "metatile": 2,
  "name": "",
  "description": ""
}

[springmeyer]

don't have the bandwidth right now to look closely, but certainly seems buggy. You might try running the latest carto from github master to see if the results are the same. You can run in from the command line against an mss file. See #245 (comment) for how I set up regression tests - might help to understand.

[nebulon42]

From a quick look this might be related to #351, which I couldn't reproduce on latest master.

[nebulon42]

Can be reproduced in latest master.

[nebulon42]

#world {
  [zoom >= 12] {
    [tourism = "attraction"] {
      marker-type: ellipse;
      marker-fill: red;
      marker-width: 20;
    }
  }
  [zoom >= 11] {
    [tourism =~ "anythingelse"] {
      marker-type: ellipse;
      marker-fill: blue;
      marker-width: 6;
    }
  }
}

produces the same problem.

#world {
  [zoom >= 11] {
    [tourism =~ "anythingelse"] {
      marker-type: ellipse;
      marker-fill: blue;
      marker-width: 6;
    }
  }
  [zoom >= 12] {
    [tourism = "attraction"] {
      marker-type: ellipse;
      marker-fill: red;
      marker-width: 20;
    }
  }
}

works.

[nebulon42]

This basically boils down to two things:

1. When there is already a filter with = all other filters are discarded in https://github.com/mapbox/carto/blob/master/lib/carto/tree/filterset.js#L237. There needs to be an exception for =~.
2. After that when the rules are merged for both definitions the higher zooms end up before the lower ones as the array is appended at the end in https://github.com/mapbox/carto/blob/master/lib/carto/tree/definition.js#L59. Thus, in https://github.com/mapbox/carto/blob/master/lib/carto/tree/definition.js#L195 those rules are not taken into account anymore. This is why the second variant in Regexes and zooms parsed wrongly #343 (comment) works as expected.

My solution was to do a index-based sorted insert in https://github.com/mapbox/carto/blob/master/lib/carto/tree/definition.js#L59. This worked for this issue but broke other things.

I'm not sure how to go on. @springmeyer Maybe you have other ideas after this analysis?

A branch with a test case and 1. can be found at: https://github.com/mapbox/carto/tree/issue_343

[springmeyer]

@nebulon42 thanks for digging into this. I don't have other ideas at this point.