Does the deck.gl integration support type H3HexagonLayer?

[cboettig]

Thanks for an amazing package, this has been incredibly useful to me.

I've been enjoying the deck.gl integration you have as well, and the examples in the docs work perfectly for me. However, I cannot get H3HexagonLayer to work. is it possible that only some of the deck.gl types are implemented here? Would it be possible to support H3?

I noticed that you have an example for H3 cells that uses the native maplibre fill-extrusion, but this approach relies on turning the hex ids to boundaries in geojson, which I am worried will not scale as well since I have a lot of small hexes. I've been able to render this layer with pydeck, but would love to be able to combine that with other maplibre layers using your package.

[giswqs]

This is an example for using H3HexagonLayer. The developer console says the H3HexagonLayer class is not registered.

from maplibre import MapOptions
from maplibre.basemaps import Carto
from maplibre.controls import NavigationControl
from maplibre.ipywidget import MapWidget as Map

m = Map(
    MapOptions(
        style=Carto.POSITRON,
        center=(-122.4, 37.74),
        zoom=12,
        hash=True,
        pitch=40,
    )
)
m.add_control(NavigationControl())

deck_grid_layer = {
    "@@type": "H3HexagonLayer",
    "id": "my-layer",
    "data": 'https://raw.githubusercontent.com/visgl/deck.gl-data/master/website/sf.h3cells.json',
    "getHexagon": "@@=hex",
}

m.add_deck_layers([deck_grid_layer], tooltip="Number of points: {{ count }}")
m

[crazycapivara]

Yes, you are right. At the moment in ipywidget only the Core and Aggregation layers from deck.gl's layer catalog are registered and therefore supported. I wanted to keep the js deps small. But as I tried to render it to html or in Shiny, where it is supported I recognized that in this case the h3 JS lib dep is missing.
I will add support in the next release :-)

[crazycapivara]

@cboettig If you install the latest feature branch with

pip install git+https://github.com/eodaGmbH/py-maplibregl@feature/color-utils

you can already test the standalone example rendering to html

[crazycapivara]

@cboettig

pip install git+https://github.com/eodaGmbH/[email protected]

makes the H3HexagonLayer also available for Ipywidget

[cboettig]

Thanks, this is working great now!

[crazycapivara]

In the upcoming release it will now also be possibile to add pydeck.Layer instances, so that you just can reuse code you already created with pydeck, except layers using the JS CSV loader (passing urls to csv files to the data param). I will check if it makes sense to include it in the JS code as well. But with pydeck.Layer you can now pass pandas.DataFrame as data parameter.

See this exmaple

[cboettig]

@crazycapivara this would be perfect, thanks. for some reason the example doesn't render correctly for me. I install from this branch, git+https://github.com/eodaGmbH/py-maplibregl@feature/color-utils

but just get a blank map, but no error.

[crazycapivara]

@cboettig Just tested with serveral Python versions (3.9, 3.11, 3.12) on Mac and Linux and it always works for me.
Which version of pydeck do you use. Did you check if the data loads successfully?

[cboettig]

@crazycapivara Thanks for the reply. I've revisited this and it seems to be working correctly now.

I see that in your example you pass a standard pydeck layer directly, instead of the dict syntax

deck_grid_layer = {
    "@@type": "H3HexagonLayer",
    "id": "my-layer",
    "data": 'https://raw.githubusercontent.com/visgl/deck.gl-data/master/website/sf.h3cells.json',
    "getHexagon": "@@=hex",
}

Using a pydeck layer instead of that kind of notation, everything works well for me now:

import pydeck as pdk
from maplibre import MapOptions
from maplibre.basemaps import Carto
from maplibre.controls import NavigationControl
from maplibre.ipywidget import MapWidget as Map
#import geopandas as gpd
#h3cells = gpd.read_file('https://raw.githubusercontent.com/visgl/deck.gl-data/master/website/sf.h3cells.json')

layer = pdk.Layer(
        "H3HexagonLayer",
        h3cells,
        get_hexagon="hex",
        get_fill_color="[255 - count, 255, count]",
        extruded=True,
        get_elevation="count",
        elevation_scale=20,
    )

m = Map(MapOptions(center=(-122.4, 37.74), zoom=12), controls=[NavigationControl()])
m.add_deck_layers([layer])
m

[cboettig]

@crazycapivara Hi Stefan, sorry to re-open this, but for some reason it seems to be able to work ok with JSON data but not CSV data. The pure pydeck code works just fine with either source.

Here's an example (same as above just converting the sample data to csv).

Works fine in pure pydeck:

import pydeck as pdk
csv_url = "https://minio.carlboettiger.info/public-biodiversity/test.csv"
layer = pdk.Layer(
        "H3HexagonLayer",
        id="gbif",
        data=csv_url,
        extruded=True,
        get_elevation="count",
        get_hexagon="hex",
        elevation_scale=50,
        elevation_range = [0,1],
        pickable=True,
        auto_highlight=True,
        get_fill_color="[255 - count, 255, count]",
    ),

view_state = pdk.ViewState(latitude=37.74, longitude=-122.4, zoom=10)
pdk.Deck(initial_view_state = view_state, layers = [layer])

But the maplibre code fails to render the hexagons for this very same layer:

from maplibre import MapOptions
from maplibre.basemaps import Carto
from maplibre.controls import NavigationControl
from maplibre.ipywidget import MapWidget as Map
m = Map(MapOptions(center=(-122.4, 37.74), zoom=10), controls=[NavigationControl()])
m.add_deck_layers([layer])
m

If I keep everything the same but switch data to the JSON url instead of the csv url, as shown previously, maplibre works as expected.

[crazycapivara]

@cboettig Yes you are right. As already mentioned above in this issue the JavaScript csv loader is not supported yet, because it is not included in the base deck.gl JavaScript library. The csv loader is an external library found here. Do you think this is an important feature?

[cboettig]

@crazycapivara ah, thanks for the explanation, apologies I overlooked that.

The feature is very helpful to me because duckdb can stream direct to S3 as csv, whereas to write out to correctly-formatted JSON I've had to materialize into RAM using geopandas first. I see loaders.gl page you link supports geoparquet, that would be even better than csv support.

Thanks again for all the help, this module is really fantastic for us.

[crazycapivara]

@cboettig Added a separate issue #112 for this request.