Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

caching tiles? #1

Open
mdsumner opened this issue Jun 5, 2020 · 3 comments
Open

caching tiles? #1

mdsumner opened this issue Jun 5, 2020 · 3 comments

Comments

@mdsumner
Copy link

mdsumner commented Jun 5, 2020

Hey there, this looks very nice - I wonder if you are interested in caching the tiles natively as ceramic does? In my mind ceramic is overloaded, it downloads tiles and (optionally) loads them as a raster, but really it should stick to download tiles for given service/extent, and have separated readers for those tile caches. There might be common ground here for us.

@16EAGLE
Copy link
Owner

16EAGLE commented Jun 8, 2020

Hi Michael, thanks – I am open to reduce redundancy and define common ground! One goal for basemaps was to serve as a dependency for moveVis (and other packages if needed) which is why I was aiming to use as little imports as possible and keep it lightweight.

In general, I am open to use ceramic for managing tile caching. What exactly do you mean with "natively as ceramic does" in contrast to how basemap caches tiles? basemaps currently caches both tiles (as PNGs) and cropped/reprojected products (as TIFS) so that running multiple queries with the same extents and projections just requires the package to read the finished TIF, while the underlying tiles as PNG would be recycled if a new extent or projection is queried.

Apart from that, there only are some functions in it to allow querying basemaps for objects crossing the dateline (e.g. visualizing unprojected movement trajectories that cross the dateline, something some users wanted to do with moveVis). It basically splits the extents at the dateline, pulls map tiles separately and merges them together, applying a coordinate system extended over the dateline towards the less-far reaching extent.

The rest actually is just the user-facing functions.

@mdsumner
Copy link
Author

mdsumner commented Jun 8, 2020

Ah I didn't think it kept the native tiles, I mean the zoom/x/y.png dirs - I'll have a closer look!

@mdsumner
Copy link
Author

mdsumner commented Jun 8, 2020

I see, I'll basically copy you and add osm and carto to ceramic (thanks!) - I think that'll be fairly easy, what you might be interested in is a scan of the ceramic cache for tiles that already exist. They look like this (it's changed as I'm updating to the new tiles/ scheme of mapbox).

If I run

ceramic::cc_location(ext, format = "png")

you see those nine tiles same as you would get with basemap i.e.

tl <- ceramic_tiles(zoom = 13, type = "mapbox.satellite")
tl$fullname

..user/AppData/Local/Cache/.ceramic/api.mapbox.com/v4/mapbox.satellite/13/4346/2860.png 

..user/AppData/Local/Cache/.ceramic/api.mapbox.com/v4/mapbox.satellite/13/4346/2861.png 
...

but stored with the full source path, zoom and type (that's what I mean by native fwiw).
So, there's no benefit for you except a possible saving of repeat downloads for users, and whatever other tile sets we could add (mapbox vector tiles is a good one I'm keen to get to).

I'll keep exploring how you've done this for good ideas I can steal ;) But, finally tile-caching is something I was obsessed with, maybe it's time is past and we don't need it anymore ... a problem I will have is invalidating the cache, when it would be better to get more recent tiles and so forth.

If I see easy opportunities for cross over I'll let you know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants