Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make handle_dates ignore geometry column #33

Merged
merged 1 commit into from
Sep 26, 2018

Conversation

ateucher
Copy link
Contributor

I was trying out the development version (because of the fix for Dates/POSIXt), and noticed that it was much slower than the version on CRAN:

# CRAN version
myurl <- "http://eric.clst.org/assets/wiki/uploads/Stuff/gz_2010_us_050_00_500k.json"
sf <- sf::st_read(myurl, quiet = T)
system.time(geojsonsf::sf_geojson(sf))
#>    user  system elapsed 
#>  1.783   0.062   1.855 

# Dev version (with handle_dates)
devtools::install_github("symbolixAU/geojsonsf")
myurl <- "http://eric.clst.org/assets/wiki/uploads/Stuff/gz_2010_us_050_00_500k.json"
sf <- sf::st_read(myurl, quiet = T)

system.time(geojsonsf::sf_geojson(sf))
#>   user  system elapsed 
#>  3.938   0.053   4.002

The difference was even more pronounced in the larger object I was working with.

Because the geometry column in sf objects is "sticky", x[dte] in the lapply in handle_dates() will always include the geometry column, meaning that it will get converted to character (super slow). Converting x to a regular data.frame means only the columns matched by dte will get converted.

@ateucher
Copy link
Contributor Author

With that fix in place, timing is back to normal:

devtools::install_github("ateucher/geojsonsf", ref = "fix-handle_dates")
myurl <- "http://eric.clst.org/assets/wiki/uploads/Stuff/gz_2010_us_050_00_500k.json"
sf <- sf::st_read(myurl, quiet = T)

system.time(geojsonsf::sf_geojson(sf))
#>    user  system elapsed 
#>  1.409   0.066   1.479 

@SymbolixAU SymbolixAU merged commit 535c314 into SymbolixAU:master Sep 26, 2018
@SymbolixAU
Copy link
Collaborator

Good spot @ateucher

@ateucher ateucher deleted the fix-handle_dates branch September 26, 2018 00:48
@ateucher
Copy link
Contributor Author

Thanks!

This was referenced Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants