Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General Performance improvements related to geometry manipulations #130

Open
rosepearson opened this issue Dec 19, 2022 · 2 comments
Open

Comments

@rosepearson
Copy link
Owner

rosepearson commented Dec 19, 2022

There are various options to explore to reduce the setup time (i.e. queuing all the function calls before beginning to execute them):

  1. Clip the tile_index_extents to the region to rasterise buffered by the radius before the for loop - will limit the number of tiles to search through in 'select_lidar_files'. Do this in the roughness _add_tiled_lidar_chunked method as well
    image
    Addressed as part of Remove the Dask Compute #131
  2. In _define_chunk_region explore buffering and clipping the shapely primatives instead of creating geopadas.GeoDataFrame's before the final step
    image
@rosepearson
Copy link
Owner Author

rosepearson commented Dec 19, 2022

Note this built on the work in #127

@rosepearson
Copy link
Owner Author

rosepearson commented Jun 19, 2023

Also look at: Not quite the same subject - but when picking this up again could look at using dask with pandas to remove the horrendous bottle neck that exists when calculating the 'open waterway elevations'. See https://github.com/rosepearson/GeoFabrics/blob/main/src/geofabrics/processor.py#L2386

Each row should be able to operate in parallel

@rosepearson rosepearson changed the title Improvements to reduce setup time prior to dask compute General Performance improvements related to geometry manipulations Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant