Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Removed unneeded dask values calls to allow lazy execution #207

Merged
merged 32 commits into from
Oct 11, 2023
Merged

Removed unneeded dask values calls to allow lazy execution #207

merged 32 commits into from
Oct 11, 2023

Conversation

rosepearson
Copy link
Owner

@rosepearson rosepearson commented Oct 9, 2023 •
edited
Loading

Various changes to enable Dask to be used more widely:

  • Make code change
  • Update tests
    • Update / create new tests
    • Ensure these tests have the expected behavour
    • Test locally and ensure tests are passing
  • Ensure tests passing on GH Actions
  • Update documentation
    • Doc strings
    • Wiki
  • Update package (version.py & pyproject.toml) version (if needed)

rosepearson and others added 30 commits October 10, 2023 10:14
@rosepearson
Removed unneeded dask values calls to allow lazy execution
3a3d2c8
fixup: Format Python code with Black
bcbaaed
@rosepearson
another move away from low level calls to higher level dask calls. Al…
4adc05c 
…so call with client around the main dask workflows
@rosepearson
merged in changes - set up for bestter parallelisation with Dask
dffccf7
fixup: Format Python code with Black
54cfe24
@rosepearson
Changes to remove Dask bottlenecks in add_coarse DEMs
d3c4d9d
fixup: Format Python code with Black
b95cb84
@rosepearson
Fixed typos
bda5b78
@rosepearson
update client with for the roughness stage
7202d14
@rosepearson
Fixed typos
27f21ed
fixup: Format Python code with Black
59ed745
@rosepearson
Fixed typos
5be595d
@rosepearson
Merge branch '206-dask-values-bottlenecks' of github.com:rosepearson/…
9292a88 
…GeoFabrics into 206-dask-values-bottlenecks
@rosepearson
Fixed typos
ed2772b
fixup: Format Python code with Black
18d9a3a
@rosepearson
Fixed typos
8415dd4
@rosepearson
Merge branch '206-dask-values-bottlenecks' of github.com:rosepearson/…
fb4145b 
…GeoFabrics into 206-dask-values-bottlenecks
@rosepearson
remove dem compute and prepare for dask with pandas
0467fe0
fixup: Format Python code with Black
d833727
@rosepearson
Add back in compute for roughness for now. Remove later.
3cb184c
@rosepearson
Re-removed the compute in roughness and corrected the buffer amount
596dd00
@rosepearson
removed accidentially added lical changes to the examples
3af3782
@rosepearson
Moved for more dask compatability
4f938ca
fixup: Format Python code with Black
98d7d9e
@rosepearson
Updated roughness use of Dask further
7c9b3fe
fixup: Format Python code with Black
d776f73
@rosepearson
Updated tests and fixed any issues found during the test updates
512f635
fixup: Format Python code with Black
642b142
@rosepearson
Updated package version
7fb2e2d 
Ready for landing if updated tests pass.
@rosepearson
Updated dem
72688fd
@rosepearson rosepearson merged commit 4b8b643 into main Oct 11, 2023
@rosepearson rosepearson deleted the 206-dask-values-bottlenecks branch October 11, 2023 07:12
@rosepearson rosepearson linked an issue Oct 13, 2023 that may be closed by this pull request
Dask values - bottlenecks #206
Closed
quinnhornblow added a commit to quinnhornblow/GeoFabrics that referenced this pull request Oct 16, 2023
@quinnhornblow
Merge pull request #1 from rosepearson/main
740abca 
Removed unneeded dask values calls to allow lazy execution (rosepearson#207)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Dask values - bottlenecks
1 participant
@rosepearson