Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise memory usage: GPU array storage and other small changes #26

Open
jakewilliami opened this issue Oct 6, 2020 · 7 comments
Open
Labels

Comments

@jakewilliami
Copy link
Owner

#24205 in julia lang https://github.com/JuliaLang/julia/issues/

@jakewilliami jakewilliami added the enhancement New feature or request label Oct 6, 2020
@jakewilliami
Copy link
Owner Author

Namely after "smart" feature selection

@jakewilliami
Copy link
Owner Author

Done in commit e911698.

@jakewilliami jakewilliami reopened this Oct 27, 2020
@jakewilliami jakewilliami changed the title Unload objects from memory if no longer needed Optimise memory usage Oct 27, 2020
@jakewilliami
Copy link
Owner Author

#25 addresses this broader issue of memory efficiency

@jakewilliami
Copy link
Owner Author

Commit 61b0382 addresses this

@jakewilliami
Copy link
Owner Author

jakewilliami commented Oct 27, 2020

Points to consider and possibly implement:

@jakewilliami
Copy link
Owner Author

Commit 70d7d74 addresses this

@jakewilliami jakewilliami added the future Not pressing label Oct 27, 2020
@jakewilliami jakewilliami added ongoing and removed future Not pressing labels Oct 27, 2020
jakewilliami added a commit that referenced this issue Oct 29, 2020
Including concrete types where possible for type stability, and array comprehension instead of `map` where performant.  Minor formatting changes.  Addresses #26.
This was referenced Oct 29, 2020
@jakewilliami
Copy link
Owner Author

jakewilliami commented Oct 29, 2020

Some overview thus-far.

These benchmarking results are from tests since we changed the algorithm to run sequentially.

Commit Benchmark Time of Tests (seconds) % Time Improvement Since Previous Listed Commit Number of Allocations Memory Allocation % Memory Improvement Since Previous Listed Commit
a4689195 30.689 a 371318354 6.38 GiB a
da9c833e 7.768 74.69 104464112 2.51 GiB 60.66
b3aec6b8 5.025 35.31 28589987 713.89 MiB 71.56
3e9be4ad 5.242 -4.32 b 46688538 990.05 MiB -38.68 b

a I did not benchmark prior to this, though it probably wouldn't be hard to checkout and rewrite some tests with an older version.
b In this commit, I had to change the output of the get_vote function from Int8 to Float64 for correctness. As a result, this is why we have a decrease in benchmarks since the previous listed commit.


NB—: time improvement since last commit can be calculated very easily:

julia> improvement(a, b) = ((a - b) / a) * 100
improvement (generic function with 1 method)

julia> improvement(30.689, 7.768)
74.68799895728111

That is to say, there was a 74.7% improvement between times 30.689 s and 7.768 s.

@jakewilliami jakewilliami changed the title Optimise memory usage Optimise memory usage: GPU array storage and other small changes Oct 30, 2020
@jakewilliami jakewilliami removed their assignment Jan 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants