You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This will result in additional overhead, but significant memory reductions because the cumulative current storage array won't need to have an additional dimension of length n = <number of threads>, it can just be stored in a matrix. I imagine this should be done in addition to #79, where we use a hierarchical parallel processing framework with chunks of the landscape parallelized across processes, and individual moving window solves parallelized via multi-threading.
To start, I'll do some basic benchmarking at different numbers of threads with the @lock method to see what level of overhead we're talking here. Might be minimal, but I imagine it will increase as the number of threads increases because we'll have a "longer line" of processes waiting to write to the array.
The text was updated successfully, but these errors were encountered:
Okay -- this was easy to implement. I've got two methods side y side to allow benchmarking -- still need to do a formal benchmark to see how this scales with increased numbers of threads. Here's the relevant commit.
Here's a script to try out both methods. With 6 threads, they're super similar in compute time, but the lock method has the benefit of much lower memory usage.
This will result in additional overhead, but significant memory reductions because the cumulative current storage array won't need to have an additional dimension of length
n = <number of threads>
, it can just be stored in a matrix. I imagine this should be done in addition to #79, where we use a hierarchical parallel processing framework with chunks of the landscape parallelized across processes, and individual moving window solves parallelized via multi-threading.To start, I'll do some basic benchmarking at different numbers of threads with the
@lock
method to see what level of overhead we're talking here. Might be minimal, but I imagine it will increase as the number of threads increases because we'll have a "longer line" of processes waiting to write to the array.The text was updated successfully, but these errors were encountered: