You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just noticed that dask/dask#11479 multiplied the number of tasks by 2.5x before and by 4x after optimization. I haven't looked at what this does to the end to end runtime but initial submission and client/scheduler side memory obviously takes a hit. Given that before/after optimization task number ratio got much worse I suspect we're loosing opportunities for trivial fusion due to this rechunking
# before https://github.com/dask/dask/pull/11479Enteroptimize210477Afteroptimize_blockwise166163Afterfuse_roots166163Aftercull77002Lowlevel77002Afterfuse71878Afterinline71878Afteroptslices71878# after https://github.com/dask/dask/pull/11479Enteroptimize535113Afteroptimize_blockwise390839Afterfuse_roots390839Aftercull301678Lowlevel301678Afterfuse293482Afterinline293482Afteroptslices293482
The text was updated successfully, but these errors were encountered:
I just noticed that dask/dask#11479 multiplied the number of tasks by 2.5x before and by 4x after optimization. I haven't looked at what this does to the end to end runtime but initial submission and client/scheduler side memory obviously takes a hit. Given that before/after optimization task number ratio got much worse I suspect we're loosing opportunities for trivial fusion due to this rechunking
The text was updated successfully, but these errors were encountered: