workload option (uniform and non-uniform)

[carstenbauer]

I wonder whether we should have a simple option workload = :uniform / workload = :nonuniform that translates into, say, DynamicScheduler(; nchunks=nthreads()) or DynamicScheduler(; nchunks=4*nthreads()) respectively.

The thing is that many people don't care about scheduling details like chunking and so on. For them, it would be nice to just have one simple switch that gives reasonable settings.

The idea came to me in the context of the macro API (#53)

@tasks for i in 1:100
    @set workload=:uniform # or maybe @hint?
	@set reducer=(+)
	sin(i)
end

We could either translate the workload specification straight into a scheduler setting (it would essentially just be an alias) or make it a @hint workload=:uniform which tries to find settings for the selected scheduler that make most sense for this workload. For example

@hint workload=:uniform
@set scheduler=:dynamic # default

could lead to DynamicScheduler(; nchunks=nthreads()) and

@hint workload=:nonuniform
@set scheduler=:dynamic # default

to DynamicScheduler(; nchunks=4*nthreads()). Similarly,

@hint workload=:nonuniform
@set scheduler=:static

could lead to StaticScheduler(; split=:scatter) to get some "poor-mans's load balancing".

[carstenbauer]

Having slept on it for a night or two, I (still) think I want to have this. Especially for beginners, we could just tell them to flip the workload switch without having to introduce the full concept of chunking and how more chunks lead to more load balancing (at the cost of more tasks etc.).

For the macro API, I think I prefer the @hint idea, that is, combine the information about the scheduler (dynamic, static, greedy, ...) with the hint that the workload is uniform or not to tune the scheduler for this case. Apart from the easy cases mentioned in the OP, the tricky case is probably

@hint workload=:uniform
@set scheduler=:greedy

in which case we might want to throw a warning that a different scheduler might be better? (how bad is it for uniform workload?)

If @MasonProtter also thinks this is something to explore/implement, I will try to add this to the function API first and later add a @hint option to the macro API.

[MasonProtter]

Yeah I think this is a good idea. The part I worry about though is if :nonuniform makes you do a :scatter split, then you can end up unknowingly writing buggy reductions, since a reduction requires only an associative operator if you do :batch, but must be associative and commutative if you use :scatter.

(by the way, I just noticed that mention of this was removed from the docstrings at some point, which I think is bad)

[carstenbauer]

That's a good point. Of course, we don't have to default to :scatter for the static scheduler. For now, we could only make it change settings for the dynamic scheduler (which is anyways the most important one, because it's the default) and show good warnings / give good advice when used in conjunction with the static or greedy scheduler (i.e. recommending a different scheduler or :scatter).

[carstenbauer]

(by the way, I just noticed that mention of this was removed from the docstrings at some point, which I think is bad)

Hm, don't know how this happend but I'll re-add this. (done)