Realm: performance regression in cuPyNumeric CFD code

[ipdemes]

This commit https://gitlab.com/StanfordLegion/legion/-/commit/b1bb4babd7e6f7d37dcadf8bc62ce41e8266bf6e resulted in large regression in cuPyNumeric CFD benchmark.

[apryakhin]

@ipdemes Are we using reductions in "CFD benchmark"?

[ipdemes]

We call "sum" only a couple of times , both shouldn't affect execution time. So there should be something else in this PR that effected performance

[eddy16112]

That PR changes the way of registering a reduction. @ipdemes Do you register a reduction after the top-level task is launched? Do you include the runtime startup into your timing?

[apryakhin]

The redop registration shouldn't be on a critical path. @eddy16112 Can you please check if we can catch this in our perf regression test suite? If we don't have redops in them I think it will worth adding a modified version of `test/realm/simple_reduce.cc" test or whatever else test is necessary to be able to catch this regression in our tests.

@ipdemes I know some context on this was in the slack thread. Could you please copy-paste some of the information from there in here, so we have everything in one place...e.g links to the benchmark results/configurations and how did you find the offending change.