Data flow: Fix a bad join

[hvitved]

Before

(726s) Cancelling evaluation of #12545 evaluator rec DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths05/4#6d4ff244/4@2f7ff6v4: No demand for this layer anymore.
(726s) Tuple counts for DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths05/4#6d4ff244/4@i1#2f7ff6v4 after 11m42s:
3969000     ~1572%     {4} r1 = SCAN `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e` OUTPUT In.1 'par', In.0 'arg', In.2, In.3 'out'

3963564     ~1607%     {4} r2 = r1 AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

9998        ~0%        {4} r3 = JOIN r1 WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepFromHidden/2#2db712ccPlus` ON FIRST 1 OUTPUT Rhs.1 'par', Lhs.1 'arg', Lhs.2, Lhs.3 'out'
0           ~0%        {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

3963564     ~1607%     {4} r4 = r2 UNION r3
3963564     ~1607%     {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

3963564     ~1472%     {5} r5 = SCAN r4 OUTPUT In.0 'par', In.2, In.1 'arg', In.3 'out', In.0 'par'

27551160000 ~1459%     {5} r6 = JOIN r4 WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::summaryCtxStep/1#1f7d276aPlus` ON FIRST 1 OUTPUT Rhs.1 'ret', Lhs.1 'arg', Lhs.2, Lhs.3 'out', Lhs.0 'par'
                       {5}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
27551146500 ~1540%     {5}    | SCAN OUTPUT In.0 'ret', In.2, In.1 'arg', In.3 'out', In.4 'par'

27555109564 ~1540%     {5} r7 = r5 UNION r6
0           ~0%        {4}    | JOIN WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepToHidden/2#6528ff55Plus` ON FIRST 2 OUTPUT Lhs.0 'ret', Lhs.2 'arg', Lhs.3 'out', Lhs.4 'par'
                       {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
0           ~0%        {4}    | SCAN OUTPUT In.1 'arg', In.3 'par', In.0 'ret', In.2 'out'

                       {4} r8 = REWRITE `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e` WITH TEST InOut.1 'par' = InOut.2
0           ~0%        {3}    | SCAN OUTPUT In.1 'par', In.0 'arg', In.3 'out'
0           ~0%        {3}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

4500        ~0%        {3} r9 = JOIN `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e_1203#join_rhs` WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepFromHidden/2#2db712ccPlus` ON FIRST 2 OUTPUT Lhs.1 'par', Lhs.2 'arg', Lhs.3 'out'
0           ~0%        {3}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

0           ~0%        {3} r10 = r8 UNION r9
                       {3}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
0           ~0%        {4}    | SCAN OUTPUT In.0 'par', In.1 'arg', In.2 'out', In.0 'par'

3955564     ~1547%     {4} r11 = SCAN r4 OUTPUT In.0 'par', In.2, In.1 'arg', In.3 'out'
3948000     ~1541%     {4}    | JOIN WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::summaryCtxStep/1#1f7d276aPlus` ON FIRST 2 OUTPUT Lhs.1 'ret', Lhs.2 'arg', Lhs.3 'out', Lhs.0 'par'
3948000     ~1541%     {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)

3948000     ~1541%     {4} r12 = r10 UNION r11
                       {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
3940000     ~1561%     {4}    | SCAN OUTPUT In.1 'arg', In.3 'par', In.0 'ret', In.2 'out'

3940000     ~1561%     {4} r13 = r7 UNION r12
                       return r13

After

Evaluated relational algebra for predicate DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths05/4#6d4ff244@a42e25u4 on iteration 1 running pipeline base with tuple counts:
        4426831   ~4%    {4} r1 = SCAN `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e` OUTPUT In.1, In.0, In.2, In.3
                     
        4421313   ~4%    {4} r2 = r1 AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
                     
          10477   ~4%    {4} r3 = JOIN r1 WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepFromHidden/2#2db712ccPlus` ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3
              0   ~0%    {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
                     
        4421313   ~4%    {4} r4 = r2 UNION r3
        4421313   ~4%    {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
                     
        4421313   ~0%    {4} r5 = SCAN r4 OUTPUT In.2, In.1, In.3, In.0
                     
                         {4} r6 = r5 AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
        4421313   ~1%    {4}    | SCAN OUTPUT In.3, In.0, In.1, In.2
                     
              0   ~0%    {4} r7 = JOIN r5 WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepToHidden/2#6528ff55Plus#swapped` ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3
                         {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
              0   ~0%    {4}    | SCAN OUTPUT In.3, In.0, In.1, In.2
                     
        4421313   ~1%    {4} r8 = r6 UNION r7
        4421307   ~0%    {4}    | JOIN WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::summaryCtxStep/1#1f7d276aPlus` ON FIRST 2 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.0
                     
                         {4} r9 = REWRITE `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e` WITH TEST InOut.1 = InOut.2
              0   ~0%    {4}    | SCAN OUTPUT In.1, In.0, In.2, In.3
              0   ~0%    {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
                     
           5518   ~1%    {4} r10 = JOIN `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::subpaths04/4#89effa1e_1203#join_rhs` WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepFromHidden/2#2db712ccPlus` ON FIRST 2 OUTPUT Lhs.1, Lhs.2, Lhs.1, Lhs.3
              0   ~0%    {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
                     
              0   ~0%    {4} r11 = r9 UNION r10
                         {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
              0   ~0%    {4}    | SCAN OUTPUT In.2, In.1, In.3, In.0
                         {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
              0   ~0%    {4}    | SCAN OUTPUT In.3, In.1, In.2, In.0
                     
        4421313   ~1%    {4} r12 = SCAN r4 OUTPUT In.0, In.2, In.1, In.3
              0   ~0%    {4}    | JOIN WITH `#DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::Subpaths::localStepToHidden/2#6528ff55Plus` ON FIRST 2 OUTPUT Lhs.0, Lhs.2, Lhs.3, Lhs.0
                         {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
              0   ~0%    {4}    | SCAN OUTPUT In.3, In.1, In.2, In.0
                     
              0   ~0%    {4} r13 = r11 UNION r12
              0   ~0%    {4}    | JOIN WITH `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeMid#f7155bb5` ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Lhs.0
                     
        4421307   ~0%    {4} r14 = r8 UNION r13
                         {4}    | AND NOT `DataFlowImpl::Impl<TaintedPath::TaintedPath::Flow::C>::PathNodeImpl.isHidden/0#dispred#8010d23f`(FIRST 1)
        4421307   ~0%    {4}    | SCAN OUTPUT In.1, In.3, In.0, In.2
                         return r14

[owen-mc]

This fixes the problem for me locally. My only hesitation is that I don't know enough about join orders to know if it definitely stops a bad join order being chosen, or if it just perturbs it to a good order in the two cases I checked.

[aschackmull]

LGTM!