pipeBytes'1 Property test fails with native implementation

[twitu]

Test suite Test.System.Process: RUNNING...

Streamly.System.Process
  pipeBytes'
Test.System.Process: closeFd: invalid argument (Bad file descriptor)
    pipeBytes' tr = map toUpper [✘]

Failures:

  test/Streamly/System/Process.hs:507:17: 
  1) Streamly.System.Process.pipeBytes' pipeBytes' tr = map toUpper
       uncaught exception: ProcessFailure
       ProcessFailure 1
       (after 7 tests)
         [121,117,121,115,111]

  To rerun use: --match "/Streamly.System.Process/pipeBytes'/pipeBytes' tr = map toUpper/"

For commit id 63060b9. However test is flaky and fails only occasionally.

[twitu]

Different test failures are related to a common issue with closing file descriptors. Below are logs or a failed run and a successful run.

Failed

child input action doing
child: closing parent direction: ParentToChild fd: 73
child: closed parent direction: ParentToChild fd: 73
child: closing child direction: ParentToChild fd: 72
child: closed child direction: ParentToChild fd: 72
child input action done
child output action doing
child: closing parent direction: ChildToParent fd: 74
Test.System.Process: closeFd: invalid argument (Bad file descriptor)
cleanupNormal waiting
cleanupNormal done

Passed

child input action doing
child: closing parent direction: ParentToChild fd: 73
child: closed parent direction: ParentToChild fd: 73
child: closing child direction: ParentToChild fd: 72
child: closed child direction: ParentToChild fd: 72
child input action done
child output action doing
child: closing parent direction: ChildToParent fd: 74
child: closed parent direction: ChildToParent fd: 74
child: closing child direction: ChildToParent fd: 75
child: closed child direction: ChildToParent fd: 75
child output action done
child: closing parent direction: ChildToParent fd: 76
child: closed parent direction: ChildToParent fd: 76
cleanupNormal waiting
cleanupNormal done

All cases are failing in the child process at the output action step where the stdout pipe's (from child to parent) parent fd is failing to close.

[twitu]

After discussion, the current cause of the bug appears to be how the process is being cloned in the ghc runtime. It is likely that ghc RTS threads implemented using pthreads, is using clone call with flags that do not copy over the file descriptors.

With this assumption the sequence of steps that leads to the error are these -

1. Parent process creates a pipe
2. File descriptors are updated in green thread 1 (GT 1)
3. Parent process "clones" a child process on GT 1
4. Child process is moved to GT 2 because of scheduling
5. GT2 does not have fd structures of pipe created in GT 1. This is related to how threads have been implemented in GHC RTS.
6. Child process tries to close fds that do not exist

This is an intermittent issue because sometimes step 4 does not happen. This error also does not occur on using the process package because the library clones parent and child process using a c ffi function, somehow GHC makes sure process created through ffi stays on the same GT. Another point that validates the above cause is that tests pass when configured to run with 1 thread only.