Fix test failures with mpich 4+.

[vehre]

coverage on master

Summary of changes

Fix possible bug in mpich 4+ that needs communicators to be cleared from messages before free.

Rationale for changes

Using mpich 4.1.2 several tests failed in the finalization phase where mpich reported, that messages were dangling on a communicator to free. This patch just reads all messages before freeing the communicator, which fixes the issue.

A second issue was that the openmpi flag was not propagated into the top-level CMakeLists.txt using set(... PARENT_SCOPE) which lead to test errors, because the generated hostsfile was not referenced and also other openmpi specific flags were not set. The flag is now a global property, that is introduced in the top-level CMakeLists.txt and this fixes the issue.

Tested with mpich 4.1.2, openmpi 4.1.5 and intel mpi 2021.13

Fix for issue #781, #783, #769, #761

Additional info and certifications

This pull request (PR) is a:

• Bug fix
• Feature addition
• Other, Please describe:

I certify that

• I certify that:
  • I have reviewed and followed the contributing guidelines
  • I will wait at least 24 hours before self-approving the PR to give another
    OpenCoarrays developer a chance to review my proposed code
  • I have not introduced errant white space (no trailing white space or white space errors may
    be introduced)
  • I have added an explanation of what these changes do and why they should be included
  • I have checked to ensure there aren't other open Pull Requests for the same change
  • I have you written new tests for these changes
  • I have successfully tested these changes locally
  • I have commented any non-trivial, non-obvious code changes
  • The commits are logically atomic, self consistent and coherent
  • The commit messages follow best practices
  • Test coverage is maintained or increased after this is merged

Code coverage data

[zbeekman]

LGTM! Thanks @vehre