Try to use membarrier on FreeBSD

[SamB]

Currently, you define real wrappers for membarrier on Linux (and wherever you randomly pick up a definition for __NR_membarrier), and everywhere else (except maybe Windows) you define fake wrappers that just fails¹.

Then, if membarrier doesn't claim to support MEMBARRIER_CMD_PRIVATE_EXPEDITED (including the case where we don't know how to call it on a given OS), the calling code to falls back to a hack using mlock(2) and mprotect(2), which unfortunately does not work on ARM/ARM64 Linux or FreeBSD (Apple don't distribute the relevant sources so I have no clue about Mac running on Apple Silicon - not even for x86-64 programs!).

This is a bit hard to spot because it's in two pieces:

1. An explicitly Linux-only bit makes sure you have __NR_membarrier:

   runtime/src/coreclr/pal/src/thread/process.cpp

   Lines 66 to 84 in f196875

   	#ifdef __linux__
   	#include <sys/syscall.h> // __NR_membarrier
   	// Ensure __NR_membarrier is defined for portable builds.
   	# if !defined(__NR_membarrier)
   	# if defined(__amd64__)
   	# define __NR_membarrier 324
   	# elif defined(__i386__)
   	# define __NR_membarrier 375
   	# elif defined(__arm__)
   	# define __NR_membarrier 389
   	# elif defined(__aarch64__)
   	# define __NR_membarrier 283
   	# elif defined(__loongarch64)
   	# define __NR_membarrier 283
   	# else
   	# error Unknown architecture
   	# endif
   	# endif
   	#endif

2. The actual wrappers
   

   runtime/src/coreclr/pal/src/thread/process.cpp

   Lines 146 to 153 in f196875

   	//
   	// Helper membarrier function
   	//
   	#ifdef __NR_membarrier
   	# define membarrier(...) syscall(__NR_membarrier, __VA_ARGS__)
   	#else
   	# define membarrier(...) -ENOSYS
   	#endif

But this misses FreeBSD, which has membarrier(2) too. Apparently they have an actual stub in libc, but have a syscall function too. The stub is declared in sys/membarrier.h and the system call number in sys/syscall.h:

#define	SYS_membarrier	584

The constants are:

enum membarrier_cmd {
	MEMBARRIER_CMD_QUERY =				0x00000000,
	MEMBARRIER_CMD_GLOBAL =				0x00000001,
	MEMBARRIER_CMD_SHARED =				MEMBARRIER_CMD_GLOBAL,
	MEMBARRIER_CMD_GLOBAL_EXPEDITED =		0x00000002,
	MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED =	0x00000004,
	MEMBARRIER_CMD_PRIVATE_EXPEDITED =		0x00000008,
	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED =	0x00000010,
	MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE =	0x00000020,
	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE =	0x00000040,

	/*
	 * RSEQ constants are defined for source compatibility but are
	 * not yet supported, MEMBARRIER_CMD_QUERY does not return
	 * them in the mask.
	 */
	MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ =		0x00000080,
	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = 0x00000100,
};

which appear to match the values for Linux:

runtime/src/coreclr/pal/src/thread/process.cpp

Lines 155 to 165 in f196875

	enum membarrier_cmd
	{
	MEMBARRIER_CMD_QUERY = 0,
	MEMBARRIER_CMD_GLOBAL = (1 << 0),
	MEMBARRIER_CMD_GLOBAL_EXPEDITED = (1 << 1),
	MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED = (1 << 2),
	MEMBARRIER_CMD_PRIVATE_EXPEDITED = (1 << 3),
	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED = (1 << 4),
	MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE = (1 << 5),
	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 << 6)
	};

(Curiously, they don't actually implement Linux compatibility for this syscall despite using the same flag bits.)

Note: for some reason, the same exact code appears in https://github.com/dotnet/runtime/blob/f196875c5d0441364f6538d2dd8fc4e3eb1ff108/src/coreclr/gc/unix/gcenv.unix.cpp, though with different line numbers.

Footnotes

1. Which is a great way to avoid boring, repetitive, and therefore bug prone code, though I note that glibc's syscall() function would set errno = ENOSYS and return -1 rather than returning -ENOSYS if the syscall is not supported. This would only truly matter if you actually had callers that distinguished between negative values or read errno, which you don'tx ↩

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.