.NET 5 Support

[AndrewIOM]

I have been working to get the RProvider running on .NET5 using the dotnet sdk.

To get RProvider to work, I had to update the following:

• Migrate all projects to the new SDK format.
• Migrate to FAKE5 for build script.
• Move paket to use the dotnet tool.
• Update to fsdocs for the documentation website.
• Remove Remoting between server and client processes, and replace with an implementation based on System.IO.Pipes. I have used another helper library to achieve this (PipeMethodCalls).
• Update associated libraries (e.g. test libraries).

I have tested this implementation using both the .NET 5 and .NET 6 preview 7 SDKs, with all tests passing and sample scripts working as expected. This is on macOS 11 Big Sur on both an Intel mac and M1 mac. I have tested against the latest R version 4.1.1 (2021-08-10) -- "Kick Things"

There was a small change required to the PipeMethodCalls library, but I was unsure if this is something that can be addressed in RProvider. Essentially, there is an issue with reflection where I needed to use GetRuntimeMethods rather than GetMethods to discover the interface's methods.

NB The only prerequisite to run is that R_HOME environment variable is set.

Addresses issues #188.
Follows discussion in #209

[hmansell]

Wow! Thanks for doing all this work!

I am no longer actively involved in the project (or in the .NET world really) but I would be happy to merge the PR if nobody else who is more involved has an opinion.

What is the backwards-compat story with these changes? What happens if someone is using RProvider with an older .NET version with which it is currently compatible?

[zyzhu]

@AndrewIOM. This is big! Thanks a lot for making it happen.

I would like to test it locally and see whether Deedle's RProvider plugin can work with this as you already target .Net 5 while Deedle is still on netstandard2.0.

[AndrewIOM]

Thanks for the input. I've had a quick look into the backwards compatability issues.

It looks like only the xunit.runner.visualstudio package is dependent on greater than netstandard2.0; this can be downgraded to version 2.4.1 for netstandard2.0.

I got RProvider itself to build on netstandard2.0 easily by replacing any references to net5.0. However, I'm not sure how to handle packaging up the RProvider.Server app, as netcoreapp older versions are no longer supported. I can invesstigate further.

A second problem is that I assumed that mono and .net framework support could be dropped and removed a check that changed the running of RProvider.Server from mono to dotnet (see RInteropClient.fs line 62). I'm not sure if figuring out how to set this up to support .NET Framework and mono could be complex, or if the same exe / dll could be called from both mono and dotnet. Any expertise appreciated.

Perhaps people on .NET Framework and mono could continue to use the previously released package and this project could support .NET Core / 5 going forward?

[jackfoxy]

Make the clean break to .NET 5. Many other projects have already done this.

[dsyme]

This is fantastic work.

[dsyme]

@AndrewIOM If you agree I would like to make you co-maintainer on this project, given the depth and quality of this work. @zyzhu I trust that would be ok?

[dsyme]

@AndrewIOM I'll marge this. but we also need to re-enable automated CI testing - the Travis support can be removed in favout of new Github actions. I'll make a start on that now, if you could help get it green that would be great

[zyzhu]

@AndrewIOM If you agree I would like to make you co-maintainer on this project, given the depth and quality of this work. @zyzhu I trust that would be ok?

Of course. 👍

BTW, @AndrewIOM, I have difficulties building it locally on Windows at the step of building test project.

error FS3053 : The type provider 'RProvider.RProvider' reported an error : The type provider constructor has thrown an exception: One or more errors occurred. (System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: Initialization of R.NET failed ---> System.ArgumentException: This 64-bit process failed
 to load the library C:\Program Files\R\4.1.0\bin\x64\R.dll. Native error message is 'The operation completed successfully.'   at DynamicInterop.UnmanagedDll.ThrowFailedLibraryLoad(String
 dllFullName, String nativeError)   at DynamicInterop.UnmanagedDll.ReportLoadLibError(String dllName, String nativeError)   at DynamicInterop.UnmanagedDll..ctor(String dllName)   at RDo
tNet.REngine..ctor(String id, String dll)   at RDotNet.REngine.CreateInstance(String id, String dll)   at RDotNet.REngine.GetInstance(String dll, Boolean initialize, StartupParameter par
ameter, ICharacterDevice device)   at [email protected](Unit unitVar) in C:\FSharp\fslaborg\RProvider\src\RProvider.Runtime\RInit.fs:line 143   --- End of inner 
exception stack trace ---   at RProvider.Server.EventLoop.runServerCommandSafe[a](FSharpFunc`2 f) in C:\FSharp\fslaborg\RProvider\src\RProvider.Server\RInteropServer.fs:line 71   at RPro
vider.Server.RInteropServer.RProvider.Internal.IRInteropServer.GetPackages() in C:\FSharp\fslaborg\RProvider\src\RProvider.Server\RInteropServer.fs:line 88   --- End of inner exception st
ack trace ---   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)   at System.Reflection.RuntimeMe
thodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)   
at PipeMethodCalls.RequestHandler`1.GetTypedResponseForRequest(SerializedPipeRequest request) in C:\FSharp\fslaborg\RProvider\paket-files\andrewiom\PipeMethodCalls\PipeMethodCalls\RequestHandler\RequestHandler.cs:line 155) [C:\FSharp\fslaborg\RProvider\tests\Test.RProvider\Test.RProvider.fsproj]

I tried it on Mac but I'm not sure how to setup R_HOME environment correctly on Mac. I got the following error message

error FS3053 : The type provider 'RProvider.RProvider' reported an error : The type provider constructor has thrown an exception: Failed to start the R.NET server within 20 seconds.To enable logging set RPROVIDER_LOG to an existing file name. [/Users/zyzhu/Projects/fslab/RProvider/tests/Test.RProvider/Test.RProvider.fsproj]

[AndrewIOM]

@dsyme Thanks for checking the code and kind words. I'm more than happy to be co-maintainer on this project and help it move forward. Would like for F# to become more widely adopted in scientific research.

@zyzhu I can have a look at the issue tomorrow, although I'm currently travelling without a Windows machine. For Mac, you can export the R_HOME env variable temporarily within an open terminal session with:

export R_HOME=somepath

Where somepath is the output of R RHOME

It would be ideal if we can set this value automatically given that the code is already retrieving it from running R RHOME.

[tpetricek]

👍 This is fantastic work and I second @dsyme's suggestion to make you a co-maintainer of the project if you are up for that!

[dsyme]

@hmansell @tpetricek @zyzhu @muehlhaus let me know if any of you no longer want to be maintainers (you all have "maintain" access right now)

thanks
don

[zyzhu]

I don't need to be maintainer anymore. We are in good hands now. 👍

[hmansell]

I don’t think it makes sense for me to be a maintainer anymore, either, sad to say…

…

On Tue, Sep 14, 2021 at 8:14 AM Zhenyong Zhu ***@***.***> wrote: I don't need to be maintainer anymore. We are in good hands now. 👍 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#210 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHPPC2MABCARXLBWGNZIP3UB44APANCNFSM5DZSGNFA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[dsyme]

@zyzhu @hmansell Thanks - you're both welcome back as maintainers any time of course, as well as contributors!!!

[muehlhaus]

I also do not need to be a maintainer of RProvider. I agree with @zyzhu and @dsyme, it is awesome to see @AndrewIOM excellent contributions driving FsLab further into data science.

[tpetricek]

I also do not need to be a maintainer, but if it is useful to have someone on the list as a backup in case everyone else gets hit by a bus, I'm happy to remain listed.

(But I'm also owner of the fslaborg organization and the www.fslab.org domain, which I'm happy to keep paying for - right now, it is paid for until October 2024...)

[dsyme]

Thank you @tpetricek, @muehlhaus - adjsutments made!

[siavash-babaei]

Thanx for all the hard work on such an important feature. I did burst with happiness and what-not. I tested a simple script on both my systems:

Running in F# Interactive in Visual Studio
FS3053: The type provider 'RProvider.RProvider' reported an error: The type provider constructor has thrown an exception: ReflectionOnly loading is not supported on this platform.

Compiler Error in fsx File
FS3053: The type provider 'RProvider.RProvider' reported an error: ... Initialization of R.NET failed
FS3005: Referenced assembly "RProvider.dll" has assembly level attribute "TypeProviderAssemblyAttribute" but no public type provider classes were found

Systems
Windows 10 Workstation and Home
Microsoft R Open 4.0.2
Visual Studio 2019 Version 16.11.3
Microsoft Desktop Runtime 5.0.10 x86 and x64
Microsoft .NET Runtime 5.0.10 x6
R_HOME: C:\Program Files\Microsoft\R Open\R-4.0.2

Tested again with R 4.1.1 and the same errors ...

I checked the nuget gallery and it was last updated on 2019-03-18. Suppose the latest updated is not reflect in nuget.

[dsyme]

Have you enabled .net core scripting in VS too?

I'll get the nuget publish key added later tonight

[siavash-babaei]

Yep: Microsoft (R) F# Interactive version 11.4.2.0 for F# 5.0

I checked last night around 23:00 EST, 27 Sep:

#r "nuget: RProvider" still yields compiler error with ... R.NET failed initialization ...

[siavash-babaei]

Doesn't work for Windows and R 4.0.2 and later.

#r "nuget: RProvider"
#r "nuget: RProvider, 2.0.0-beta"
#r "nuget: RProvider, *-*"

All get compilers errors with R.NET initialization failed.

@AndrewIOM Thank you for the light. This tunnel was so dark & cramp. You essentially brought something valuable back from the dead. Hallelujah JesusIOM, no pun intended ...