using custom ALC for module written in PowerShell (not c#)

[gaelcolas]

Thank you for this repo, lots of information in there!
I'm trying to understand if there's a way to adapt those approaches in modules written in PowerShell.
Let me describe my scenario, what I understand, and what I think I don't know...

Scenario

I would like to fork & rename the Powershell-Yaml module to experiment with some things, while I still use the Powershell-Yaml module (i.e. for building that module).

• I'd like my fork to be able to use a newer version of the YamlDotNet assembly
• I don't want to wrap YamlDotNet in another assembly
• I think I need the assembly resolution to work at module parse time

What I think I understand

1. As the YamlDotNet library is compatible with netstandard 2.0, I can use the same dll for PS 5.1 AND PS 7+.
2. The dependency of YamlDotNet is at the 'class-level' (well, parse time for my new PS module) so it's closer to scenario 2 described in your repo here.
3. As per your sample:

the registration of AssemblyResolve needs to happen in a separate assembly (resolver.dll in this sample), which needs to be loaded before the above assembly"

What I'd like clarification/guidance on

Could the resolver DLL be loaded before parse time using RequiredAssemblies or ScriptToProcess in the ModuleManifest?
If I list the assemblies in the module manifest, like so, should it work?
RequiredAssemblies = @('resolver.dll','conflict.dll')
Will the loading order be respected (so that registration of AssemblyResolve happen first)?
Why don't you use this approach in your sample and prefer a nested module? Is it for the OnImport() call?
Are nested module (resolver.dll) loaded before the RootModule psm1 is parsed?
Since the module (let's call it MyModule) isn't a binary module, in IsAssemblyMatching() the requestingAssembly will be null? Does that mean I can't select which library will it resolve to, so anything calling it in my PowerShell session will now go to that newer lib?

Thanks for the clarification!

[SeeminglyScience]

Unfortunately there is no built in way to do this. The closest you can get is to manually create your own ALC, hold on to instances of the Assembly objects within, and only reference types via Assembly.GetType(string name). I wouldn't really recommend trying that, but it might work.

@daxian-dbw did some pretty extensive research into the possibility of adding support for script based ALC isolation, you can read about the blockers that were discovered in his RFC on the topic. The conclusion was that it is not feasible to support the scenario.

[daxian-dbw]

@gaelcolas I'm so sorry that I didn't notice this issue until Steve mentioned it within the team.

@SeeminglyScience is right, you cannot use ALC for a script module, unless some fundamental support is built into powershell, such as in type resolution, assembly loading, and caching/caching invalidation.

I wrote another RFC in 2021 trying to tackle this problem, which has lots of details about what needs to be done and what problems we will be facing. The RFC was closed in the end because of some usability problems and potentially breaking changes in behavior (see the summary).

[michaeltlombardi]

@daxian-dbw, @SeeminglyScience,

I wonder if instead of a general-purpose load/unload for PowerShell script modules themselves it would be possible to devise a pattern for a dependency wrapper module and make that easier.

For example, instead of being able to define RequiredAssemblies = @('YamlDotNet.dll') we could define RequiredAssemblies = @('MyModuleDeps.dll') which pulls in the dependencies we need, loads them via ALC, and surfaces them (in a new namespace? I'm not savvy enough about C# to have an idea on methodology here) for use in MyModule. We'd still need to define a binary module to go with our script modules, but we could abstract the dependency load/unload problem to that binary module.

One of the biggest drawbacks to the ALC model right now is that the majority of community modules are written purely in PowerShell without any C# and the community of practice doesn't have particularly strong experience with the toolchain for testing and publishing C# code.

Moving first-party module dependencies into the ALC (in my case, YamlDotNet for PlatyPS and MarkDig for the markdown renderer) goes a long way towards helping avoid conflicts for community modules, but doesn't fully resolve the problem. That leaves people with the choice of migrating to a fully binary module (much higher friction for typical PowerShell authors), trying to reimplement the functionality they need in PowerShell itself, or giving up on that functionality.

If there was a functional pattern for being able to get at those classes and methods without having to write the entire module in C#, I can think that would make using external libraries much less prone to conflicts, even if still higher friction than merely vendoring the library directly for usage.

As it stands right now, using any external libraries in a script module risks dependency conflicts and there doesn't seem to be a way around that without switching to a binary module all-up.

[daxian-dbw]

For example, instead of being able to define RequiredAssemblies = @('YamlDotNet.dll') we could define RequiredAssemblies = @('MyModuleDeps.dll') which pulls in the dependencies we need, loads them via ALC, and surfaces them for use in MyModule.

We already have the sample code for this pattern -- you basically will need a bridge assembly to wrap the dependency and expose the dependent types/APIs in a different way from the bridge assembly. Then the script module then can just depend on the bridge assembly.

However, having a bridge assembly to forward the calls to real dependency types/APIs has limitations and also increase complexity to the module design. It's non-trivial work to powershell users who are not too familiar with C#.

The dependency for script module should ideally be mitigated by PowerShellGet. In an ideal world, script modules would not need to ship dependency assemblies along with the module, but just declare it, and then it's the PowerShellGet's job to sort out the dependency and always use the latest version of the dependency assembly.

[SeeminglyScience]

We already have the sample code for this pattern -- you basically will need a bridge assembly to wrap the dependency and expose the dependent types/APIs in a different way from the bridge assembly. Then the script module then can just depend on the bridge assembly.

Dongbo laid it out excellently here but just in case you read this thinking it can be applied to the script portion of a module - you still can't directly reference the dependencies from script using this pattern. You would need to write a bunch of proxy APIs in order to use them in the script portion.