GStreamer

[mjakeman]

This PR implements the groundwork for GStreamer, as well as substantial amount of GStreamer itself (enough for a music player, with caveats). It implements parts of GStreamer, GstBase, GstAudio, GstVideo, and GstPbutils.

This PR depends on:

• Full Interface Support #110: Interface Support (only partially)
• Closures Handling #119: Closures and Delegate Signatures

Remaining Tasks
Here's the laundry list of TODOs before we can merge this:

• Rebase on develop: Needs changes to work with Scriban 3.0 and our reworked templates.
• GIR Files: Regenerate relevant GStreamer gir files and add them to gir-files.
• Object Struct Fields: Decide on safe vs unsafe access to GObject fields
  • See GstObject->Flags for an example of safe vs unsafe access
• GhostPad Constructor Problems: GhostPad's constructor implements custom behaviour which we can't easily implement in a C# constructor (validation of construct time properties).
  • For now, I've created a static New() method.
  • A long-term solution to this could be C# 10 Factories, which are static methods that support object initialiser syntax (see https://www.youtube.com/watch?t=6135&v=kU-t1qsEbqk)
• Need design for GstIterator: How should this work with C# Enumerable/Enumerator? Tentative design in WIP: GstIterator #157.
• Need design for GstTagList: GstTagList needs a rethink now that implicit Values have been removed. (Edit: TagList's API has been simplified to use GValue directly for now. This seems to work fine).
• Decide on Public API: Mainly regarding access e.g. Error, WrapPointerAs, GetHandle, etc.
  • We also need to decide on GStreamerGlueGetProperty/GStreamerGlueSetProperty.
  • Rename WrapPointerAs and GetHandle to WrapHandle/GetHandle
• Fix Delegate Signatures (Closures Handling #119): We need a way to marshal between managed signatures and unmanaged signatures
  • As part of this, remove Timeout->FuncData as it does not work Dropped from this PR

Future Changes
GstMessage and many other structs depends on support for custom Value types in the generator. This will allow us to use ref struct rather than the complicated and unsustainable (maintenance-wise) marshalling I'm using at the moment. This can reduce method bodies from >20 lines to sub-5, as we won't need to explicitly marshal anymore. It's probably much safer this way as well, as there is little chance of memory corruption from forgetting to free or update something.

Fixes #112

@gircore/coreteam

[badcel]

I fully agree on all points. I have one suggestion regarding the SetProperty methods.

We already have a private void SetGProperty(Value value, string? name) method and a corresponding getter method. I would suggest that we rename those methods to SetProperty / GetProperty to create the desired overloads. Additionally we should make those methods protected so they are accessable from Element. This looks like the most simple and clean approach to me. I think we would avoid some kind of descriptor for now. We can add it later if we see the need.

On a side note, I don't know why the name parameter is last in the parameter list? This looks kind of strange to me. I would suggest switching the order of the parameters: protected void SetProperty(string name, Value value).

From my point of view just go ahead all your suggested changes / planings are great. So let's make smaller iterations and get this into develop 👍

---

Less important remark regarding Element.cs, there is this code:

 // We intentionally throw an exception if the type of value cannot be wrapped
 // TODO: Support boxing arbitrary managed types
Value val;
if (value?.GetType().IsAssignableTo(typeof(GObject.Object)) ?? false)
    val = Value.From((Object) value!);
else
    val = Value.From(value);
                    
GStreamerGlueSetProperty(property, val);

I think we should in some later point in time verify if this code could somehow be part of the GObject library so the protected SetProperty API allows to set arbitrary objects and is doing the right thing. The signature would be s.th. like protected void SetProperty(string name, object value). I'm not sure if we can reach that goal, but perhaps you just add a TODO flag and we reconsider later?

[badcel]

Great work, this is amazing 👍 It looks like it involved a lot of research work, too.

I did not test if it works as there are no samples. But as the code itself looks good we can fix any remaining issues as they arise.

Nevertheless I left several comments which we should discuss but there is no show stopper.

I think we should add some simple samples but this is for another PR.

[mjakeman]

That should be everything, although I might have missed something here and there.

I'm working on a few samples at the moment, I'll open a new PR for that once we land this one.

@badcel

[badcel]

Great work this is quite big 👍

[Inrego]

I just now found this project, and it does indeed seem very interesting. However, still only the few first plugins seem to be implemented. Are there any plans to support more plugins, and if yes, what's the timeline? From what I gather, you've made a rather impressive code generator, so it sounds like it could be as simple as grabbing the remaining gir-files and generate source based on them?
The source of gir files you've previously linked seems dead, but here's a newer source I've found that seems rather up-to-date:
https://gitlab.freedesktop.org/gstreamer/gir-files-rs

[badcel]

Thank you. Currently GStreamer is not a priority for me as there are other things missing.

For GStreamer itself it is possible that there is some manual written code missing. As I don't have any experience with GStreamer this is all a bit blurry for me. If you know your way around GStreamer I'm grateful for any contribution and willing to support new contributors to get the needed code landed.

Adding new GStreamer bindings requires adding them to our gir files repository first. To do this the corresponding files must be added to the configuration and perhaps the CI script a little bit adopted. If this is done the gir files get updated automatically.

The next step would be to use the new gir files in the generator.

If you are interested in certain plugins feel free to add them to the gir files repository like described above.

If you want to see improved GStreamer support adding more sophisticated samples would help to unveil missing bits and pieces.