WIP: Investigate using CFFI's API mode

[mgeier]

This is an experiment to find out if switching to API mode is feasible.

It probably only makes sense if all of those items are fulfilled:

• works on Linux
• works on macOS
• works on Windows
• automatic wheel generation for manylinux 32/64-bit (travis-ci?)
• automatic wheel generation for macOS 64-bit (travis-ci?)
• automatic wheel generation for Windows 32/64-bit (appveyor?)
• all of the above again, just this time with PyPy
• Debian package (and in turn package for Raspberry Pi)

[mgeier]

This could give some inspiration for travis-ci and appveyor: https://github.com/pygame/pygame
They build ALSA manually on manylinux.

https://github.com/pypa/auditwheel can probably be used to handle the ALSA and JACK libraries for Linux.

And of course this could streamline the whole process: https://github.com/joerick/cibuildwheel
Or this: https://github.com/matthew-brett/multibuild?

There might be some interesting stuff here: https://github.com/MacPython/numpy-wheels

travis-ci could probably deploy to https://packagecloud.io/:
https://docs.travis-ci.com/user/deployment/packagecloud/
https://blog.packagecloud.io/eng/2017/01/17/continuous-delivery-python-applications-using-travis-ci-packagecloud/

See also https://github.com/mgeier/python-rtmixer/issues/1#issuecomment-307664621 and https://github.com/tgarc/pastream.

Some PyPy-related links:
https://stackoverflow.com/questions/30822873/how-do-i-install-pypy-on-appveyor
ogrisel/python-appveyor-demo#8

[tgarc]

hmm so what inspired you to change your mind? What do you really get from having a compiled soundfile module?

[mgeier]

I didn't change my mind. As before, I still think a compiled module is an option, and I'm still not sure if it's a good idea.
I just wanted to explore this a little more and see how it would look like.

What do you really get from having a compiled soundfile module?

I'm not sure. I would hope for less problems with broken libraries the user has already installed.
Also, it would allow to include the RingBuffer directly into the sounddevice module.
And if it's compiled anyway, I could probably implement sd.play()/sd.rec()/sd.playrec() with a callback implemented in C, making them more robust.

The downside would be (apart from the hassle of setting the whole thing up) that a lot of wheels would have to be created, albeit automatically.

Either way, whatever I learn here, I can and will probably use for https://github.com/mgeier/python-rtmixer/, where ABI mode isn't even an option.

[mgeier]

Here is an example where artifacts for Windows are created (and provided for download) by appveyor:
https://github.com/metabrainz/picard/blob/master/appveyor.yml

And another one using PyPy on Windows:
https://github.com/PyCQA/pyflakes/blob/master/.appveyor.yml

[larsoner]

The downside would be (apart from the hassle of setting the whole thing up) that a lot of wheels would have to be created, albeit automatically.

If there is any interest in coming back to this, cibuildwheel wasn't too bad to get working over on VisPy for wheel building and PyPi deployment.

I'm happy to help work on this, especially if it means that python-rtmixer's low-jitter features could eventually live in python-sounddevice directly. Not sure whether or not that was the plan...

[mgeier]

If there is any interest in coming back to this

I'm not sure for the sounddevice module. I think it is a great advantage that it can be installed without needing a compiler. This can be helpful on "exotic" hardware (assuming they have a pre-compiled PortAudio library) and on alternative Python interpreters.

And I'm actually not really sure about the advantages of switching from ABI to API mode.

Ideally, I'd like to support both, but I'm not sure if that's possible without extreme contortions.

I'm happy to help work on this

It would be great to have auto-generated wheels for https://github.com/spatialaudio/python-rtmixer!

A pull request would be very welcome. And please let me know when I need to set up an Azure account or whatever for that.

especially if it means that python-rtmixer's low-jitter features could eventually live in python-sounddevice directly. Not sure whether or not that was the plan...

I'm not sure either. But I'm open for suggestions and discussions.

But why would you need a combination of both?

Is there something you cannot do with python-rtmixer?

[larsoner]

But why would you need a combination of both?

Is there something you cannot do with python-rtmixer?

No, python-rtmixer does what I need (except for the jitter on Windows :) ). But in theory having the rtmixer functionality built into python-sounddevice, rather in a separate package, could help with performance, adoption, maybe other things. But it's not so critical.

[mgeier]

But in theory having the rtmixer functionality built into python-sounddevice, rather in a separate package, could help with performance, adoption, maybe other things.

Combining them would mean that the sounddevice module cannot be installed anymore without a compiler (on a system for which no pre-compiled wheel package is available).
I think this would actually be hurtful for adoption, or am I missing something?

As for performance, I don't really know if that's true. Did you try it? Does the performance improve when using API mode?

But it's not so critical.

I guess not. But either way, the python-rtmixer project is still kind of work-in-progress. It would have to mature quite a bit until a "fusion" with the sounddevice module could be seriously considered.

[larsoner]

Combining them would mean that the sounddevice module cannot be installed anymore without a compiler (on a system for which no pre-compiled wheel package is available).
I think this would actually be hurtful for adoption, or am I missing something?

At this point I mostly consider this a solved problem with wheels. You set up wheel builds for all standard platforms (Linux, Windows, macOS) and then you're good to go. But if there are more exotic platforms you're trying to support (Raspberry Pi?) then indeed it doesn't make sense.

As for performance, I don't really know if that's true. Did you try it? Does the performance improve when using API mode?

Haven't tried

But either way, the python-rtmixer project is still kind of work-in-progress. It would have to mature quite a bit until a "fusion" with the sounddevice module could be seriously considered.

Fair enough!

[mgeier]

But if there are more exotic platforms you're trying to support (Raspberry Pi?) then indeed it doesn't make sense.

I don't know any numbers but according to some issues here and some questions on stackoverflow there are at least some Raspberry Pi users interested in this.

I'm not explicitly supporting it (because I cannot test it myself), but I think it is nice if it does work.