Reintegrate Melody Mania

[achimmihca]

What does this PR do?

As promised, this PR reintegrates most of Melody Mania into UltraStar Play. 🎶

I will create follow-up tickets shortly for future work and user feedback.

Squashed Commits

I decided to save you the long list of history with thousands of commits.
Instead, there is just one squashed commit to get the current code base of Melody Mania.
Other commits in this PR are adjustments to make the code work again in UltraStar Play.

What is not included

Only things not included are

• third-party assets from the Unity asset store, because of license conflicts (Asset Store is not compatible with MIT license)
  • To compile without these libs, there are dummy implementations for the used API. You can opt-out of the dummy implementation via preprocessor define symbols, e.g. HAS_VLC_UNITY.
• AI tools, because they are integrated suboptimally in Melody Mania, which would result in GB of additional size.

What is added additionally

• Added a button in the main scene to inform interested players about Melody Mania such that players can have even more karaoke fun, plus they support my work financially so I can continue this. Hope this is ok for you.

Closes Issue(s)

• close online multiplayer mode #68
  • Minimal Viable Product (MVP) for online multiplayer has been added. It allows to connect with other players, select a song together, and see singing results of others. In Melody Mania this works via Steam integration. For USPlay, one has to do connect manually via IP address, possibly with manual port-forwarding.
  • What is not included is matchmaking and voice chat features.
  • The implementation exchanges some messages, see subclasses of NetcodeMessageDto. I must admit, I did not like the Unity online multiplayer API and probably worked more against it than with it. Probably suboptimal.
• close Support multiple channels in recording device (e.g. left and right microphone) using PortAudio #85
  • Support multiple channels via custom PortAudio integration for Unity.
  • This only includes pre-built PortAudio for Windows, not yet for Linux and macOS. Mobile platforms are not supported by PortAudio.
• close Remove voice from songs with Spleeter #106
  • Current Spleeter integration is done suboptimally, by making use of Spleeter CLI executable. Would be nice to run the model via Unity API directly. But this requires audio pre-processing.
  • Similarly, current integration of Basic Pitch is done suboptimally.
  • The executable CLI is not included here, because very big and not a good solution in the first place.
  • The current solution works for Melody Mania, but should be migrated to use Unity API and run on GPU before targeting more platforms.
• close API & App/Web Interface to manage USPlay  #176
  • There is an http endpoint to change USPlay settings. The endpoint is used by the Companion App for example.
  • Improving the HTTP API is future work
• close Medley support #238
  • Medley Support has been added
• close Jukebox feature #242, close Solo/Band quickplay mode / jukebox mode with spontaneous singing #405
  • There is a mod for a similar Jukebox feature. This is not enabled by default though.
  • Feel free to improve and extend the mod as needed.
• close Party modes #243
  • Party mode (aka. Team & Tournament) has been added
• close Song Queue #247
  • Song Queue has been added
• close Support reading the audio track of the video file as #MP3 #272
  • You can use a supported video format also as audio file
• close Assign mic to player by singing into the mic #297
  • You can sing into a mic to assign it to players
• close Folder view in song selection menu #321
  • You can navigate your song lib by folder
• close Audio desyncs from Video & Lyrics on some songs when using the "Skip to next lyrics" feature #323
  • Audio should be in sync with current Unity version
• close Advanced search UI, add an option to have sublevels for Artists / Folders / Editions / Years / Genre #325
  • You can use C# expressions to filter songs. There are also some new checkboxes to filter the song list.
• close Support more audio and video file formats by integrating LibVLC or ffmpeg #362
  • I integrated libVLC and ffmpeg from Unity asset store. However, these cannot be included in this repo because the Asset Store license is not compatible with MIT license.
  • Would need a different libVLC or ffmpeg integration for Unity. But this is an undertaking by iteself and thus out of scope. Feel free to do this, it would be greatly appeciated.
• close Sing all duet lyrics with single player #363
  • You can select to sing both lyrics of a duet song (makes sense only for songs without overlapping notes)
• close M3U as playlist format #371
  • You can use m3u as playlist format, but read-only
• close Song Select: Replace character quick jump with scrollbar #372
  • New UI made this obsolete
• close Import MIDI file with lyrics events (aka. KAR file) #380
  • You can import MIDI file with lyrics in the song editor. You can even play a MIDI file with lyrics directly. However, some MIDI files may have issues because the used lib for MIDI support has some limitations.
• close Navigate main game using the companion app #381
  • The Companion App has controls to simulate keyboard and mouse events in the game
• close Create Steam release #382
  • Melody Mania is on Steam
• close Refactor user input fields #388
  • UI has been refactored
• close Failed to create initial recording device profile #391
  • I think I fixed this at some point, not 100% sure though
• close Runtime loaded scripts for user defined code (modding) #394
  • You can alter the game by using custom C# code that is loaded at runtime. Note however that this will not work on AOT builds (e.g. IL2CPP, iOS, Android for Google Play store)
• close Companion App on iOS does not connect to main game #400
  • The Companion App for Melody Mania that can be found on the Apple App Store has the required permissions to do the UDP broadcast.
• close Mic-assignment for Companion App #407
  • The Companion App is assigned to a player automatically when their names are equal
• close Import covers from folder without #COVER-Tag in txt (feature request) #408
  • I think there is a mod for this. However, this is not enabled by default.
• close Resolution change leads to wrong position of mouse pointer #410
  • RenderTextures are re-created when screen resolution changes
• close Monospace font instead of proportional font for song timeline #411
  • New UI made this obsolete
• close Loop short videos #416
  • If a background video has a duration of less than half the duration of the audio file, then the background video will loop
• close Import lyrics in LRC format #421
  • You can import lyrics in LRC format in the song editor
  • Additionally, there is a feature to split words on syllable using NHyphenator.
• close Desync of Song Playback and Midi/Lyrics in Editor when using variable bit rate (VBR) #427
  • Audio should be in sync with current Unity version

Additional Notes

🎤 This is a major milestone for the project and me. 🌟
It includes all the features that I had envisioned back then when I created the Kickstarter, plus integration of AI tools to make song creation easier.

What matters more for me though is the personal growth that I gained from this project.
Not only did I broaden my skillset with a range of interesting technologies, but I also learned some important lessons for life.
I now know better what and how I want to continue with respect to my career.

From Steam I have some numbers to quantify the potential that I saw initially in this project. I still think there is much more potential.

But from now on I want to distribute the work across several shoulders.
I tried to do way to much for one person: dev, ops, cross-platform, QA, support, marketing, brand, legal, tax, ...

I plan to keep maintaining Melody Mania.
However, with the reintegration into UltraStar Play, I am looking forward to contributions from the community. This allows me to shift focus to new horizons, e.g., to spead the word, create an (often requested) video tutorial for song creation with AI, or port the core game to a webapp with YouTube integration.

Thank you for your support over the years!
The positive feedback that I received for UltraStar Play and Melody Mania has been a heartwarming motivation for me.

[bohning]

Congrats on this massive project!

[marwin89]

Thanks so much Andreas! This is very important for the whole scene.
When I'm back I try to find some developers with unity skills to totally complete and layout ultrastar play.