Android: support Media Controls #539

123mpozzi · 2024-10-18T09:07:04Z

Description

Implement the media controls in Android (via the MediaSession API) to make the system display the current media information on the lock-screen, in notifications, and within the control-center.

Additionally, this PR also introduces background-playback support

PR-planning

✅ iOS part: iOS: support Media Controls #532
📍 Android part
Both will be merged together on a base branch

Overview pics

Changes

Implement media controls

Use a MediaSession service to hold a strong reference to the player instance
Add serializers for MediaControlConfig
Update manifest permissions to allow running the background service

Implement Background playback

Modify the PlayerView lifecycle to prevent auto-pausing the background playback on app minimize
- Fix playerEventRelay being null in some scenarios

Changes to samples

To allows having playback without the main app being opened

Media Controls enabled by default in all samples
- No background playback, so it pauses the app on minimizing it
Background Playback
- It does not pause the playback when minimizing the app
- Has media controls enabled (since we need a notification, we favor media controls over a simple notification)

Note

Note: both samples will destroy the player instance (hence destroying the media control widgets) when going back to the homepage. That is expected, a more advanced sample could be part of a follw-up (e.g. letting the widget live in the homescreen as well, until we open a sample with media session disabled).

Config Flags Design

MediaControlConfig.isEnabled decides whether the service starts or not
PlayerConfig.PlaybackConfig.isBackgroundPlaybackEnabled decides whether to detach the player from the view, basically handling whether the playback should automatically pause (when set tofalse) or not (when set to true)
- MediaControlConfig.isEnabled has to be true for background playback to work, since it needs a notification

Testing

Please do some manual testing, with a physical device if possible 🙏

Things to look for

Media Controls (use any Media Control sample apart from Background Playback)

widget is created
- in notification-centre
- in lock-screen
minimizing the app stops the playback
- when re-opening the app, the player data is synced (e.g. current playback time)

Background Playback feature (use the new "Background Playback" sample)

widget is created
- in notification-centre
- in lock-screen
minimizing the app does NOT stop the playback
- when re-opening the app, the player data is synced (e.g. current playback time)
does not work without media session being enabled as well (via MediaControlConfig)

Activity lifecycle changes

basic samples have no background playback
- by default
- after running the new samples
basic samples playerview is not empty after running the new samples

Checklist

🗒 CHANGELOG entry - entry added already in base PR

To implement lock-screen controls, a player instance must be 'kept alive', so that the player is not garbage-collected or destroyed by the system when the view is destroyed. There are 2 ways to do this: - init the player on the service itself (like in the Android SDK sample) - init the player as usual, and somehow pass it to the service, to store an additional reference This first version implements the second option. The app has 1 strong reference, and the service has 1 strong reference to the player. So that whenever the app dies, the background playback still goes on. Current Issues: - Whenever the app gets back, the player instance should get fetched back as well! - Another limitation with this implementation is that we're calling `setupMediaSession` on `player.ts`'s `inizitalize()`. But when changing sources (e.g. playback view, click back, playback view), the source in the media session does not get overridden with the source from the new view.

Via native by using player module Note: The actual media session module can just be a native one (like offline module) as it is just called via native (player module)

This module will be Android-only.

The old player was not getting destroyed when a new one is being put in charge of the media session

This has to be handled via the activity lifecycle. Took `BackgroundPlaybackScreen` from Android SDK samples as the example Also fix `playerEventRelay` being null in some conditions

As modules represent something native that can be re-used from the JS side, but this is not the case here

123mpozzi · 2024-10-18T10:26:02Z

android/src/main/java/com/bitmovin/player/reactnative/RNPlayerView.kt

+            EVENT_CLASS_TO_REACT_NATIVE_NAME_MAPPING,
+            ::emitEventFromPlayer,
+        )
+


I had to move playerEventRelay initialization here because it was null during the first invocations of onStop in the activity lifecycle.
This is due to the custom setPlayer we have, which also sets playerEventRelay

Limitation section will need an update once the lock-screen implementation into Android SDK is finished

123mpozzi · 2024-10-18T12:31:59Z

src/lockScreenControlConfig.ts

Note that the Limitations section is not modified.
It will be update once the lock-screen feature is fully released on the Android SDK

A 'fake' media session was being initialized here because the Android SDK by a design mistake had mediaSession as not nullable.

android/src/main/java/com/bitmovin/player/reactnative/RNPlayerView.kt

android/src/main/java/com/bitmovin/player/reactnative/converter/JsonConverter.kt

android/src/main/java/com/bitmovin/player/reactnative/PlayerModule.kt

android/src/main/java/com/bitmovin/player/reactnative/services/MediaSessionPlaybackService.kt

example/src/App.tsx

Pair-programming session w Mario Improve general player view and player management Fix: - Restore the correct player from media session - Media Session not getting destroyed as the `@ReactMethod onDestroy()` in `PlayerModule.kt` was nullifying the media session manager inside the promise block. However, the main thread was already killed from JS, so that code was unreached Spot and comment a bug in `PlayerView` creation in Android SDK side

As this config is not existing on Native, we have to set a default value directly here Because it may not be there in the Json

123mpozzi · 2024-11-06T13:46:33Z

Testing looks good. The only thing that I noticed is the "app gets killed in the background" scenario, where if we navigate to the home screen the playback correctly continues. Then we kill the app and the playback continues. When we open the app again and go to the same video we create a new media session, so there are effectively 2 active sessions playing back at the same time. I guess we should probably destroy the old session or continue the old one - up to discussion.

Address with the changes to state-management in 36e88e6

hawk23 · 2024-11-06T14:33:04Z

android/src/main/java/com/bitmovin/player/reactnative/PlayerModule.kt

@@ -25,6 +26,8 @@ class PlayerModule(context: ReactApplicationContext) : BitmovinBaseModule(contex
     */
    private val players: Registry<Player> = mutableMapOf()

+    var mediaSessionPlaybackManager: MediaSessionPlaybackManager? = null


As discussed offline, let's init the mediaSessionPlaybackManager once here, instead of every time there is a player created with media controls support enabled.

Address in cf7c34f and finally in f0b06d7

hawk23 · 2024-11-06T14:38:59Z

android/src/main/java/com/bitmovin/player/reactnative/RNPlayerViewManager.kt

@@ -296,6 +297,7 @@ class RNPlayerViewManager(private val context: ReactApplicationContext) : Simple
            block()
        } catch (e: Exception) {
            Log.e(MODULE_NAME, "Error while executing command", e)
+            // TODO: re throw


Please remove and create ticket for this

Address in 5549678

hawk23 · 2024-11-06T14:39:16Z

android/src/main/java/com/bitmovin/player/reactnative/RNPlayerView.kt

@@ -190,6 +215,7 @@ class RNPlayerView(
        // is responsible for destroying the player in the end.
        playerView.player = null
        playerView.onDestroy()
+        (playerView.parent as? ViewGroup)?.removeView(playerView)


That was just for debugging, please remove

Address in 5549678

123mpozzi · 2024-11-06T15:59:09Z

Since 6d62572 a patch is no longer needed for testing as the Media Session API is now part of a stable release

zigavehovec

Great work 💪 Only some nit left from my side.

zigavehovec · 2024-11-07T08:24:51Z

CHANGELOG.md

@@ -4,15 +4,16 @@

 ### Added

- `LockScreenControlConfig` to configure the lock screen information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen and within the control center
+- `MediaControlConfig` to configure the media control information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen, in notifications, and within the control center


Nit: Maybe a bit more specific?

Suggested change

- `MediaControlConfig` to configure the media control information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen, in notifications, and within the control center

- `MediaControlConfig` to configure the media control information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen and the Control Center on iOS and as a notification on Android

I'd prefer to keep it as it is since the widget also pops up in the lock-screen in Android as well

zigavehovec · 2024-11-07T08:29:05Z

android/src/main/java/com/bitmovin/player/reactnative/PlayerModule.kt

@@ -211,6 +222,7 @@ class PlayerModule(context: ReactApplicationContext) : BitmovinBaseModule(contex
     */
    @ReactMethod
    fun destroy(nativeId: NativeId, promise: Promise) {
+        mediaSessionPlaybackManager.destroy(nativeId)


Should this be inside the resolveOnUiThreadWithPlayer block?

We spotted a bug here: the media session was not destroyed if put inside the resolve block.

The issue is that, at this point, the main thread is already killed from JS, so if we put this inside the resolve block, the code is not reached

zigavehovec · 2024-11-07T09:34:54Z

android/src/main/java/com/bitmovin/player/reactnative/RNPlayerView.kt

+        // remove player from view so it does not get paused when entering background
+        private fun removePlayerForBackgroundPlayback() {
+            playerInMediaSessionService = null
+            playerView?.player?.let {


Nit: I'm not a big fan of using let for nullability checks as the code gets an extra indentation, consider replacing it with:
val player = playerView?.player ?: return

It's also generally nice to avoid it and name the parameter if the code is more than 1 line.

Address in b12d6b0

123mpozzi added 11 commits October 18, 2024 10:52

Move the player instance creation to the service

1579ed5

Bind the player correctly to the service on player creation

dc1f4bf

Via native by using player module Note: The actual media session module can just be a native one (like offline module) as it is just called via native (player module)

Remove media session module from typescript

f08ca29

This module will be Android-only.

Fix media session player not getting replaced properly

07a70c6

The old player was not getting destroyed when a new one is being put in charge of the media session

Introduce the lock-screen controls sample

257ca00

Revert changes to basic playback

ae88186

Fix auto-pausing when minimizing the app

140317a

This has to be handled via the activity lifecycle. Took `BackgroundPlaybackScreen` from Android SDK samples as the example Also fix `playerEventRelay` being null in some conditions

Fix every sample having background playback

904d8c0

Fix every sample having background playback after service is on

34f63e1

Cleanup code

7fde45f

123mpozzi self-assigned this Oct 18, 2024

123mpozzi changed the base branch from development to feature/enable-lock-screen-controls October 18, 2024 09:07

123mpozzi added 3 commits October 18, 2024 11:14

Cleanup unnecessary code and comments

5c04e05

Remove unnecessary decorator

ec76873

Make the connection manager a normal class instead of a module

9a39c49

As modules represent something native that can be re-used from the JS side, but this is not the case here

123mpozzi commented Oct 18, 2024

View reviewed changes

123mpozzi added 2 commits October 18, 2024 12:28

Add newline at end of the file

2d14fb7

Update docs with android details

40be6a7

Limitation section will need an update once the lock-screen implementation into Android SDK is finished

123mpozzi commented Oct 18, 2024

View reviewed changes

123mpozzi mentioned this pull request Oct 18, 2024

iOS: support Media Controls #532

Merged

1 task

123mpozzi marked this pull request as ready for review October 18, 2024 12:37

123mpozzi requested a review from a team as a code owner October 18, 2024 12:37

123mpozzi added 7 commits October 23, 2024 13:49

Simplify LockScreenControlConfig serialization

e239fce

Update docs for bg-playback support on Android

a3698fd

Add bg playback sample

44008f9

temp: tentative changes

64a572c

Unify most of the logic for bg and mediasession

96eafe9

Rename bg manager more generic and run ktlint

fca2dda

Remove unnecessary onCreate

08d0dd5

A 'fake' media session was being initialized here because the Android SDK by a design mistake had mediaSession as not nullable.

hawk23 reviewed Nov 6, 2024

View reviewed changes

123mpozzi added 6 commits November 6, 2024 09:27

Rename sample as well

b97c8c1

Fix wrong default value

2ee1f11

Remove default value in case of nil

d244a2c

Export bg playback handling in a func for onStop

8d2fdb3

Fix default value

2f09d63

As this config is not existing on Native, we have to set a default value directly here Because it may not be there in the Json

123mpozzi changed the title ~~Android: Enable Lock-Screen Controls~~ Android: Enable Media Controls Nov 6, 2024

123mpozzi changed the title ~~Android: Enable Media Controls~~ Android: support Media Controls Nov 6, 2024

123mpozzi added 2 commits November 6, 2024 15:11

Move statement inside block

e22981e

Remove media control sample since it is enabled by default

a702767

123mpozzi requested review from zigavehovec and hawk23 November 6, 2024 14:24

Move initialization of media session manager outside init block

cf7c34f

hawk23 reviewed Nov 6, 2024

View reviewed changes

123mpozzi added 3 commits November 6, 2024 15:39

Optimize var

f0b06d7

Document android limitation

ad495fb

Remove unnecessary code

5549678

123mpozzi requested a review from hawk23 November 6, 2024 14:45

Remove correct line

adb0ea1

hawk23 approved these changes Nov 6, 2024

View reviewed changes

Update to stable version having Media Session API

6d62572

zigavehovec approved these changes Nov 7, 2024

View reviewed changes

Do not use let for nullability check to save indentation space

b12d6b0

Base automatically changed from feature/enable-lock-screen-controls to feature/base-enable-lock-screen-controls November 7, 2024 10:34

123mpozzi merged commit 134e11e into feature/base-enable-lock-screen-controls Nov 7, 2024
9 checks passed

123mpozzi deleted the feature/android-enable-lock-screen-controls branch November 7, 2024 10:35

123mpozzi mentioned this pull request Nov 7, 2024

Support Media Controls #554

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android: support Media Controls #539

Android: support Media Controls #539

123mpozzi commented Oct 18, 2024 •

edited

Loading

123mpozzi Oct 18, 2024 •

edited

Loading

123mpozzi Oct 18, 2024

123mpozzi commented Nov 6, 2024

hawk23 Nov 6, 2024

123mpozzi Nov 6, 2024

hawk23 Nov 6, 2024

123mpozzi Nov 6, 2024

hawk23 Nov 6, 2024

123mpozzi Nov 6, 2024

123mpozzi commented Nov 6, 2024 •

edited

Loading

zigavehovec left a comment •

edited

Loading

zigavehovec Nov 7, 2024

123mpozzi Nov 7, 2024

zigavehovec Nov 7, 2024

123mpozzi Nov 7, 2024

zigavehovec Nov 7, 2024

123mpozzi Nov 7, 2024

	- `MediaControlConfig` to configure the media control information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen, in notifications, and within the control center
	- `MediaControlConfig` to configure the media control information for the application. When `isEnabled` is `true`, the current media information will be shown on the lock-screen and the Control Center on iOS and as a notification on Android

Android: support Media Controls #539

Android: support Media Controls #539

Conversation

123mpozzi commented Oct 18, 2024 • edited Loading

Description

PR-planning

Overview pics

Changes

Implement media controls

Implement Background playback

Changes to samples

Config Flags Design

Testing

Things to look for

Media Controls (use any Media Control sample apart from Background Playback)

Background Playback feature (use the new "Background Playback" sample)

Activity lifecycle changes

Checklist

123mpozzi Oct 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

123mpozzi commented Nov 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

123mpozzi commented Nov 6, 2024 • edited Loading

zigavehovec left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

123mpozzi commented Oct 18, 2024 •

edited

Loading

123mpozzi Oct 18, 2024 •

edited

Loading

123mpozzi commented Nov 6, 2024 •

edited

Loading

zigavehovec left a comment •

edited

Loading