2916: Add tts on android

[bahaaTuffaha]

Short description

Adding text-to-speech functionality.

Proposed changes

• TtsPlayer.tsx : deals with html content and it passes it to react-native-tts as sentences. used debounce function from lodash to handle rapid volume change. I added an expandPlayer state just to keep player expanded when changes from sentence to sentence. This component located at app level where it wraps around the NavigationContainer to be able to be floating at the bottom of the screen while scrolling.

• useTtsPlayer.ts: Is a hook used to pass data (content, title, visibility, volume) from across the app to TtsPlayer.tsx using useContext.

• Categories.tsx, News.tsx, Events.tsx each one is using the useTtsPlayer.ts hook to pass content and title.

• Created a component Slider.tsx instead of adding more libraires and used the ones we have: react-native-reanimated & react-native-gesture-handler to create slider that can be used anywhere other than the volume.

• Added ReadAloud HeaderButtonTitle to be selected from the kebab menu to show the player. This button will show up only in Integreat/IntegreatTestCms and at third level pages.

• The TtsPlayer will stop when the app is in the background also if you back out from a 3rd level page.

• Notes:

  • Not Tested on iOS !
  • Why I passed sentences to tts.speak()? due to react-native-tts doesn't have a functional pause also passing word by word will sound more robotic.
  • Persian is not supported at react-native-tts.
  • Pausing on words is not possible cause there is no way to know when its finished reading word and pausing is not supported for android.

Side effects

• Categories.tsx, News.tsx, Events.tsx, Header.tsx

Testing

• At integreat : Go to any 3rd level content in categories or news.
• Open the kebab menu on the right and select Read aloud.
• The player will show up press play to make it start reading content.

Resolved issues

Fixes: #2916

---

[LeandraH]

Works pretty well so far! A couple of points about the functionality:

• If I change the volume while it's playing, it starts re-reading the current fragment. If possible, I think it should keep reading at the same spot.
• In using, I thought the volume bar was for updating the speed of reading. Maybe that's just me but maybe not, we should see if other people have that problem.
• For Persian (and any other languages where it doesn't work), I think we should hide the button to read the text aloud, and close the overlay if it's open when someone switches languages.
• In emulated iOS, no sound comes out and I get this warning: Excessive number of pending callbacks: 501. Some pending callbacks that might have leaked by never being called from native code: {"1000":{},"1001":{},"1002":{},"1003":{},"1004":{},"1005":{},"1006":{},"1007":{},"1008":{},"1009":{},"1010":{},"1011":{},"1012":{},"1013":{},"1014":{},"1015":{},"1016":{},"1017":{},"1018":{},"1019":{},"1020":{},"1021":{},"1022":{},"1023":{},"1024":{},"1025":{},"1026":{},"1027":{},"1028":{},"1029":{},"1030":{},"1031":{},"1032":{},"1033":{},"1034":{},"1035":{},"1036":{},"1037":{},"1038":{},"1039":{},"1040":{},"1041":{},"1042":{},"1043":{},"1044":{},"1045":{},"1046":{},"1047":{},"1048":{},"1049":{},"...(truncated keys)...":451} But I guess you did say that it's not yet tested for iOS :D

As discussed, I only half-checked the PR, let me know when it's ready for review again :)

[f1sh1918]

i couldn't test it since its crashing for me on emulator when i click play on Android 13

[f1sh1918]

Works for me on android. good work.
Since its a draft some improvements are missing i guess

Shouldn't here stand the title of what will be read?
Even add shadows/elevation for the container and the play button

The icons should switch i guess since the direction arrows are wrong

I think it would be better to create a fixed area for this player instead of overlapping content, but this may be also done in an additional tasks, since it will need some effort.

[LeandraH]

It unfortunately also doesn't play any sound on real iOS, there is just this warning that pops up:

[bahaaTuffaha]

It unfortunately also doesn't play any sound on real iOS, there is just this warning that pops up:

Maybe I should define iosVoiceId: '', for ios
https://github.com/ak1394/react-native-tts?tab=readme-ov-file#speaking

[LeandraH]

It unfortunately also doesn't play any sound on real iOS, there is just this warning that pops up:

Maybe I should define iosVoiceId: '', for ios https://github.com/ak1394/react-native-tts?tab=readme-ov-file#speaking

Hmm, putting in the example 'com.apple.ttsbundle.Moira-compact' didn't change the behavior.

There is also a very minor bug that I found: when opening the TTS player, the line with the last update jumps down, and back up when you close it. I attached a video of the behavior.

Simulator.Screen.Recording.-.iPhone.16.-.2024-10-28.at.15.34.08.mp4

[bahaaTuffaha]

It unfortunately also doesn't play any sound on real iOS, there is just this warning that pops up:

Maybe I should define iosVoiceId: '', for ios https://github.com/ak1394/react-native-tts?tab=readme-ov-file#speaking

Hmm, putting in the example 'com.apple.ttsbundle.Moira-compact' didn't change the behavior.

There is also a very minor bug that I found: when opening the TTS player, the line with the last update jumps down, and back up when you close it. I attached a video of the behavior.

This behavior is intended to leave space for the player to prevent it from covering the content.
I think iosVoiceId should be taken from voices()
Tts.voices().then(voices => console.log(voices));
The voices should be filtered by the language and passed to speak.

[LeandraH]

This behavior is intended to leave space for the player to prevent it from covering the content.

That's a good idea but leads to jumping in this case. The padding should probably be added under the last update stamp instead of before.

[LeandraH]

I think iosVoiceId should be taken from voices()
Tts.voices().then(voices => console.log(voices));
The voices should be filtered by the language and passed to speak.

I unfortunately still don't get any sound out. But the iOS can be fixed in a separate issue, I think you can for now work on getting the PR ready for reviewing for Android :)

[steffenkleinle]

Thanks for the PR, looks quite good already and I like the implementation. I just think that the TtsPlayer component should be split up to separate concerns and increase readability.

[steffenkleinle]

❓ Are these all icons from a design on figma?

[bahaaTuffaha]

If you noticed this icon in Figma has size 28x28 so I had to change it to 24x24 and clean it up...
some Icons like the play button is not separated (black circle, play icon) so I had to improvise 😅.
I use Inkscape and svgomg for cleaning.

[steffenkleinle]

I think it would be cool to request cleaned up ones from UI/UX instead as they have some weird or most likely unnecessary syntax/clipping rules.

[bahaaTuffaha]

sure

[steffenkleinle]

🔧 Do we really need three new colors? I'd prefer to reuse existing ones (at least where applicable).

[bahaaTuffaha]

Yes ... and these shades could make a difference in the look.

[codescene-delta-analysis]

✅ Code Health Quality Gates: OK

Change in average Code Health of affected files: +0.08 (9.70 -> 9.78)

View detailed results in CodeScene

[steffenkleinle]

Just tested it with english and this works really good, nice job and thank you for taking on this big thing!

Nevertheless, I think your code still needs some polishing and cleaning up. Especially the concerns should be clear and listeners and useEffects should only be used if they are really necessary (in my opinion this should only be the case to initialize everything and for the tts-finish listener to continue reading. If you have any questions about my comments or need some help, just let me know!

[steffenkleinle]

This looks really good already, thanks for taking care of all those comments! A little more to go but we are getting there :)

[steffenkleinle]

🔧 Both setIsPlaying and sentenceIndex should not be necessary anymore, could you adjust TtsPlayer accordingly?

[bahaaTuffaha]

Passing sentenceIndex is needed for playPrevious(sentenceIndex) and playNext(sentenceIndex)

[steffenkleinle]

There is no need to pass sentence index to these functions, they can just directly read and use it

[bahaaTuffaha]

If I pass it directly it won't keep playing.. it will stop at an index for some reason.

[steffenkleinle]

One more thing: Personally I'd be in favor of always showing the header item read aloud if the feature is available (that is, the current language is supported and it is enabled in the build config). So in contrast to now, we would also show the header item even if there is nothing to read. The player would then just say nothing to read. That would keep the UI mostly consistent (except if switching to an unsupported language).
@hauf-toni do you have a different opinion on this? At the moment the header item read aloud is only shown if the current page also has something to read.