go2rtc with webrtc mode broken on Android devices?

[Jonnehs]

Checklist:

• Please try updating to the latest available version.
• Please try clearing your browser cache.

[REQUIRED] Card diagnostic information:

card_version: 6.0.0-beta.8
browser: >-
  Mozilla/5.0 (Linux; Android 15; Pixel 9 Pro Build/AP3A.241105.008; wv)
  AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/130.0.6723.107
  Mobile Safari/537.36 Home Assistant/2024.10.3-14058 (Android 15; Pixel 9 Pro)
date: 2024-11-30T14:32:52.693Z
frigate_versions:
  75bfa410040f18a96b467b3a6e2367ef: 5.5.1/0.14.1-f4f3cfa
lang: en_GB
timezone: Europe/London
git:
  hash: a869109
  build_date: Mon, 14 Oct 2024 00:01:23 GMT
  commit_date: Sun, 13 Oct 2024 16:50:47 -0700
ha_version: 2024.11.3
config:
  type: custom:frigate-card
  cameras:
    - camera_entity: camera.doorbell
      live_provider: go2rtc
      go2rtc:
        modes:
          - webrtc
  menu:
    buttons:
      microphone:
        enabled: true
        type: toggle
        alignment: opposing
      frigate:
        enabled: true
      cameras:
        enabled: false
      substreams:
        enabled: false
      live:
        enabled: true
      clips:
        enabled: false
      snapshots:
        enabled: false
      download:
        enabled: false
      camera_ui:
        enabled: false
      timeline:
        enabled: false
      media_player:
        enabled: false
      mute:
        enabled: true
    style: overlay
    button_size: 50
  performance: {}
  live:
    preload: true
    microphone:
      always_connected: true
  view:
    default: live

[REQUIRED] Description of problem:

The stream does not load on the android device. It works fine on a PC browser but on android the picture just keeps updating with snapshots most of the time, and the audio/microphone buttons do not appear. Occassionally they do appear after a very long wait, and the microphone/audio buttons do appear, but the feed is immediately frozen. This is on new/clean dashboard with only a single card for the doorbell feed. Another dashboard I have with all my frigate camera cards on it does load (not including the doorbell) and they play fine, so i dont think its a phone performance issue - more so that the go2rtc/webrtc functions/stream are not loading.

[OPTIONAL] Last working release (if known):

New configuration for Reolink PoE doorbell im testing to replace existing hardware. Want this frigate card to facilitate the notifications for doorbell presses with two way audio.

[dermotduffy]

The stream does not load on the android device. It works fine on a PC browser but on android the picture just keeps updating with snapshots most of the time, and the audio/microphone buttons do not appear.

This means your phone is not loading the WebRTC stream for some reason.

Occassionally they do appear after a very long wait, and the microphone/audio buttons do appear, but the feed is immediately frozen. This is on new/clean dashboard with only a single card for the doorbell feed. Another dashboard I have with all my frigate camera cards on it does load (not including the doorbell) and they play fine, so i dont think its a phone performance issue - more so that the go2rtc/webrtc functions/stream are not loading.

Exactly. Unfortunately, that could be for any number of reasons (only some of which are related to the card). I assume the Android is on the same wireless network as the PC/browser? If you're a developer, it would really help to get the Javascript console logs -- it might explain why it's not loading (that's not that easy to do if you're unfamiliar with the process though).

[Jonnehs]

I am not a developer, but I am willing to give it a go - if you can point me in the right direction or give me some instructions?

The phone is not on the same LAN as the phone/frigate, its a separate subnet but routable with no firewall restrictions. It works from an external browser though so I would assume the networking side is all handled through home assistant anyway?

[dermotduffy]

I am not a developer, but I am willing to give it a go - if you can point me in the right direction or give me some instructions?

It'll be a bit of a journey if you're not used to it. I'd start here: https://developer.chrome.com/docs/devtools/remote-debugging

The phone is not on the same LAN as the phone/frigate, its a separate subnet but routable with no firewall restrictions. It works from an external browser though so I would assume the networking side is all handled through home assistant anyway?

If it works on a browser externally, it should be fine on your phone too. Suggest trying both a browser on the Android device, and separately the Home Assistant companion app.

I would assume the networking side is all handled through home assistant anyway?

Yes. Everything you need should be routed via the Frigate integration running as part of your Home Assistant.

BTW: Might be good to try a different Android device if you happen to have one handy.

[Jonnehs]

Yes it doesn't work on the wife's phone either, same results. Doesn't work from a browser on the phone, neither externally off the wifi or local.

[Jonnehs]

Ok, what am I looking for in here? Network tab is spamming away but it seems to be just loading the new snapshot JPGs

[Jonnehs]

I have the dump from chrome://webrtc-internals in another tab while the connection was being attempted, don't really know what I am looking for. The webrtc internals seems to imply its repeatedly connecting and disconnecting.
webrtc_internals_dump (1).txt

[Jonnehs]

error eventually appears in the console;

live-go2rtc-fb41c006.js:1 WebSocket connection to 'wss://q69jqpnhip1eir89j8ozrkbdtza7hv2u.ui.nabu.casa/api/frigate/frigate/mse/api/ws?src=Doorbell&authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI0YTRmNzc2YWYxMzc0NDI2OGNiZWVkNDczYjNlNWY4MSIsInBhdGgiOiIvYXBpL2ZyaWdhdGUvZnJpZ2F0ZS9tc2UvYXBpL3dzIiwicGFyYW1zIjpbWyJzcmMiLCJEb29yYmVsbCJdXSwiaWF0IjoxNzMzMDUzNDU1LCJleHAiOjE3MzMxMzk4NTV9.CR7p-tvYQEu20Ycep6KcemjmDMfFcLzUKSR0OcwMHiA' failed: Error in connection establishment: net::ERR_NAME_NOT_RESOLVED
onconnect @ live-go2rtc-fb41c006.js:1
(anonymous) @ live-go2rtc-fb41c006.js:1

Name not resolved would be a dns error, I suppose? Makes no sense as the page is loading, why would the lookup for go2rtc be failing?

[dermotduffy]

Name not resolved would be a dns error, I suppose? Makes no sense as the page is loading, why would the lookup for go2rtc be failing?

That does indeed look like it might be related to your problem. It looks like the DNS resolution of your Home Assistant instance may be failing (the path /api/frigate/frigate/mse/api/ws is the path that will proxy to go2rtc behind the scenes, so it's failing before it even gets to go2rtc or to Frigate). Having said that, given that you are presumably connected to a dashboard on that same host (?), I cannot explain why your name resolution would suddenly stop working.

[Jonnehs]

Name not resolved would be a dns error, I suppose? Makes no sense as the page is loading, why would the lookup for go2rtc be failing?

That does indeed look like it might be related to your problem. It looks like the DNS resolution of your Home Assistant instance may be failing (the path /api/frigate/frigate/mse/api/ws is the path that will proxy to go2rtc behind the scenes, so it's failing before it even gets to go2rtc or to Frigate). Having said that, given that you are presumably connected to a dashboard on that same host (?), I cannot explain why your name resolution would suddenly stop working.

I would ignore the DNS error to be honest, think it was a red herring - it never came up again. Can't possibly be failing to resolve the name when the page is loading from that same host, and the snapshot JPGs on the card are updating. Cant get the error to reproduce since, and it behaves the same way using the IP internally on the wifi with no DNS involved so ..

I did notice my docker IPs for the frigate container are in the candidate responses, but that is true from a browser on the PC too and it connects and loads. Other than that I couldn't see anything obvious as an error.

[dermotduffy]

Just to be sure, I tried your config on my Android device, and I see the same (broken!) behavior. Works fine in browser, but not on Android. Something must have changed. Let me look into it.

[dermotduffy]

... and today I come back to this issue and it's all working fine on my Android device, with this simple config (status bar on so I can clearly see the webrtc logo):

type: custom:frigate-card
cameras:
  - camera_entity: camera.office
    live_provider: go2rtc
    go2rtc:
      modes:
        - webrtc
status_bar:
  style: outside
  position: bottom

I cannot explain why it didn't work yesterday. Leaving this open to hear from others, and/or see if I can reproduce...

[Jonnehs]

I dont see the webRTC logo with your status bar on mobile, it is there on PC browser. Can't really explain this at all to be honest, I presume it must be something to do with my setup but no idea how to track it down any further

[Jonnehs]

Ok, same phone, same circumstances - it works when i use the direct Frigate https admin UI. The mic button appears, stream instantly loads and i can do 2 way talk - how do we debug this further to find out what is breaking through the HASS UI, integration and/or frigate card?

[dermotduffy]

it works when i use the direct Frigate https admin UI

To clarify, you mean outside of HA entirely, or within HA & the card?

[Jonnehs]

it works when i use the direct Frigate https admin UI

To clarify, you mean outside of HA entirely, or within HA & the card?

Yeah, just browsing to the HTTPS address of frigate itself, bypassing HA and the card

[Jonnehs]

Hm, you know - updating to the 6.0.0 release has fixed this for me

Correction, it loads internally on LAN but if you turn off WIFI it still doesn't load the audio etc. But it does work on the LAN.

[dermotduffy]

Maybe try the latest release: https://github.com/dermotduffy/frigate-hass-card/releases/tag/v6.0.12

Bit of a long-shot, but it included some fixes to loading webrtc streams.

[Jonnehs]

thanks, I will, but it seems like this must be a network issue of some kind now as it works fine off my LAN/wifi but broken when on the network. Must be something to do with the webrtc candidates, but it works from a LAN web browser on PC - maddening issue to track down.