WebRTC stream freezes

[vuminhkh]

Hello,

I'm writing a react native app using react-native-webrtc to communicate with go2rtc via frigate and home assistant. It works great in webrtc mode on my Tapo C210, Hikvision doorbell, but on my Annke FCD600 cameras with high resolution 3632 x 1632 in H264, the video stream freezes every seconds or even don't start at all (in debug log the stream receive mute event frequently and takes really long time to establish the connection, the bi directional audio stream is ok). If I reduces the bit rate drastically, then it works a little bit better but it still freezes from time to time, in night mode it freezes less than day mode. In the same time on all other cameras, it works great, if you guys have an idea how to debug this, I'll be very thankful. Basically, if i'm not mistaken Annke is just Hikvision rebranded, so I'm not sure what can be different between my Annke stream and Hikvision doorbell streams. I'm using go2rtc 1.8.5. Thank you in advance for your help.

[vuminhkh]

The hikvision doorbell's video in 1920x1080 also freezes from time to time.

[AlexxIT]

It depends if you using WebRTC inside or outside your home network.

[vuminhkh]

I have the same problem inside or outside my network. I checked the stream configuration when i'm inside my network, it's IP v6 UDP which is used. Anyway with Tapo C210 (2304 x 1296), I have the best streaming experience, never freezes inside or outside. For the moment, for my Annke cameras, I get the video stream from home assistant in HLS mode (with 3, 4 seconds lag but no freezing) and audio + microphone stream via WebRTC as a work around. I really wish to be able to use WebRTC though but I don't know how to optimize to avoid freezing.

[AlexxIT]

Have you read recommendations?
https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-rtsp

[vuminhkh]

Thanks, I prefix the rtsp stream with ffmpeg, it's still 'glitchy'
glichy_camera: ffmpeg:rtsp://username:password@192.168.1.123/live/ch00_1

[vuminhkh]

I don't know how yet, but frigate on safari on my phone works better than react-native-webrtc. It's perhaps the library which has issues, but I still don't know why the tapo stream works well with react-native-webrtc but not the annke streams.

[vuminhkh]

but in the same time, it's glitchy both on ios and android.

[AlexxIT]

Have you tried very glitchy version?

[vuminhkh]

I tested with very glitchy version also with the same result. I even run a local video file with go2rtc ffmpeg:/media/frigate/recordings/2024-05-28/08/visio_phone/31.28.mp4 with the same glitchy result. Recently, I moved to a new edge erpoe5 router, and new wifi nova mw6 wifi mesh. On go2rtc remote address is "remote_addr": "udp4 prflx 192.168.2.44:61564 related :0",. My camera is wired, with my Mac also wired then the video is smooth, but on wifi, the stream is glitchy. I then performed some iperf3 udp stress test 5 udp streams for a total of 100 Mbps in 30 seconds, there are no packet loss via wifi but slightly bigger jitter
Via wifi

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.01  sec  71.5 MBytes  20.0 Mbits/sec  1.045 ms  0/51764 (0%)  receiver
[  6]   0.00-30.01  sec  71.5 MBytes  20.0 Mbits/sec  0.654 ms  0/51743 (0%)  receiver
[  9]   0.00-30.01  sec  71.5 MBytes  20.0 Mbits/sec  0.624 ms  0/51743 (0%)  receiver
[ 11]   0.00-30.01  sec  71.5 MBytes  20.0 Mbits/sec  0.592 ms  0/51743 (0%)  receiver
[ 13]   0.00-30.01  sec  71.5 MBytes  20.0 Mbits/sec  0.606 ms  0/51743 (0%)  receiver
[SUM]   0.00-30.01  sec   357 MBytes  99.9 Mbits/sec  0.704 ms  0/258736 (0%)  receiver

Via ethernet

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.00  sec  71.5 MBytes  20.0 Mbits/sec  0.569 ms  0/51794 (0%)  receiver
[  6]   0.00-30.00  sec  71.5 MBytes  20.0 Mbits/sec  0.569 ms  0/51794 (0%)  receiver
[  9]   0.00-30.00  sec  71.5 MBytes  20.0 Mbits/sec  0.594 ms  0/51794 (0%)  receiver
[ 11]   0.00-30.00  sec  71.5 MBytes  20.0 Mbits/sec  0.593 ms  0/51794 (0%)  receiver
[ 13]   0.00-30.00  sec  71.5 MBytes  20.0 Mbits/sec  0.595 ms  0/51794 (0%)  receiver
[SUM]   0.00-30.00  sec   358 MBytes   100 Mbits/sec  0.584 ms  0/258970 (0%)  receiver

Do you know if with just slightly more jitter and no udp packet loss, it can explain the glitchy stream ? Thanks.

[vuminhkh]

I think about buying more robuste matos like ubiquiti wifi but the perf tests make me wonder, because the difference is not at all significant between wifi and ethernet for 358 Mbytes in 30 seconds.

[AlexxIT]

You can disable UDP for WebRTC in latest go2rtc
https://github.com/AlexxIT/go2rtc/blob/master/internal/webrtc/README.md

[vuminhkh]

yes, I confirm that no lag when it's tcp. Do you have an explanation ? My stress test seems to show that I don't have any issue in UDP with my Wifi but perhaps packets arrive a little bit more in disorder over Wifi and it makes react native webrtc too much effort to reorder packets ? In the same time TCP does not work when i'm not in my network as I block everything except for the port 80 and 443 for home assistant. I'll try from react native side to choose exclusively TCP when Home Assistant (and so Frigate) is in the same network as my cellphone. But when i'm outside of my network, do you have a solution ? Thanks.

[vuminhkh]

Anyway the glitchy problem also applied to frigate from chrome on mac os, so it seems to be not related uniquely to react native webrtc.

[AlexxIT]

A lot of people complain about WebRTC UDP for high bitrates on external access.
If the bitrate is very high and the channel is bad enough - you won't get a picture at all.
But I haven't heard of any complaints on the local network.

Better solution is to use UDP for local and TCP for external.

webrtc:
  listen: :8555/tcp  # this is default
  ice_servers: []  # this is disable STUN/UDP external access (enabled by default)
  candidates:
    - stun:8555  # this is add external TCP candidate for your public IP

Also you need to port forward 8555 TCP port on your router if you has public IP-address.

[vuminhkh]

You are right, I'm surprised myself, UDP over 4G even works better than my local wifi. I live at least 10 meters from my nearest neighbour and stress test showed decent speed and jitter. I'll try anyway to invest in some serious Wifi matos to see if it helps.