Notification is slow after update to stable

[balajeek]

Describe the problem you are having

I was running frigate 0.9.4-amd64 for long time, 2 days ago i decide to update and changed the image to frigate:stable which updated to (0.11.1-2eada21)

since then the notification is very slow, for example when someone arrives at driveway i get notification around 40 to 90 seconds later, before it use to be instant and i can look thru the windows as when it happens.

the notification automation is set thru blueprint on home assistant. I created another notificaiton automation for another camera and its also the same, takes time to arrive.

Where can i start my troubleshooting?

Version

(0.11.1-2eada21)

Frigate config file

not sure if you needed for my issue

Relevant log output

No errors in the frigate log that i notice

FFprobe output from your camera

not a camera issue

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

different models

Any other information that may be helpful

No response

[NickM-27]

There's a new notification blueprint for 0.10+ so recommend switching to that first

[balajeek]

i removed the automations and downloaded the new blueprint and setup the automation for notification, will check for a day when there is activity and update you on that.

Also when i went to HACS from HASS it showed me that i am running Frigate v2.3 and v3.0 is available. I believe its the Frigate integration for home assistant. would it be safe to update? becaz of some warning message that shows about some changes: https://github.com/blakeblackshear/frigate-hass-integration/releases

[NickM-27]

If you are running frigate 0.11 then you should be on 3.0 for the integration

[balajeek]

Great, going ahead with the update.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[balajeek]

Actually I updated integration i have been testing for some days and indeed the notifications are delayed around 30-50 seconds, Where can i start to troubleshoot?

[NickM-27]

Is it the notifications or the automation that is delayed? If it's the notification then it's going to be because firebase is being slow to deliver the notification. If you enable the local websocket for notifications it'll use more battery but should be immediate.

[balajeek]

I am not sure if i understand, I am a software developer and technically very good, but have no expertise on tools that i don't know.

1. how can i find if automation is delayed?
2. I don't even know if frigate uses firebase
3. when you say enable local websocket you mean in the camera itself?
4. finally i don't use any device on battery

It was all working great until i updated the latest frigate.
Any troubleshooting tips will be helpful. Thank you.

[NickM-27]

I am not sure if i understand, I am a software developer and technically very good, but have no expertise on tools that i don't know.

1. how can i find if automation is delayed?

Check home assistant automations, does the automation show a trigger time when the event happens or after

2. I don't even know if frigate uses firebase

It's not frigate that uses firebase it is home assistant which uses firebase to notify mobile devices

3. when you say enable local websocket you mean in the camera itself?

Talking about home assistant local websocket for notifications

4. finally i don't use any device on battery

What devices are you notifying then?

[balajeek]

I still did not get time to investigate between frigate video detection and hass notification to find which cause delay. But I noticed something today, while i was working from home today, sitting facing the window, i saw the garbage truck pass by, i opened the frigate ui and did live video feed, i notice the live feed is actually a full minute delay. What could be the problem?

[NickM-27]

If it's a full minute behind then there's either something going on with the camera or if you're proxying the connection something with that

[balajeek]

I am going to check on another camera and see what happens, will be back shortly...

[balajeek]

The other camera (front door) seem work fine, no delays, when i walked i got the alert also with 5 sec.
So i guess its the driveway camera, may be i should power cycle the camera to see if it have any benefit.

[balajeek]

So, i did power cycle, then looked at frigate ui live feed, the camera time and my computers time where in sync. so i thought problem solved. then i walked in front of camera to trigger the notification, did not get any, came back home (approx min later), and i see myself walking in front of the camera on live feed, and i notice the delay of min even on the live feed. So its confirmed thats the delay its causing my notification also delayed.

The camera is Wyze cam, its been running for a long time before even i updated the frigate to lastest. I opened up wyze app and there the time is in sync with my computer, but on frigate ui live feed its about 1+ min delayed.

[NickM-27]

I've had a lot of problems with wyze cams personally and really don't recommend them. Comparing the wyze app to the rtsp feed is not a valid comparsion because the app does not use rtsp and wyze cameras do not support rtsp except for a number of different third party solutions.

[balajeek]

Yes, i understand, and i have heard before about wyze not being reliable, it was working fine for me till this issue and i will at some point move to better camera soon.

I also agree wzye app vs rtsp comparision is not true, I tried open rtsp from vlc and vlc shows fine.

[NickM-27]

Pasting your frigate config may help give more info

[balajeek]

Here is my full config, the camera in question is "driveway"

mqtt:
  host: 192.168.99.13
ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

database:
  path: /media/frigate/clips/frigate.db

cameras:
  #garage:
  #  ffmpeg:
  #    inputs:
  #      - path: rtsp://<redacted>
  #        roles:
  #          - detect
  #          - rtmp
  #  detect:
  #    width: 1920
  #    height: 1080
  #    fps: 5

  backyard:
    ffmpeg:
      inputs:
        - path: rtsp://<redacted>/backyard
          roles:
            - detect
            - rtmp
            - record
            - clips
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - animal
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: true
      retain:
        default: 3
        objects:
          person: 3
          animal: 3
    record:
      enabled: true
      retain_days: 3
      events:
        #post_capture: 10
        retain:
          default: 3
          objects:
            person: 3
            animal: 3
            vehicle: 3
    #motion:
    #  mask:
    #    - 760,671,1113,829,1492,1020,1920,1013,1920,1080,1562,1080,1443,1080,1358,1080,957,1080,0,1080,0,656
    #    - 0,0,0,269,1920,290,1920,213,1920,0,0,0
    #zones:
    #  zone_driveway:
    #    coordinates: 1376,446,1920,653,1920,986,1723,991,1530,992,1290,929,982,742,811,676,675,646,201,599,26,571,26,493


  front:
    ffmpeg:
      inputs:
        - path: rtsp://<redacted>
          roles:
            - detect
        - path: rtsp://<redacted>
          roles:            
            - rtmp
        - path: rtsp://<redacted>/live
          roles:            
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - animal
        - vehicle
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: true
      retain:
        default: 7
        objects:
          person: 7
          animal: 2
          vehicle: 3
    record:
      enabled: true
      retain_days: 7
      events:
        #post_capture: 10
        retain:
          default: 3
          objects:
            person: 3
            animal: 1
            vehicle: 3
    motion:
      mask:
        - 460,600,468,839,171,838,94,677
        - 1187,263,1169,168,1125,178,1139,268
        - 704,0,704,49,486,47,481,0
        - 360,88,416,82,428,73,421,38,367,40,343,41,324,26,302,36,298,59,320,70,299,88,286,104,256,102,243,124,252,245,291,236
        - 91,339,78,240,124,217,128,156,108,127,92,95,134,75,203,42,281,0,0,0,0,220,63,366
    zones:
      zone_front_walkway:
        coordinates: 1277,1052,1148,162,800,163,440,272,244,1013

  driveway:
    ffmpeg:
      inputs:
        - path: rtsp://username:[email protected]/live
          roles:
            - detect
            - rtmp
            - record
            - clips
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - animal
        - vehicle
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: true
      retain:
        default: 7
        objects:
          person: 3
          animal: 2
          vehicle: 3
    record:
      enabled: true
      retain_days: 7
      events:
        #post_capture: 10
        retain:
          default: 1
          objects:
            person: 1
            animal: 1
            vehicle: 1
    motion:
      mask:
        - 708,775,1051,907,0,553,1920,1013,1920,1080,1562,1080,1443,1080,1358,1080,957,1080,0,1080,0,700
        - 0,0,0,269,1920,290,1920,213,1920,0,0,0
    zones:
      zone_driveway:
        coordinates: 1486,586,1920,653,1920,986,1723,991,1395,1000,1133,1056,932,834,812,812,721,781,594,762,226,736,0,755,0,581
      zone_road:
        coordinates: 916,444,1074,446,1220,436,1452,432,1810,447,1892,521,0,554,0,423

detectors:
  coral:
    type: edgetpu
    device: usb
  #cpu:
  #  type: cpu

[NickM-27]

You can try using udp instead for that camera and see https://docs.frigate.video/configuration/camera_specific#udp-only-cameras

[balajeek]

so i just copy this line?

ffmpeg:
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport udp -timeout 5000000 -use_wallclock_as_timestamps 1

[NickM-27]

Yes, apply it to the camera that is facing this issue

[balajeek]

I tried, i get just green color in the camera feed. and I also see this error in the logs:

ERROR : driveway: Unable to read frames from ffmpeg process.
ERROR : driveway: ffmpeg process is not running. exiting capture thread...

[NickM-27]

Looks like the camera doesn't support udp then

[balajeek]

Well, i guess its time to change the camera then, any recommendation? thats not super expensive

[NickM-27]

Depends what you're using it for. I replaced my indoor wyze cams with eufy C24 and have been happy with it.

[balajeek]

Another thing i noticed just now, my eufy 360 camera on the backyard is actually do the delay of 2+ min.
on eufy app and on vlc its fine, but on frigate its 2+ min delay.

[balajeek]

does eufy c24 does 24/7 rtsp?

[NickM-27]

Maybe your computer that runs frigate is having some issue processing the network data or something. And yes the eufy c24 supports 24/7 rtsp out of the box

[balajeek]

Maybe your computer that runs frigate is having some issue processing the network data or something. And yes the eufy c24 supports 24/7 rtsp out of the box

its actually running on qnap 451, i never had issues before as i mentioned before. It all started with me updating it to latest frigate. I will look into to see if any network related issues exist.

[balajeek]

I ordered the eufy c24 yesterday and its delivered today, going to setup this camera and replace the wyze. any specific setting i should set?

[NickM-27]

You'll need to enable the RTSP feed in settings.

[balajeek]

Yes I will have to do that, anything specific on frigate config side, can just replace the url from my driveway camera and that be it?

[NickM-27]

The resolution of the stream is 1920x1080 so you'll want to update the detect size to match that.

[balajeek]

Thanks, i set it up and started watching the live fee on frigate UI, the time was in sync , then when i triggered in front of the camera it turned into the same 1+ min delay and now the live feed shows with delay.

I need to figure out how i am going to troubleshot. I have only 3 cameras, one backyard which is eufy floodlight pro 360 which does not do 24/7 rtsp, so i can judge that, 2nd one near the front door which is wired, 3rd one this new eufy c24. So i can guess that wifi camera have delays.

wonder if i can restart my ubiquity gateway and access point to see if it make any difference, other than that i am not sure how would I start my troubleshoot.

[NickM-27]

Not totally sure, I use both of mine with 1 second delay on the detect stream

[balajeek]

in the process of reworking the network and hopefully it will smooth out, while i was working on it, I notice this

Does frigate take that much cpu (its always near 45+%) even though i have google coral plugged in and the log shows "edge tpu found" and my config "cpu" is not set.

Its running on QNAP Nas with Intel Celeron @1.50GHz /16GB Ram

[NickM-27]

@balajeek recommend reading #3860

[balajeek]

Good point, thanks for the doc, and thanks for the super quick response as always.
Going to check my config and camera settings..

[balajeek]

I checked all camera and made sure the resolution match with whats being streamed. now a days its taking 60% memory of my 16gb on qnap, so what qnap does is shuts down my containers saying " nas run out of memory and clearing some applications"

this is as of today (cpu and memory)

Is there any quick way to just disable cameras recording and see how it consumes, then slowly turn one by one?

[balajeek]

Finally i got some time today to replace the wyze cam to eufy C24 hoping my problem will be solved.
I still get the min delay on the frigate UI and notifications.

I also get this error repeating every few min:

[2022-12-17 16:24:32] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-12-17 16:24:32] ffmpeg.driveway.detect         ERROR   : [rtsp @ 0x557b9cfbb0c0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
[2022-12-17 16:24:32] ffmpeg.driveway.detect         ERROR   : Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[2022-12-17 16:24:32] ffmpeg.driveway.detect         ERROR   : Output file #0 does not contain any stream
[2022-12-17 16:24:37] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
[2022-12-17 16:24:37] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
[2022-12-17 16:24:42] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.

[NickM-27]

That's odd, what is the wifi signal for the camera like? (-dBm) I've had nothing but stable connection from it although I have a solid network

[NickM-27]

also what is your current config with it?

[balajeek]

its the same i posted above, all i changed is the camera rtsp url

[balajeek]

That's odd, what is the wifi signal for the camera like? (-dBm) I've had nothing but stable connection from it although I have a solid network

Do you think its a wifi issue?
I checked on my unifi for the wifi strength, it shows "Excellent (100%)" and hover in the range between -69dBm and -72dBm

But i notice something called "AP/Client Signal Balance = Poor or Good"

[NickM-27]

It is one potential cause. What CPU do you have? and if you disable hardware acceleration for that camera

cameras:
  driveway:
    ffmpeg:
      hwaccel_args: []

does it speed up at all?

[balajeek]

I updated the config as you mentioned, its few mins now and i see no errors yet, and the stream time is in sync with correct time, no delays yet. I will watch for sometime.

And about my CPU:

• frigate is running on QNAP TS-453Be, Intel Celeron 1500MHz, 4 cores, 16GB Memory

[NickM-27]

You might have better luck using the i965 driver https://docs.frigate.video/configuration/hardware_acceleration#intel-based-cpus-10th-generation-via-quicksync

[balajeek]

You might have better luck using the i965 driver https://docs.frigate.video/configuration/hardware_acceleration#intel-based-cpus-10th-generation-via-quicksync

okay will give it a try. thanks.

[balajeek]

I added this in the docker-compose yaml, its running okay and i am monitoring.

But i have to say, Eufy C24 picture/video quality is not better than the Wyze Outdoor V2 i replaced, specially the night mode in color is so clear on wyze with Nighvision off. on Eufy its hard to see a persons/cars clearly though. We will see how will it works for sometime.

[NickM-27]

I used mine inside my office, with the wyze official rtsp firmware and it was awful and unstable. Using ir at night it seems good on the eufy, no idea what outside use would be like

[balajeek]

Here is the comparison at night, 1st pic is wyze, 2nd is eufy

**pic removed after couple of days **

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.