[Detector Support]: Frigate+ ONNX Config

[sebastian-tischer]

Describe the problem you are having

Hi team,

I just retrained my latest mobiledet to yolonas model and wanted to test it compared to my Google Coral.

Strangely, the CPU jumped to 100% and never went down - so I assume, I have an error in my config :(

Maybe you can help me with that please? :)

Hardware:
Intel XEON 2,80 GHz
32 GB RAM
NVIDIA T400 4GB GPU

Currently I use 6 cameras.

Thanks a lot and warm regards
Sebastian

Version

0.15.0-87e7b62

Frigate config file

model:
  path: plus://e5ddf8ea17eb00f7dd969a53858051d2

detectors:
  onnx:
    type: onnx

ffmpeg:
  hwaccel_args: preset-nvidia

docker-compose file or Docker CLI command

-

Relevant Frigate log output

2024-12-27 23:51:56.370705391  [INFO] Preparing Frigate...
2024-12-27 23:51:56.375792338  [INFO] Starting Frigate...
2024-12-27 23:51:58.191560381  [2024-12-27 23:51:58] frigate.util.config            INFO    : Checking if frigate config needs migration...
2024-12-27 23:51:58.260563720  [2024-12-27 23:51:58] frigate.util.config            INFO    : frigate config does not need migration...
2024-12-27 23:51:58.473981105  [2024-12-27 23:51:58] frigate.app                    INFO    : Starting Frigate (0.15.0-87e7b62)
2024-12-27 23:51:58.487022535  [2024-12-27 23:51:58] peewee_migrate.logs            INFO    : Starting migrations
2024-12-27 23:51:58.487281486  [2024-12-27 23:51:58] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-12-27 23:51:58.493926080  [2024-12-27 23:51:58] frigate.app                    INFO    : Recording process started: 339
2024-12-27 23:51:58.499139952  [2024-12-27 23:51:58] frigate.app                    INFO    : Review process started: 348
2024-12-27 23:51:58.501107262  [2024-12-27 23:51:58] frigate.app                    INFO    : go2rtc process pid: 99
2024-12-27 23:51:58.511519887  [2024-12-27 23:51:58] detector.onnx                  INFO    : Starting detection process: 355
2024-12-27 23:51:58.512299207  [2024-12-27 23:51:58] frigate.detectors.plugins.onnx INFO    : ONNX: loaded onnxruntime module
2024-12-27 23:51:58.513062223  [2024-12-27 23:51:58] frigate.detectors.plugins.onnx INFO    : ONNX: loading /config/model_cache/e5ddf8ea17eb00f7dd969a53858051d2
2024-12-27 23:51:58.543369994  [2024-12-27 23:51:58] frigate.app                    INFO    : Output process started: 374
2024-12-27 23:51:58.738668604  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Klingel: 392
2024-12-27 23:51:58.738930243  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Einfahrt: 393
2024-12-27 23:51:58.738932366  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Eingang: 394
2024-12-27 23:51:58.749647463  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Garten: 400
2024-12-27 23:51:58.778431737  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Haus: 417
2024-12-27 23:51:58.785538970  [2024-12-27 23:51:58] frigate.app                    INFO    : Camera processor started for Garage: 418
2024-12-27 23:51:58.830279719  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Klingel: 433
2024-12-27 23:51:58.840241237  [2024-12-27 23:51:58] frigate.detectors.plugins.onnx INFO    : ONNX: /config/model_cache/e5ddf8ea17eb00f7dd969a53858051d2 loaded
2024-12-27 23:51:58.862565312  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Einfahrt: 437
2024-12-27 23:51:58.918055660  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Eingang: 452
2024-12-27 23:51:58.934997627  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Garten: 459
2024-12-27 23:51:58.958106846  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Haus: 469
2024-12-27 23:51:58.990620193  [2024-12-27 23:51:58] frigate.app                    INFO    : Capture process started for Garage: 479
2024-12-27 23:51:59.143806603  [2024-12-27 23:51:59] frigate.api.fastapi_app        INFO    : Starting FastAPI app
2024-12-27 23:51:59.193989821  [2024-12-27 23:51:59] frigate.audio_manager          INFO    : Audio processor started (pid: 493)
2024-12-27 23:51:59.276883227  [2024-12-27 23:51:59] frigate.api.fastapi_app        INFO    : FastAPI started
2024-12-27 23:52:03.999073128  [2024-12-27 23:52:03] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:52:03.999342965  [2024-12-27 23:52:03] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:52:18.946286794  [2024-12-27 23:52:18] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:52:18.946291189  [2024-12-27 23:52:18] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:52:18.946292511  [2024-12-27 23:52:18] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x560b9958dcc0] Error opening input: Invalid data found when processing input
2024-12-27 23:52:18.946294520  [2024-12-27 23:52:18] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:52:18.946307396  [2024-12-27 23:52:18] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:52:24.092095136  [2024-12-27 23:52:24] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:52:24.092098336  [2024-12-27 23:52:24] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:52:28.976914826  [2024-12-27 23:52:28] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:52:28.976918325  [2024-12-27 23:52:28] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:52:28.976919696  [2024-12-27 23:52:28] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x560748f1dcc0] Error opening input: Invalid data found when processing input
2024-12-27 23:52:28.976920804  [2024-12-27 23:52:28] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:52:28.976921919  [2024-12-27 23:52:28] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:52:34.123624538  [2024-12-27 23:52:34] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:52:34.123731249  [2024-12-27 23:52:34] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:52:39.012237044  [2024-12-27 23:52:39] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:52:39.016476691  [2024-12-27 23:52:39] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:52:39.016480068  [2024-12-27 23:52:39] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x555929a35cc0] Error opening input: Invalid data found when processing input
2024-12-27 23:52:39.016481337  [2024-12-27 23:52:39] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:52:39.016482382  [2024-12-27 23:52:39] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:52:44.164236107  [2024-12-27 23:52:44] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:52:44.168356608  [2024-12-27 23:52:44] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:52:49.045934972  [2024-12-27 23:52:49] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:52:49.049935859  [2024-12-27 23:52:49] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:52:49.049939475  [2024-12-27 23:52:49] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x5623a4faecc0] Error opening input: Invalid data found when processing input
2024-12-27 23:52:49.049940749  [2024-12-27 23:52:49] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:52:49.049941834  [2024-12-27 23:52:49] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:52:54.201473063  [2024-12-27 23:52:54] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:52:54.201607514  [2024-12-27 23:52:54] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:52:59.076816634  [2024-12-27 23:52:59] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:52:59.076820148  [2024-12-27 23:52:59] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:52:59.076821478  [2024-12-27 23:52:59] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x557587f2fcc0] Error opening input: Invalid data found when processing input
2024-12-27 23:52:59.076822530  [2024-12-27 23:52:59] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:52:59.076823598  [2024-12-27 23:52:59] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:53:03.580177361  [2024-12-27 23:53:03] frigate.record.maintainer      WARNING : Too many unprocessed recording segments in cache for Klingel. This likely indicates an issue with the detect stream, keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:04.224208044  [2024-12-27 23:53:04] frigate.video                  ERROR   : Klingel: Unable to read frames from ffmpeg process.
2024-12-27 23:53:04.224771045  [2024-12-27 23:53:04] frigate.video                  ERROR   : Klingel: ffmpeg process is not running. exiting capture thread...
2024-12-27 23:53:08.573377511  [2024-12-27 23:53:08] frigate.record.maintainer      WARNING : Too many unprocessed recording segments in cache for Klingel. This likely indicates an issue with the detect stream, keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:53:09.121461684  [2024-12-27 23:53:09] watchdog.Klingel               ERROR   : Ffmpeg process crashed unexpectedly for Klingel.
2024-12-27 23:53:09.121644639  [2024-12-27 23:53:09] watchdog.Klingel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-27 23:53:09.121886873  [2024-12-27 23:53:09] ffmpeg.Klingel.detect          ERROR   : [in#0 @ 0x55603dd9dcc0] Error opening input: Invalid data found when processing input
2024-12-27 23:53:09.121888687  [2024-12-27 23:53:09] ffmpeg.Klingel.detect          ERROR   : Error opening input file rtsp://127.0.0.1:8554/Klingel-Detect.
2024-12-27 23:53:09.121889852  [2024-12-27 23:53:09] ffmpeg.Klingel.detect          ERROR   : Error opening input files: Invalid data found when processing input
2024-12-27 23:53:13.579381327  [2024-12-27 23:53:13] frigate.record.maintainer      WARNING : Too many unprocessed recording segments in cache for Klingel. This likely indicates an issue with the detect stream, keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:18.581571029  [2024-12-27 23:53:18] frigate.record.maintainer      WARNING : Too many unprocessed recording segments in cache for Klingel. This likely indicates an issue with the detect stream, keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:53:23.586815696  [2024-12-27 23:53:23] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:53:28.611719151  [2024-12-27 23:53:28] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:33.607428296  [2024-12-27 23:53:33] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:38.611411645  [2024-12-27 23:53:38] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:43.614257496  [2024-12-27 23:53:43] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:53:48.621498761  [2024-12-27 23:53:48] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:03.636915157  [2024-12-27 23:54:03] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 11 and discarding the rest...
2024-12-27 23:54:13.641774107  [2024-12-27 23:54:13] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 9 and discarding the rest...
2024-12-27 23:54:18.646190859  [2024-12-27 23:54:18] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:23.655750182  [2024-12-27 23:54:23] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:28.657628353  [2024-12-27 23:54:28] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:54:33.666109582  [2024-12-27 23:54:33] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:38.665135301  [2024-12-27 23:54:38] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:43.674998530  [2024-12-27 23:54:43] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 7 and discarding the rest...
2024-12-27 23:54:48.673821644  [2024-12-27 23:54:48] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-12-27 23:54:53.685149185  [2024-12-27 23:54:53] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Klingel. Keeping the 6 most recent segments out of 8 and discarding the rest...

Install method

Docker Compose

Object Detector

Other

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

[NickM-27]

What is your docker compose

[NickM-27]

You likely want to tune your motion detection

[sebastian-tischer]

Then just two questions before you hopefully enjoy your night ;)

1.) why did the issue didn't happened before? Coral no issues.

2.) in my configuration, what would you tweak based on my issues?:)

[NickM-27]

1. The onnx model is larger and requires more resources to prepare the images
2. I can't say based on config file, you need to do it live with your cameras in the motion tuner in the ui

[sebastian-tischer]

Thats an amazing hint, thanks a lot Nick, never checked it out :)

I have one camera (doorbird), which only activates IR when motion detected (own algo). There is lots of noise in the camera - I needed to increase the levels - lets see if it still detects through the day with these high values.

Furthermore I adjusted all cameras, I initially had other values - 20 and 15 I guess.

Now GPU & CPU are around 30-40%.

I attach the video of the doorbird as well as new metrics - what do you think to the Doorbird seetings and metrics?

record.mov

[NickM-27]

You may need to tone that down a bit so things are detected during the day