[Detector Support]: RKNN YOLO-NAS doesn't detect objects

[ShyneTurtle]

Describe the problem you are having

I'm currently setting up Frigate on a RK3588 based NAS. My issue is that when using the YOLO-NAS-M pretrained model downloaded at runtime by frigate, no detections occur. I tried using the debug view and cropped frames are getting sent to the RK-NPU but no object is ever detected, and no event is reported either.

Version

ghcr.io/blakeblackshear/frigate:0.14.0-beta4-rk

Frigate config file

detect:
  enabled: true
  fps: 10

detectors: # required
  rknn: # required
    type: rknn # required
    # number of NPU cores to use
    # 0 means choose automatically
    # increase for better performance if you have a multicore NPU e.g. set to 3 on rk3588
    num_cores: 3

model: # required
  # name of model (will be automatically downloaded) or path to your own .rknn model file
  # possible values are:
  # - deci-fp16-yolonas_s
  # - deci-fp16-yolonas_m
  # - deci-fp16-yolonas_l
  # - /config/model_cache/your_custom_model.rknn
  path: deci-fp16-yolonas_m
  # width and height of detection frames
  width: 720
  height: 720
  # pixel format of detection frame
  # default value is rgb but yolo models usually use bgr format
  input_pixel_format: bgr # required
  # shape of detection frame
  input_tensor: nhwc
  # needs to be adjusted to model, see below
  labelmap_path: /labelmap/coco-80.txt # required

logger:
  default: debug
  logs:
    frigate.mqtt: error

mqtt:
  host: localhost
  user: *
  password: *

record:
  enabled: true
  retain:
    days: 7
    mode: all
  events:
    retain:
      default: 14

cameras:
  cam240:
    ffmpeg:
      inputs:
        - path: rtsp://*:*@192.168.1.240/av_stream/ch0
          input_args: preset-rtsp-generic
          hwaccel_args: preset-rk-h264
          roles:
            - detect
  cam241:
    ffmpeg:
      inputs:
        - path: rtsp://*:*@192.168.1.241/av_stream/ch0
          input_args: preset-rtsp-generic
          hwaccel_args: preset-rk-h264
          roles:
            - detect
  cam242:
    ffmpeg:
      inputs:
        - path: rtsp://*:*@192.168.1.242/av_stream/ch0
          input_args: preset-rtsp-generic
          hwaccel_args: preset-rk-h264
          roles:
            - detect
version: 0.14

docker-compose file or Docker CLI command

services:
  frigate:
    image: ghcr.io/blakeblackshear/frigate:0.14.0-beta4-rk
    #image: marca711/frigate:latest-rk
    container_name: frigate
    privileged: true
    restart: unless-stopped
    shm_size: "256mb"
    devices: # Since we run in privileged it's not required but fuck it
      - /dev/mpp_service:/dev/mpp_service
      - /dev/dri:/dev/dri
      - /dev/v4l:/dev/v4l
      - /dev/rga:/dev/rga
      - /dev/dma_heap:/dev/dma_heap
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/dri/renderD129:/dev/dri/renderD129
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - ./media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000

Relevant log output

-

Operating system

Debian

Install method

Docker Compose

Object Detector

RKNN

Any other information that may be helpful

No response

[NickM-27]

You have the wrong model size in the configuration it should be width and height of 320

[ShyneTurtle]

I had tried it before and it didn't work so i increased the resolution thinking that it would help but it didn't.

Reverting to 320*320 did fix it thank you !

[TadyTheFish]

May I ask why is that? (noob here)

[NickM-27]

Models are trained on a specific size and expect to receive an image of that size

[TadyTheFish]

aa ok so deci-fp16-yolonas_s is trained to 320x320. Got it thank you