Viseron 3.0.0b12 Deepstack face recognition Error

[pratbhoir]

Hi, I am using Viseron 3.0.0b12 And continuous 24/7 is working great as it uses fewer resources. I am using Deepstack for object detection and face recognition. The Object Detection is working great, as I am using person detection to record and save mp4.
But for face recognition part is giving the below error.

Because of that, The faces are not getting tracked
I have a few additional questions,

1. How can I save the recordings in different tiers like 4 days in one folder path and older than that till 10 days in another folder path?
2. Are there ways In Timeline or any plans to add a feature, where we can export the mp4 file from the selected Start Time to the selected End Time?
3. Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?

Viseron Config

## Start by adding some cameras
ffmpeg:
  camera:
    tapocam01:  # This value has to be unique across all cameras
      name: Living Room
      host: XXXXXXXXXXX
      port: 554
      path: /stream1
      username: XXXXXXXX
      password: XXXXXXXXX
      recorder:
        idle_timeout: 60
        lookback: 15
        max_recording_time: 600
        create_event_clip: true
        storage:
          tiers:
            - path: /data
              events:
                max_age:
                  days: 14
              continuous:
                max_age:
                  days: 4

## Then add an object detector
deepstack:
  host: XXXXXXXXXXX
  port: 5001
  object_detector:
    cameras:
      tapocam01:
        scan_on_motion_only: false
        fps: 1
        labels:
          - label: person
            confidence: 0.4
            trigger_recorder: true
  face_recognition:
    save_unknown_faces: true
    min_confidence: 0.5
    cameras:
      tapocam01:
        labels:
          - person
    labels:
      - person

mog2:
  motion_detector:
    cameras:
      tapocam01:  # Attach detector to the configured camera_2 above
        fps: 1
        recorder_keepalive: true
        max_recorder_keepalive: 60 # keeps recording liver for seconds after motion detection is over
        area: 0.08 # we can go uptop 0.001 for triggerring small changes
        detect_shadows: false
        # history: 500
        threshold: 5
        width: 1280
        height: 720
        trigger_recorder: true

nvr:
  tapocam01:  # Run NVR for Living Room camera

Error Log

viseron3  | Traceback (most recent call last):
viseron3  |   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3  |     self.run()
viseron3  |   File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3  |     self._target(*self._args, **self._kwargs)
viseron3  |   File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3  |     self.process(
viseron3  |   File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3  |     self.face_recognition(
viseron3  |   File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3  |     detection["box"]["x_min"] + x1,
viseron3  | KeyError: 'box'
viseron3  | �[31m[2024-12-16 11:42:04] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting�[0m
viseron3  | �[31m[2024-12-16 11:42:04] [ERROR   ] [root] - Uncaught thread exception in thread viseron.domains.post_processor�[0m
viseron3  | Traceback (most recent call last):
viseron3  |   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3  |     self.run()
viseron3  |   File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3  |     self._target(*self._args, **self._kwargs)
viseron3  |   File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3  |     self.process(
viseron3  |   File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3  |     self.face_recognition(
viseron3  |   File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3  |     detection["box"]["x_min"] + x1,
viseron3  | KeyError: 'box'
viseron3  | �[31m[2024-12-16 11:42:19] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting�[0m
viseron3  | �[31m[2024-12-16 11:42:19] [ERROR   ] [root] - Uncaught thread exception in thread viseron.domains.post_processor�[0m
viseron3  | Traceback (most recent call last):
viseron3  |   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3  |     self.run()
viseron3  |   File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3  |     self._target(*self._args, **self._kwargs)
viseron3  |   File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3  |     self.process(
viseron3  |   File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3  |     self.face_recognition(
viseron3  |   File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3  |     detection["box"]["x_min"] + x1,
viseron3  | KeyError: 'box'
viseron3  | �[31m[2024-12-16 11:42:34] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting�[0m
viseron3  | �[31m[2024-12-16 11:42:34] [ERROR   ] [root] - Uncaught thread exception in thread viseron.domains.post_processor�[0m
viseron3  | Traceback (most recent call last):
viseron3  |   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3  |     self.run()
viseron3  |   File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3  |     self._target(*self._args, **self._kwargs)
viseron3  |   File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3  |     self.process(
viseron3  |   File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3  |     self.face_recognition(
viseron3  |   File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3  |     detection["box"]["x_min"] + x1,
viseron3  | KeyError: 'box'

[pratbhoir]

I found the issue, The response coming from DeepStack is without box property.

Response

{
    "success": true,
    "predictions": [
        {
            "confidence": 0.8262921,
            "userid": "pratik",
            "y_min": 481,
            "x_min": 378,
            "y_max": 1806,
            "x_max": 1321
        },
        {
            "confidence": 0.73627937,
            "userid": "papa",
            "y_min": 1216,
            "x_min": 1568,
            "y_max": 2246,
            "x_max": 2372
        }
    ],
    "duration": 0
}

I have done the below changes locally in /src/viseron/components/deepstack/face_recognition.py and seems to be working fine now

class FaceRecognition(AbstractFaceRecognition):
    """DeepSTack face recognition processor."""

    def __init__(self, vis: Viseron, config, camera_identifier) -> None:
        super().__init__(
            vis, COMPONENT, config[CONFIG_FACE_RECOGNITION], camera_identifier
        )

        self._ds_config = config
        self._ds = DeepstackFace(
            ip=config[CONFIG_HOST],
            port=config[CONFIG_PORT],
            api_key=config[CONFIG_API_KEY],
            timeout=config[CONFIG_TIMEOUT],
            min_confidence=config[CONFIG_FACE_RECOGNITION][CONFIG_MIN_CONFIDENCE],
        )

    def face_recognition(
        self, shared_frame: SharedFrame, detected_object: DetectedObject
    ) -> None:
        """Perform face recognition."""
        frame = self._camera.shared_frames.get_decoded_frame_rgb(shared_frame)
        x1, y1, x2, y2 = calculate_absolute_coords(
            (
                detected_object.rel_x1,
                detected_object.rel_y1,
                detected_object.rel_x2,
                detected_object.rel_y2,
            ),
            self._camera.resolution,
        )
        cropped_frame = frame[y1:y2, x1:x2].copy()

        try:
            detections = self._ds.recognize(
                cv2.imencode(".jpg", cropped_frame)[1].tobytes()
            )
        except ds.DeepstackException as error:
            self._logger.error("Error calling deepstack: %s", error)

        for detection in detections:
            if detection["userid"] != "unknown":
                self._logger.debug(f"Face found: {detection}")
                self.known_face_found(
                    detection["userid"],
                    (
                        detection["x_min"] + x1,
                        detection["y_min"] + y1,
                        detection["x_max"] + x2,
                        detection["y_max"] + y2,
                    ),
                    shared_frame,
                    confidence=detection["confidence"],
                )
            else:
                self.unknown_face_found(
                    (
                        detection["x_min"] + x1,
                        detection["y_min"] + y1,
                        detection["x_max"] + x2,
                        detection["y_max"] + y2,
                    ),
                    shared_frame,
                    confidence=detection["confidence"],
                )

[roflcoopter]

Thanks for finding the issue, will fix it in dev soon.

Strange tho, it has definitely worked before and DeepStack has not been updated in a long time

[roflcoopter]

Sorry totally missed the questions.

• How can I save the recordings in different tiers like 4 days in one folder path and older than that till 10 days in another folder path?

You configure it like this:

storage:
  recorder:
    tiers:
      - path: /data
        events:
          max_age:
            days: 4
      - path: /data2
        events:
          max_age:
            days: 14 # Add 10 to the first tiers `max_age`

• Are there ways In Timeline or any plans to add a feature, where we can export the mp4 file from the selected Start Time to the selected End Time?

Yes i am working on that as we speak!

• Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?

What do you mean here exactly? The URL looks fine to me

[pratbhoir]

Hi @roflcoopter , thanks for your reply! I absolutely love what you’ve created and would be thrilled to see it become even more amazing.

Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?

The problem with this is, when I am accessing the mp4 files, I’m encountering a 404 error and noticed some strange behaviour while trying to find a workaround:

1. When I placed a .mp4 file in /data/thumbnails/tapocam01/test.mp4, it wasn't accessible. However, when I changed the file extension to .jpg, it became accessible via the URL. Is there a filter in place that only allows access to .jpg files and not .mp4 files?
2. In the Recordings folder, neither .mp4 nor .jpg files were accessible. It seems like routing might only be configured for the Thumbnails folder, not the Recordings folder.

I know the above might seem trivial, but I just want to send a notification in Home Assistant with the thumbnail and video URL, which, when clicked, will open the thumbnail and video.

[roflcoopter]

Glad you like it! v3 is a huge bump in functionality for Viseron, hence why it has taken me so long.

Since you have create_event_clip: true the file should end up in /data/recordings/tapocam01/06:51:52.mp4

Could you enable debug logging and send me the logs from when the recording ends and the video is created?

logger:
  default_level: debug