zishuo/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2025-09-26 19:41:29 +08:00
Code Issues Packages Projects Releases Wiki Activity

Refactor face recognition (#17368)

Browse Source 
* Refactor face recognition to allow for running lbph or embedding

* Cleanup

* Use weighted average for faces

* Set correct url

* Cleanup

* Update docs

* Update docs

* Use scipy trimmed mean

* Normalize

* Handle color and gray landmark detection

* Upgrade to new arcface model

* Implement sigmoid function

* Rename

* Rename to arcface

* Fix

* Add face recognition model size to ui config

* Update toast
This commit is contained in:
Nicolas Mowen
2025-03-25 18:59:03 -06:00
committed by GitHub
parent 2c3ea5b74e
commit b18d1fb970
9 changed files with 573 additions and 188 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
docs/docs/configuration/face_recognition.md
View File

@@ -7,7 +7,7 @@ Face recognition identifies known individuals by matching detected faces with pr
## Model Requirements
Frigate has support for CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally. A lightweight face landmark detection model is also used to align faces before running them through the face recognizer.
### Face Detection
Users running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
@@ -19,9 +19,19 @@ Frigate needs to first detect a `face` before it can recognize a face.
:::
### Face Recognition
Frigate has support for two face recognition model types:
- **small**: Frigate will use CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally on the CPU.
- **large**: Frigate will run a face embedding model, this is only recommended to be run when an integrated or dedicated GPU is available.
In both cases a lightweight face landmark detection model is also used to align faces before running them through the face recognizer.
## Minimum System Requirements
Face recognition is lightweight and runs on the CPU, there are no significantly different system requirements than running Frigate itself.
Face recognition is lightweight and runs on the CPU, there are no significantly different system requirements than running Frigate itself when using the `small` model.
When using the `large` model an integrated or discrete GPU is recommended.
## Configuration
@@ -47,6 +57,7 @@ Fine-tune face recognition with these optional parameters:
### Recognition
- `model_size`: Which model size to use, options are `small` or `large`
- `recognition_threshold`: Recognition confidence score required to add the face to the object as a sub label.
- Default: `0.9`.
- `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this.
@@ -107,3 +118,7 @@ This can happen for a few different reasons, but this is usually an indicator th
- If you train with only a few images per person, especially if those images are very similar, the recognition model becomes overly specialized to those specific images.
- When you provide images with different poses, lighting, and expressions, the algorithm extracts features that are consistent across those variations.
- By training on a diverse set of images, the algorithm becomes less sensitive to minor variations and noise in the input image.
### I see scores above the threshold in the train tab, but a sub label wasn't assigned?
The Frigate face recognizer collects face recognition scores from all of the frames across the person objects lifecycle. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if there is a prominent person recognized. This avoids cases where a single high confidence recognition result would throw off the results.