This commit is contained in:
Endre Simo
2022-11-09 11:27:42 +02:00
5 changed files with 69 additions and 64 deletions

View File

@@ -3,24 +3,24 @@
[![CI](https://github.com/esimov/pigo/actions/workflows/ci.yml/badge.svg)](https://github.com/esimov/pigo/actions/workflows/ci.yml) [![CI](https://github.com/esimov/pigo/actions/workflows/ci.yml/badge.svg)](https://github.com/esimov/pigo/actions/workflows/ci.yml)
[![go.dev reference](https://img.shields.io/badge/pkg.go.dev-reference-007d9c?logo=go)](https://pkg.go.dev/github.com/esimov/pigo/core) [![go.dev reference](https://img.shields.io/badge/pkg.go.dev-reference-007d9c?logo=go)](https://pkg.go.dev/github.com/esimov/pigo/core)
[![license](https://img.shields.io/github/license/esimov/pigo)](./LICENSE) [![license](https://img.shields.io/github/license/esimov/pigo)](./LICENSE)
[![release](https://img.shields.io/badge/release-v1.4.5-blue.svg)](https://github.com/esimov/pigo/releases/tag/v1.4.5) [![release](https://img.shields.io/badge/release-v1.4.6-blue.svg)](https://github.com/esimov/pigo/releases/tag/v1.4.6)
[![pigo](https://snapcraft.io/pigo/badge.svg)](https://snapcraft.io/pigo) [![pigo](https://snapcraft.io/pigo/badge.svg)](https://snapcraft.io/pigo)
Pigo is a pure Go face detection, pupil/eyes localization and facial landmark points detection library based on ***Pixel Intensity Comparison-based Object detection*** paper (https://arxiv.org/pdf/1305.4537.pdf). Pigo is a pure Go face detection, pupil/eyes localization and facial landmark points detection library based on the **[Pixel Intensity Comparison-based Object detection](https://arxiv.org/pdf/1305.4537.pdf)** paper.
| Rectangle face marker | Circle face marker | Rectangle face marker | Circle face marker
|:--:|:--: |:--:|:--:
| ![rectangle](https://user-images.githubusercontent.com/883386/40916662-2fbbae1a-6809-11e8-8afd-d4ed40c7d4e9.png) | ![circle](https://user-images.githubusercontent.com/883386/40916683-447088a8-6809-11e8-942f-3112c10bede3.png) | | ![rectangle](https://user-images.githubusercontent.com/883386/40916662-2fbbae1a-6809-11e8-8afd-d4ed40c7d4e9.png) | ![circle](https://user-images.githubusercontent.com/883386/40916683-447088a8-6809-11e8-942f-3112c10bede3.png) |
### Motivation ### Motivation
The reason why Pigo has been developed is because almost all of the currently existing solutions for face detection in the Go ecosystem are purely bindings to some C/C++ libraries like `OpenCV` or `dlib`, but calling a C program trough `cgo` introduces huge latencies and implies a significant trade-off in terms of performance. Also in many cases installing OpenCV on various platforms is cumbersome. The reason why Pigo has been developed is because almost all of the currently existing solutions for face detection in the Go ecosystem are purely bindings to some C/C++ libraries like `OpenCV` or `dlib`, but calling a C program through `cgo` introduces huge latencies and implies a significant trade-off in terms of performance. Also, in many cases installing OpenCV on various platforms is cumbersome.
**The Pigo library does not require any additional modules or third party applications to be installed**, though in case you wish to run the library in a real time desktop application you might need to have Python and OpenCV installed. Head over to this [subtopic](#real-time-face-detection-running-as-a-shared-object) for more details. **The Pigo library does not require any additional modules or third party applications to be installed**, although you might need to install Python and OpenCV if you wish to run the library in a real time desktop application. Head over to this [subtopic](#real-time-face-detection-running-as-a-shared-object) for more details.
### Key features ### Key features
- [x] Does not require OpenCV or any 3rd party modules to be installed - [x] Does not require OpenCV or any 3rd party modules to be installed
- [x] High processing speed - [x] High processing speed
- [x] There is no need for image preprocessing prior detection - [x] There is no need for image preprocessing prior to detection
- [x] There is no need for the computation of integral images, image pyramid, HOG pyramid or any other similar data structure - [x] There is no need for the computation of integral images, image pyramid, HOG pyramid or any other similar data structure
- [x] The face detection is based on pixel intensity comparison encoded in the binary file tree structure - [x] The face detection is based on pixel intensity comparison encoded in the binary file tree structure
- [x] Fast detection of in-plane rotated faces - [x] Fast detection of in-plane rotated faces
@@ -53,7 +53,7 @@ Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/
![puploc](https://user-images.githubusercontent.com/883386/62784340-f5b3c100-bac6-11e9-865e-a2b4b9520b08.png) ![puploc](https://user-images.githubusercontent.com/883386/62784340-f5b3c100-bac6-11e9-865e-a2b4b9520b08.png)
### Facial landmark points detection ### Facial landmark points detection
**v1.3.0** marks a new milestone in the library evolution, Pigo being able for facial landmark points detection. The implementation is based on [Fast Localization of Facial Landmark Points](https://arxiv.org/pdf/1403.6888.pdf). **v1.3.0** marks a new milestone in the library evolution, Pigo being able to detect facial landmark points. The implementation is based on [Fast Localization of Facial Landmark Points](https://arxiv.org/pdf/1403.6888.pdf).
Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/master/examples/facial_landmark Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/master/examples/facial_landmark
@@ -63,14 +63,7 @@ Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/
Install Go, set your `GOPATH`, and make sure `$GOPATH/bin` is on your `PATH`. Install Go, set your `GOPATH`, and make sure `$GOPATH/bin` is on your `PATH`.
```bash ```bash
$ export GOPATH="$HOME/go" $ go install github.com/esimov/pigo/cmd/pigo@latest
$ export PATH="$PATH:$GOPATH/bin"
```
Next download the project and build the binary file.
```bash
$ go get -u -f github.com/esimov/pigo/cmd/pigo
$ go install
``` ```
### Binary releases ### Binary releases
@@ -83,8 +76,8 @@ The library can be accessed as a snapcraft function too.
## API ## API
Below is a minimal example of using the face detection API. Below is a minimal example of using the face detection API.
First you need to load and parse the binary classifier, then convert the image to grayscale mode, First, you need to load and parse the binary classifier, then convert the image to grayscale mode,
and finally to run the cascade function which returns a slice containing the row, column, scale and the detection score. and finally run the cascade function which returns a slice containing the row, column, scale and the detection score.
```Go ```Go
cascadeFile, err := ioutil.ReadFile("/path/to/cascade/file") cascadeFile, err := ioutil.ReadFile("/path/to/cascade/file")
@@ -190,7 +183,7 @@ Go (Golang) Face detection library.
Shift detection window by percentage (default 0.1) Shift detection window by percentage (default 0.1)
``` ```
**Important notice:** In case you wish to run also the pupil/eyes localization, then you need to use the `plc` flag and provide a valid path to the pupil localization cascade file. The same applies for facial landmark points detection, only that this time the parameter accepted by the `flpc` flag is a directory pointing to the facial landmark points cascade files found under `cascades/lps`. **Important notice:** In case you also wish to run the pupil/eyes localization, then you need to use the `plc` flag and provide a valid path to the pupil localization cascade file. The same applies for facial landmark points detection, only that this time the parameter accepted by the `flpc` flag is a directory pointing to the facial landmark points cascade files found under `cascades/lps`.
### CLI command examples ### CLI command examples
You can also use the `stdin` and `stdout` pipe commands: You can also use the `stdin` and `stdout` pipe commands:
@@ -208,16 +201,16 @@ Using the `empty` string as value for the `-out` flag will skip the image genera
## Real time face detection (running as a shared object) ## Real time face detection (running as a shared object)
In case you wish to test the library real time face detection capabilities, the `examples` folder contains a few demos written in Python. If you wish to test the library's real time face detection capabilities, the `examples` folder contains a few demos written in Python.
**But why Python you might ask?** Because the Go ecosystem is (still) missing a cross platform and system independent library for accessing the webcam. **But why Python you might ask?** Because the Go ecosystem is (still) missing a cross platform and system independent library for accessing the webcam.
In the Python program we are accessing the webcam and transfer the pixel data as a byte array through `cgo` as a **shared object** to the Go program where the core face detection is happening. But as you can imagine this operation is not cost effective, resulting in lower frame rates than the library is capable of. In the Python program we access the webcam and transfer the pixel data as a byte array through `cgo` as a **shared object** to the Go program where the core face detection is happening. But as you can imagine this operation is not cost effective, resulting in lower frame rates than the library is capable of.
## WASM (Webassembly) support 🎉 ## WASM (Webassembly) support 🎉
**Important note: in order to run the Webassembly demos at least Go 1.13 is required!** **Important note: In order to run the Webassembly demos at least Go 1.13 is required!**
Starting from version **v1.4.0** the library has been ported to [**WASM**](http://webassembly.org/). This proves the library real time face detection capabilities, constantly producing **~60 FPS**. Starting from version **v1.4.0** the library has been ported to [**WASM**](http://webassembly.org/). This proves the library's real time face detection capabilities, constantly producing **~60 FPS**.
### WASM demo ### WASM demo
To run the `wasm` demo select the `wasm` folder and type `make`. To run the `wasm` demo select the `wasm` folder and type `make`.

View File

@@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
set -e set -e
VERSION="1.4.5" VERSION="1.4.6"
PROTECTED_MODE="no" PROTECTED_MODE="no"
export GO15VENDOREXPERIMENT=1 export GO15VENDOREXPERIMENT=1

View File

@@ -15,6 +15,7 @@ import (
"math" "math"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"time" "time"
"github.com/disintegration/imaging" "github.com/disintegration/imaging"
@@ -134,8 +135,8 @@ func main() {
start := time.Now() start := time.Now()
// Progress indicator // Progress indicator
ind := utils.NewProgressIndicator("Detecting faces...", time.Millisecond*100) spinner := utils.NewSpinner("Detecting faces...", time.Millisecond*100)
ind.Start() spinner.Start()
det = &faceDetector{ det = &faceDetector{
angle: *angle, angle: *angle,
@@ -160,7 +161,7 @@ func main() {
dst = os.Stdout dst = os.Stdout
} else { } else {
fileTypes := []string{".jpg", ".jpeg", ".png"} fileTypes := []string{".jpg", ".jpeg", ".png"}
ext := filepath.Ext(det.destination) ext := filepath.Ext(strings.ToLower(det.destination))
if !inSlice(ext, fileTypes) { if !inSlice(ext, fileTypes) {
log.Fatalf("Output file type not supported: %v", ext) log.Fatalf("Output file type not supported: %v", ext)
@@ -177,8 +178,8 @@ func main() {
faces, err := det.detectFaces(*source) faces, err := det.detectFaces(*source)
if err != nil { if err != nil {
ind.StopMsg = fmt.Sprintf("Detecting faces... %s failed ✗%s\n", errorColor, defaultColor) spinner.StopMsg = fmt.Sprintf("Detecting faces... %s failed ✗%s\n", errorColor, defaultColor)
ind.Stop() spinner.Stop()
log.Fatalf("Detection error: %s%v%s", errorColor, err, defaultColor) log.Fatalf("Detection error: %s%v%s", errorColor, err, defaultColor)
} }
@@ -199,24 +200,27 @@ func main() {
out = os.Stdout out = os.Stdout
} else { } else {
f, err := os.Create(*jsonf) f, err := os.Create(*jsonf)
if err != nil {
log.Printf("%s", err)
}
defer f.Close() defer f.Close()
if err != nil { if err != nil {
ind.StopMsg = fmt.Sprintf("Detecting faces... %s failed ✗%s\n", errorColor, defaultColor) spinner.StopMsg = fmt.Sprintf("Detecting faces... %s failed ✗%s\n", errorColor, defaultColor)
ind.Stop() spinner.Stop()
log.Fatalf(fmt.Sprintf("%sCould not create the json file: %v%s", errorColor, err, defaultColor)) log.Fatalf(fmt.Sprintf("%sCould not create the json file: %v%s", errorColor, err, defaultColor))
} }
out = f out = f
} }
} }
ind.StopMsg = fmt.Sprintf("Detecting faces... %s✔%s", successColor, defaultColor) spinner.StopMsg = fmt.Sprintf("Detecting faces... %s✔%s", successColor, defaultColor)
ind.Stop() spinner.Stop()
if len(dets) > 0 { if len(dets) > 0 {
log.Printf(fmt.Sprintf("\n%s%d%s face(s) detected", successColor, len(dets), defaultColor)) log.Printf("\n%s%d%s face(s) detected", successColor, len(dets), defaultColor)
if *jsonf != "" && out == os.Stdout { if *jsonf != "" && out == os.Stdout {
log.Printf(fmt.Sprintf("\n%sThe detection coordinates of the found faces:%s", successColor, defaultColor)) log.Printf("\n%sThe detection coordinates of the found faces:%s", successColor, defaultColor)
} }
if out != nil { if out != nil {
@@ -225,10 +229,10 @@ func main() {
} }
} }
} else { } else {
log.Printf(fmt.Sprintf("\n%sno detected faces!%s", errorColor, defaultColor)) log.Printf("\n%sno detected faces!%s", errorColor, defaultColor)
} }
log.Printf(fmt.Sprintf("\nExecution time: %s%.2fs%s\n", successColor, time.Since(start).Seconds(), defaultColor)) log.Printf("\nExecution time: %s%.2fs%s\n", successColor, time.Since(start).Seconds(), defaultColor)
} }
// detectFaces run the detection algorithm over the provided source image. // detectFaces run the detection algorithm over the provided source image.
@@ -238,6 +242,9 @@ func (fd *faceDetector) detectFaces(source string) ([]pigo.Detection, error) {
// Check if source path is a local image or URL. // Check if source path is a local image or URL.
if utils.IsValidUrl(source) { if utils.IsValidUrl(source) {
src, err := utils.DownloadImage(source) src, err := utils.DownloadImage(source)
if err != nil {
return nil, err
}
// Close and remove the generated temporary file. // Close and remove the generated temporary file.
defer src.Close() defer src.Close()
defer os.Remove(src.Name()) defer os.Remove(src.Name())
@@ -289,17 +296,17 @@ func (fd *faceDetector) detectFaces(source string) ([]pigo.Detection, error) {
ImageParams: *imgParams, ImageParams: *imgParams,
} }
cascadeFile, err := ioutil.ReadFile(det.cascadeFile)
if err != nil {
return nil, fmt.Errorf("error reading the facefinder cascade file")
}
contentType, err := utils.DetectFileContentType(det.cascadeFile) contentType, err := utils.DetectFileContentType(det.cascadeFile)
if err != nil { if err != nil {
return nil, err return nil, err
} }
if contentType != "application/octet-stream" { if contentType != "application/octet-stream" {
return nil, errors.New("the provided cascade classifier is not valid.") return nil, fmt.Errorf("the provided cascade classifier is not valid")
}
cascadeFile, err := ioutil.ReadFile(det.cascadeFile)
if err != nil {
return nil, err
} }
p := pigo.NewPigo() p := pigo.NewPigo()
@@ -310,22 +317,34 @@ func (fd *faceDetector) detectFaces(source string) ([]pigo.Detection, error) {
return nil, err return nil, err
} }
if len(det.puploc) > 0 { plcReader := func() (*pigo.PuplocCascade, error) {
pl := pigo.NewPuplocCascade() plc := pigo.NewPuplocCascade()
cascade, err := ioutil.ReadFile(det.puploc) cascade, err := ioutil.ReadFile(det.puploc)
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("error reading the puploc cascade file")
} }
plc, err = pl.UnpackCascade(cascade) plc, err = plc.UnpackCascade(cascade)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return plc, nil
}
if len(det.flploc) > 0 { if len(det.puploc) > 0 {
flpcs, err = pl.ReadCascadeDir(det.flploc) plc, err = plcReader()
if err != nil { if err != nil {
return nil, err return nil, err
} }
}
if len(det.flploc) > 0 {
plc, err = plcReader()
if err != nil {
return nil, fmt.Errorf("the puploc cascade file is required: use the -plc flag")
}
flpcs, err = plc.ReadCascadeDir(det.flploc)
if err != nil {
return nil, fmt.Errorf("error reading the facial landmark points directory")
} }
} }
@@ -561,9 +580,9 @@ func (fd *faceDetector) encodeImage(dst io.Writer) error {
var err error var err error
img := dc.Image() img := dc.Image()
switch dst.(type) { switch dst := dst.(type) {
case *os.File: case *os.File:
ext := filepath.Ext(dst.(*os.File).Name()) ext := filepath.Ext(strings.ToLower(dst.Name()))
switch ext { switch ext {
case "", ".jpg", ".jpeg": case "", ".jpg", ".jpeg":
err = jpeg.Encode(dst, img, &jpeg.Options{Quality: 100}) err = jpeg.Encode(dst, img, &jpeg.Options{Quality: 100})

View File

@@ -261,7 +261,9 @@ func (pg *Pigo) RunCascade(cp CascadeParams, angle float64) []Detection {
// We need to make this comparison to filter out multiple face detection regions. // We need to make this comparison to filter out multiple face detection regions.
func (pg *Pigo) ClusterDetections(detections []Detection, iouThreshold float64) []Detection { func (pg *Pigo) ClusterDetections(detections []Detection, iouThreshold float64) []Detection {
// Sort detections by their score // Sort detections by their score
sort.Sort(det(detections)) sort.Slice(detections, func(i, j int) bool {
return detections[i].Q < detections[j].Q
})
calcIoU := func(det1, det2 Detection) float64 { calcIoU := func(det1, det2 Detection) float64 {
// Unpack the position and size of each detection. // Unpack the position and size of each detection.
@@ -304,12 +306,3 @@ func (pg *Pigo) ClusterDetections(detections []Detection, iouThreshold float64)
} }
return clusters return clusters
} }
// Implement sorting function on detection values.
type det []Detection
func (q det) Len() int { return len(q) }
func (q det) Swap(i, j int) { q[i], q[j] = q[j], q[i] }
func (q det) Less(i, j int) bool {
return q[i].Q < q[j].Q
}

View File

@@ -29,8 +29,8 @@ const (
defaultColor = "\x1b[0m" defaultColor = "\x1b[0m"
) )
// NewProgressIndicator instantiates a new progress indicator. // NewSpinner instantiates a new progress indicator.
func NewProgressIndicator(msg string, d time.Duration) *ProgressIndicator { func NewSpinner(msg string, d time.Duration) *ProgressIndicator {
return &ProgressIndicator{ return &ProgressIndicator{
mu: &sync.RWMutex{}, mu: &sync.RWMutex{},
delay: d, delay: d,