apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-05 16:48:03 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Model] Support YOLOv8 (#1137)

Browse Source 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
This commit is contained in:
WJJ1995
2023-01-16 11:24:23 +08:00
committed by GitHub
parent a4b94b2c93
commit 02bd22422e
28 changed files with 1448 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/vision/detection/scaledyolov4/README.md Normal file → Executable file
View File

@@ -11,7 +11,7 @@ English | [简体中文](README_CN.md)
Visit the official [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4) github repository, follow the guidelines to download the `scaledyolov4.pt` model, and employ `models/export.py` to get the file in `onnx` format. If you have any problems with the exported `onnx` model, refer to [ScaledYOLOv4#401](https://github.com/WongKinYiu/ScaledYOLOv4/issues/401) for solution. Visit the official [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4) github repository, follow the guidelines to download the `scaledyolov4.pt` model, and employ `models/export.py` to get the file in `onnx` format. If you have any problems with the exported `onnx` model, refer to [ScaledYOLOv4#401](https://github.com/WongKinYiu/ScaledYOLOv4/issues/401) for solution.
```bash ```bash
# Download the ScaledYOLOv4 model file # Download the ScaledYOLOv4 model file
@@ -38,8 +38,6 @@ For developers' testing, models exported by ScaledYOLOv4 are provided below. Dev
| [ScaledYOLOv4-P6+BoF](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p6_.onnx) | 487MB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4)，GPL-3.0 License | | [ScaledYOLOv4-P6+BoF](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p6_.onnx) | 487MB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4)，GPL-3.0 License |
| [ScaledYOLOv4-P7](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p7.onnx) | 1.1GB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4)，GPL-3.0 License | | [ScaledYOLOv4-P7](https://bj.bcebos.com/paddlehub/fastdeploy/scaled_yolov4-p7.onnx) | 1.1GB | - | This model file is sourced from [ScaledYOLOv4](https://github.com/WongKinYiu/ScaledYOLOv4)，GPL-3.0 License |
## Detailed Deployment Documents ## Detailed Deployment Documents
- [Python Deployment](python) - [Python Deployment](python)
@@ -48,4 +46,4 @@ For developers' testing, models exported by ScaledYOLOv4 are provided below. Dev
## Release Note ## Release Note
- Document and code are based on [ScaledYOLOv4 CommitID: 6768003](https://github.com/WongKinYiu/ScaledYOLOv4/commit/676800364a3446900b9e8407bc880ea2127b3415) - Document and code are based on [ScaledYOLOv4 CommitID: 6768003](https://github.com/WongKinYiu/ScaledYOLOv4/commit/676800364a3446900b9e8407bc880ea2127b3415)

1
examples/vision/detection/yolor/README.md Normal file → Executable file
View File

@@ -36,7 +36,6 @@ For developers' testing, models exported by YOLOR are provided below. Developers
| [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-570-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor)，GPL-3.0 License | | [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-570-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor)，GPL-3.0 License |
| [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-573-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor)，GPL-3.0 License | | [YOLOR-D6](https://bj.bcebos.com/paddlehub/fastdeploy/yolor-d6-paper-573-640-640.onnx) | 580MB | - | This model file is sourced from [YOLOR](https://github.com/WongKinYiu/yolor)，GPL-3.0 License |
## Detailed Deployment Documents ## Detailed Deployment Documents
- [Python Deployment](python) - [Python Deployment](python)

3
examples/vision/detection/yolov5/README.md Normal file → Executable file
View File

@@ -6,7 +6,6 @@ English | [简体中文](README_CN.md)
- （1）The *.onnx provided by [Official Repository](https://github.com/ultralytics/yolov5/releases/tag/v7.0) can be deployed directly； - （1）The *.onnx provided by [Official Repository](https://github.com/ultralytics/yolov5/releases/tag/v7.0) can be deployed directly；
- （2）The YOLOv5 v7.0 model trained by personal data should employ `export.py` in [YOLOv5](https://github.com/ultralytics/yolov5) to export the ONNX files for deployment. - （2）The YOLOv5 v7.0 model trained by personal data should employ `export.py` in [YOLOv5](https://github.com/ultralytics/yolov5) to export the ONNX files for deployment.
## Download Pre-trained ONNX Model ## Download Pre-trained ONNX Model
For developers' testing, models exported by YOLOv5 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository) For developers' testing, models exported by YOLOv5 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository)
@@ -27,4 +26,4 @@ For developers' testing, models exported by YOLOv5 are provided below. Developer
## Release Note ## Release Note
- Document and code are based on [YOLOv5 v7.0](https://github.com/ultralytics/yolov5/tree/v7.0) - Document and code are based on [YOLOv5 v7.0](https://github.com/ultralytics/yolov5/tree/v7.0)

3
examples/vision/detection/yolov5lite/README.md Normal file → Executable file
View File

@@ -60,7 +60,6 @@ For developers' testing, models exported by YOLOv5Lite are provided below. Devel
| [YOLOv5Lite-c](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-c-sim-512.onnx) | 18MB | 50.9% | This model file is sourced from[YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite)，GPL-3.0 License | | [YOLOv5Lite-c](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-c-sim-512.onnx) | 18MB | 50.9% | This model file is sourced from[YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite)，GPL-3.0 License |
| [YOLOv5Lite-g](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-g-sim-640.onnx) | 21MB | 57.6% | This model file is sourced from [YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite)，GPL-3.0 License | | [YOLOv5Lite-g](https://bj.bcebos.com/paddlehub/fastdeploy/v5Lite-g-sim-640.onnx) | 21MB | 57.6% | This model file is sourced from [YOLOv5-Lite](https://github.com/ppogg/YOLOv5-Lite)，GPL-3.0 License |
## Detailed Deployment Documents ## Detailed Deployment Documents
- [Python Deployment](python) - [Python Deployment](python)
@@ -69,4 +68,4 @@ For developers' testing, models exported by YOLOv5Lite are provided below. Devel
## Release Note ## Release Note
- Document and code are based on [YOLOv5-Lite v1.4](https://github.com/ppogg/YOLOv5-Lite/releases/tag/v1.4) - Document and code are based on [YOLOv5-Lite v1.4](https://github.com/ppogg/YOLOv5-Lite/releases/tag/v1.4)

6
examples/vision/detection/yolov6/README.md Normal file → Executable file
View File

@@ -8,8 +8,6 @@ English | [简体中文](README_CN.md)
- （1）The *.onnx provided by [Official Repository](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) can directly conduct deployemnt； - （1）The *.onnx provided by [Official Repository](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) can directly conduct deployemnt；
- （2）Personal models trained by developers should export the ONNX model. Refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. - （2）Personal models trained by developers should export the ONNX model. Refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment.
## Download Pre-trained ONNX Model ## Download Pre-trained ONNX Model
For developers' testing, models exported by YOLOv6 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository) For developers' testing, models exported by YOLOv6 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository)
@@ -20,8 +18,6 @@ For developers' testing, models exported by YOLOv6 are provided below. Developer
| [YOLOv6t](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6t.onnx) | 58MB | 41.3% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6)，GPL-3.0 License | | [YOLOv6t](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6t.onnx) | 58MB | 41.3% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6)，GPL-3.0 License |
| [YOLOv6n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6n.onnx) | 17MB | 35.0% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6)，GPL-3.0 License | | [YOLOv6n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov6n.onnx) | 17MB | 35.0% | This model file is sourced from [YOLOv6](https://github.com/meituan/YOLOv6)，GPL-3.0 License |
## Detailed Deployment Documents ## Detailed Deployment Documents
- [Python Deployment](python) - [Python Deployment](python)
@@ -30,4 +26,4 @@ For developers' testing, models exported by YOLOv6 are provided below. Developer
## Release Note ## Release Note
- Document and code are based on [YOLOv6 0.1.0 version](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) - Document and code are based on [YOLOv6 0.1.0 version](https://github.com/meituan/YOLOv6/releases/tag/0.1.0)

2
examples/vision/detection/yolov7/README.md Normal file → Executable file
View File

@@ -20,8 +20,6 @@ python models/export.py --grid --dynamic --weights PATH/TO/yolov7.pt
python models/export.py --grid --dynamic --end2end --weights PATH/TO/yolov7.pt python models/export.py --grid --dynamic --end2end --weights PATH/TO/yolov7.pt
``` ```
## Download the pre-trained ONNX model
To facilitate testing for developers, we provide below the models exported by YOLOv7, which developers can download and use directly. (The accuracy of the models in the table is sourced from the official library) To facilitate testing for developers, we provide below the models exported by YOLOv7, which developers can download and use directly. (The accuracy of the models in the table is sourced from the official library)
| Model | Size | Accuracy | Note | | Model | Size | Accuracy | Note |

4
examples/vision/detection/yolov7end2end_ort/README.md Normal file → Executable file
View File

@@ -31,13 +31,11 @@ For developers' testing, models exported by YOLOv7End2EndORT are provided below.
| [yolov7-d6-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-d6-end2end-ort-nms.onnx) | 511MB | 56.6% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7)，GPL-3.0 License | | [yolov7-d6-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-d6-end2end-ort-nms.onnx) | 511MB | 56.6% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7)，GPL-3.0 License |
| [yolov7-e6e-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-e6e-end2end-ort-nms.onnx) | 579MB | 56.8% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7)，GPL-3.0 License | | [yolov7-e6e-end2end-ort-nms](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7-e6e-end2end-ort-nms.onnx) | 579MB | 56.8% | This model file is sourced from [YOLOv7](https://github.com/WongKinYiu/yolov7)，GPL-3.0 License |
## Detailed Deployment Documents ## Detailed Deployment Documents
- [Python Deployment](python) - [Python Deployment](python)
- [C++ Deployment](cpp) - [C++ Deployment](cpp)
## Release Note ## Release Note
- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) - Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1)

5
examples/vision/detection/yolov7end2end_trt/README.md Normal file → Executable file
View File

@@ -6,8 +6,6 @@ The YOLOv7End2EndTRT deployment is based on [YOLOv7](https://github.com/WongKinY
- （1）*.pt provided by [Official Repository](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1) should [Export the ONNX Model](#Export-the-ONNX-Model) to complete the deployment. The deployment of *.trt and *.pose models is not supported. - （1）*.pt provided by [Official Repository](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1) should [Export the ONNX Model](#Export-the-ONNX-Model) to complete the deployment. The deployment of *.trt and *.pose models is not supported.
- （2）The YOLOv7 model trained by personal data should [Export the ONNX Model](#%E5%AF%BC%E5%87%BAONNX%E6%A8%A1%E5%9E%8B). Please refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. - （2）The YOLOv7 model trained by personal data should [Export the ONNX Model](#%E5%AF%BC%E5%87%BAONNX%E6%A8%A1%E5%9E%8B). Please refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment.
## Export the ONNX Model ## Export the ONNX Model
```bash ```bash
@@ -37,7 +35,6 @@ For developers' testing, models exported by YOLOv7End2EndTRT are provided below.
- [Python Deployment](python) - [Python Deployment](python)
- [C++ Deployement](cpp) - [C++ Deployement](cpp)
## Release Note ## Release Note
- Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1) - Document and code are based on [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/tree/v0.1)

29
examples/vision/detection/yolov8/README.md Executable file
View File

@@ -0,0 +1,29 @@
English | [简体中文](README_CN.md)
# YOLOv8 Ready-to-deploy Model
- The deployment of the YOLOv8 model is based on [YOLOv8](https://github.com/ultralytics/ultralytics) and [Pre-trained Model Based on COCO](https://github.com/ultralytics/ultralytics)
- （1）The *.onnx provided by [Official Repository](https://github.com/ultralytics/ultralytics) can be deployed directly；
- （2）The YOLOv8 model trained by personal data should employ `export.py` in [YOLOv8](https://github.com/ultralytics/ultralytics) to export the ONNX files for deployment.
## Download Pre-trained ONNX Model
For developers' testing, models exported by YOLOv8 are provided below. Developers can download them directly. (The accuracy in the following table is derived from the source official repository)
| Model | Size | Accuracy | Note |
|:---------------------------------------------------------------- |:----- |:----- |:---- |
| [YOLOv8n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8n.onnx) | 12.1MB | 37.3% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8s](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx) | 42.6MB | 44.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8m](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8m.onnx) | 98.8MB | 50.2% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8l](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8l.onnx) | 166.7MB | 52.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8x](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8x.onnx) | 260.3MB | 53.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
## Detailed Deployment Documents
- [Python Deployment](python)
- [C++ Deployment](cpp)
- [Serving Deployment](serving)
## Release Note
- Document and code are based on [YOLOv8](https://github.com/ultralytics/ultralytics)

29
examples/vision/detection/yolov8/README_CN.md Normal file
View File

@@ -0,0 +1,29 @@
[English](README.md) | 简体中文
# YOLOv8准备部署模型
- YOLOv8部署模型实现来自[YOLOv8](https://github.com/ultralytics/ultralytics),和[基于COCO的预训练模型](https://github.com/ultralytics/ultralytics)
- （1）[官方库](https://github.com/ultralytics/ultralytics)提供的*.onnx可直接进行部署；
- （2）开发者基于自己数据训练的YOLOv8模型，可使用[YOLOv8](https://github.com/ultralytics/ultralytics)中的`export.py`导出ONNX文件后，完成部署。
## 下载预训练ONNX模型
为了方便开发者的测试，下面提供了YOLOv8导出的各系列模型，开发者可直接下载使用。（下表中模型的精度来源于源官方库）
| 模型 | 大小 | 精度 | 备注 |
|:---------------------------------------------------------------- |:----- |:----- |:---- |
| [YOLOv8n](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8n.onnx) | 12.1MB | 37.3% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8s](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx) | 42.6MB | 44.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8m](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8m.onnx) | 98.8MB | 50.2% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8l](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8l.onnx) | 166.7MB | 52.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
| [YOLOv8x](https://bj.bcebos.com/paddlehub/fastdeploy/yolov8x.onnx) | 260.3MB | 53.9% | This model file is sourced from [YOLOv8](https://github.com/ultralytics/ultralytics)，GPL-3.0 License |
## 详细部署文档
- [Python部署](python)
- [C++部署](cpp)
- [服务化部署](serving)
## 版本说明
- 本版本文档和代码基于[YOLOv8](https://github.com/ultralytics/ultralytics) 编写

14
examples/vision/detection/yolov8/cpp/CMakeLists.txt Executable file
View File

@@ -0,0 +1,14 @@
PROJECT(infer_demo C CXX)
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
# Specify the fastdeploy library path after downloading and decompression
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
# Add FastDeploy dependent header files
include_directories(${FASTDEPLOY_INCS})
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
# Add FastDeploy library dependencies
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})

90
examples/vision/detection/yolov8/cpp/README_CN.md Normal file
View File

@@ -0,0 +1,90 @@
[English](README.md) | 简体中文
# YOLOv8 C++部署示例
本目录下提供`infer.cc`快速完成YOLOv8在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
在部署前，需确认以下两个步骤
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
以Linux上CPU推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.3以上(x.x.x>=1.0.3)
```bash
mkdir build
cd build
# 下载 FastDeploy 预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
tar xvf fastdeploy-linux-x64-x.x.x.tgz
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
make -j
# 1. 下载官方转换好的 YOLOv8 ONNX 模型文件和测试图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov8s.onnx
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
# CPU推理
./infer_demo yolov8s.onnx 000000014439.jpg 0
# GPU推理
./infer_demo yolov8s.onnx 000000014439.jpg 1
# GPU上TensorRT推理
./infer_demo yolov8s.onnx 000000014439.jpg 2
```
运行完成可视化结果如下图所示
<img width="640" src="https://user-images.githubusercontent.com/67993288/184309358-d803347a-8981-44b6-b589-4608021ad0f4.jpg">
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md)
如果用户使用华为昇腾NPU部署, 请参考以下方式在部署前初始化部署环境:
- [如何使用华为昇腾NPU部署](../../../../../docs/cn/faq/use_sdk_on_ascend.md)
## YOLOv8 C++接口
### YOLOv8类
```c++
fastdeploy::vision::detection::YOLOv8(
const string& model_file,
const string& params_file = "",
const RuntimeOption& runtime_option = RuntimeOption(),
const ModelFormat& model_format = ModelFormat::ONNX)
```
YOLOv8模型加载和初始化，其中model_file为导出的ONNX模型格式。
**参数**
> * **model_file**(str): 模型文件路径
> * **params_file**(str): 参数文件路径，当模型格式为ONNX时，此参数传入空字符串即可
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
> * **model_format**(ModelFormat): 模型格式，默认为ONNX格式
#### Predict函数
> ```c++
> YOLOv8::Predict(cv::Mat* im, DetectionResult* result)
> ```
>
> 模型预测接口，输入图像直接输出检测结果。
>
> **参数**
>
> > * **im**: 输入图像，注意需为HWC，BGR格式
> > * **result**: 检测结果，包括检测框，各个框的置信度, DetectionResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
### 类成员变量
#### 预处理参数
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
> > * **size**(vector&lt;int&gt;): 通过此参数修改预处理过程中resize的大小，包含两个整型元素，表示[width, height], 默认值为[640, 640]
> > * **padding_value**(vector&lt;float&gt;): 通过此参数可以修改图片在resize时候做填充(padding)的值, 包含三个浮点型元素, 分别表示三个通道的值, 默认值为[114, 114, 114]
> > * **is_no_pad**(bool): 通过此参数让图片是否通过填充的方式进行resize, `is_no_pad=ture` 表示不使用填充的方式，默认值为`is_no_pad=false`
> > * **is_mini_pad**(bool): 通过此参数可以将resize之后图像的宽高这是为最接近`size`成员变量的值, 并且满足填充的像素大小是可以被`stride`成员变量整除的。默认值为`is_mini_pad=false`
> > * **stride**(int): 配合`stris_mini_pad`成员变量使用, 默认值为`stride=32`
- [模型介绍](../../)
- [Python部署](../python)
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)

105
examples/vision/detection/yolov8/cpp/infer.cc Executable file
View File

@@ -0,0 +1,105 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"
void CpuInfer(const std::string& model_file, const std::string& image_file) {
auto model = fastdeploy::vision::detection::YOLOv8(model_file);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}
auto im = cv::imread(image_file);
fastdeploy::vision::DetectionResult res;
if (!model.Predict(im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}
std::cout << res.Str() << std::endl;
auto vis_im = fastdeploy::vision::VisDetection(im, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}
void GpuInfer(const std::string& model_file, const std::string& image_file) {
auto option = fastdeploy::RuntimeOption();
option.UseGpu();
auto model = fastdeploy::vision::detection::YOLOv8(model_file, "", option);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}
auto im = cv::imread(image_file);
fastdeploy::vision::DetectionResult res;
if (!model.Predict(im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}
std::cout << res.Str() << std::endl;
auto vis_im = fastdeploy::vision::VisDetection(im, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}
void TrtInfer(const std::string& model_file, const std::string& image_file) {
auto option = fastdeploy::RuntimeOption();
option.UseGpu();
option.UseTrtBackend();
option.SetTrtInputShape("images", {1, 3, 640, 640});
auto model = fastdeploy::vision::detection::YOLOv8(model_file, "", option);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}
auto im = cv::imread(image_file);
fastdeploy::vision::DetectionResult res;
if (!model.Predict(im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}
std::cout << res.Str() << std::endl;
auto vis_im = fastdeploy::vision::VisDetection(im, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}
int main(int argc, char* argv[]) {
if (argc < 4) {
std::cout << "Usage: infer_demo path/to/model path/to/image run_option, "
"e.g ./infer_model ./yolov8s.onnx ./test.jpeg 0"
<< std::endl;
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
"with gpu; 2: run with gpu and use tensorrt backend."
<< std::endl;
return -1;
}
if (std::atoi(argv[3]) == 0) {
CpuInfer(argv[1], argv[2]);
} else if (std::atoi(argv[3]) == 1) {
GpuInfer(argv[1], argv[2]);
} else if (std::atoi(argv[3]) == 2) {
TrtInfer(argv[1], argv[2]);
}
return 0;
}

78
examples/vision/detection/yolov8/python/README_CN.md Normal file
View File

@@ -0,0 +1,78 @@
[English](README.md) | 简体中文
# YOLOv8 Python部署示例
在部署前，需确认以下两个步骤
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
本目录下提供`infer.py`快速完成YOLOv8在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
```bash
#下载部署示例代码
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd examples/vision/detection/yolov8/python/
#下载yolov8模型文件和测试图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov8.onnx
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
# CPU推理
python infer.py --model yolov8.onnx --image 000000014439.jpg --device cpu
# GPU推理
python infer.py --model yolov8.onnx --image 000000014439.jpg --device gpu
# GPU上使用TensorRT推理
python infer.py --model yolov8.onnx --image 000000014439.jpg --device gpu --use_trt True
```
运行完成可视化结果如下图所示
<img width="640" src="https://user-images.githubusercontent.com/67993288/184309358-d803347a-8981-44b6-b589-4608021ad0f4.jpg">
## YOLOv8 Python接口
```python
fastdeploy.vision.detection.YOLOv8(model_file, params_file=None, runtime_option=None, model_format=ModelFormat.ONNX)
```
YOLOv8模型加载和初始化，其中model_file为导出的ONNX模型格式
**参数**
> * **model_file**(str): 模型文件路径
> * **params_file**(str): 参数文件路径，当模型格式为ONNX格式时，此参数无需设定
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
> * **model_format**(ModelFormat): 模型格式，默认为ONNX
### predict函数
> ```python
> YOLOv8.predict(image_data)
> ```
>
> 模型预测结口，输入图像直接输出检测结果。
>
> **参数**
>
> > * **image_data**(np.ndarray): 输入数据，注意需为HWC，BGR格式
> **返回**
>
> > 返回`fastdeploy.vision.DetectionResult`结构体，结构体说明参考文档[视觉模型预测结果](../../../../../docs/api/vision_results/)
### 类成员属性
#### 预处理参数
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
> > * **size**(list[int]): 通过此参数修改预处理过程中resize的大小，包含两个整型元素，表示[width, height], 默认值为[640, 640]
> > * **padding_value**(list[float]): 通过此参数可以修改图片在resize时候做填充(padding)的值, 包含三个浮点型元素, 分别表示三个通道的值, 默认值为[114, 114, 114]
> > * **is_no_pad**(bool): 通过此参数让图片是否通过填充的方式进行resize, `is_no_pad=True` 表示不使用填充的方式，默认值为`is_no_pad=False`
> > * **is_mini_pad**(bool): 通过此参数可以将resize之后图像的宽高这是为最接近`size`成员变量的值, 并且满足填充的像素大小是可以被`stride`成员变量整除的。默认值为`is_mini_pad=False`
> > * **stride**(int): 配合`stris_mini_padide`成员变量使用, 默认值为`stride=32`
## 其它文档
- [YOLOv8 模型介绍](..)
- [YOLOv8 C++部署](../cpp)
- [模型预测结果说明](../../../../../docs/api/vision_results/)
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)

58
examples/vision/detection/yolov8/python/infer.py Executable file
View File

@@ -0,0 +1,58 @@
import fastdeploy as fd
import cv2
import os
def parse_arguments():
import argparse
import ast
parser = argparse.ArgumentParser()
parser.add_argument("--model", default=None, help="Path of yolov8 model.")
parser.add_argument(
"--image", default=None, help="Path of test image file.")
parser.add_argument(
"--device",
type=str,
default='cpu',
help="Type of inference device, support 'cpu' or 'gpu' or 'kunlunxin'.")
parser.add_argument(
"--use_trt",
type=ast.literal_eval,
default=False,
help="Wether to use tensorrt.")
return parser.parse_args()
def build_option(args):
option = fd.RuntimeOption()
if args.device.lower() == "gpu":
option.use_gpu()
if args.device.lower() == "ascend":
option.use_ascend()
if args.use_trt:
option.use_trt_backend()
option.set_trt_input_shape("images", [1, 3, 640, 640])
return option
args = parse_arguments()
# Configure runtime, load model
runtime_option = build_option(args)
model = fd.vision.detection.YOLOv8(args.model, runtime_option=runtime_option)
# Predicting image
if args.image is None:
image = fd.utils.get_detection_test_image()
else:
image = args.image
im = cv2.imread(image)
result = model.predict(im)
# Visualization
vis_im = fd.vision.vis_detection(im, result)
cv2.imwrite("visualized_result.jpg", vis_im)
print("Visualized result save in ./visualized_result.jpg")

1
fastdeploy/vision.h
View File

@@ -29,6 +29,7 @@
#include "fastdeploy/vision/detection/contrib/yolov7/yolov7.h" #include "fastdeploy/vision/detection/contrib/yolov7/yolov7.h"
#include "fastdeploy/vision/detection/contrib/yolov7end2end_ort.h" #include "fastdeploy/vision/detection/contrib/yolov7end2end_ort.h"
#include "fastdeploy/vision/detection/contrib/yolov7end2end_trt.h" #include "fastdeploy/vision/detection/contrib/yolov7end2end_trt.h"
#include "fastdeploy/vision/detection/contrib/yolov8/yolov8.h"
#include "fastdeploy/vision/detection/contrib/yolox.h" #include "fastdeploy/vision/detection/contrib/yolox.h"
#include "fastdeploy/vision/detection/contrib/rknpu2/model.h" #include "fastdeploy/vision/detection/contrib/rknpu2/model.h"
#include "fastdeploy/vision/detection/ppdet/model.h" #include "fastdeploy/vision/detection/ppdet/model.h"

140
fastdeploy/vision/detection/contrib/yolov5/yolov5_pybind.cc Executable file → Normal file
View File

@@ -16,54 +16,82 @@
namespace fastdeploy { namespace fastdeploy {
void BindYOLOv5(pybind11::module& m) { void BindYOLOv5(pybind11::module& m) {
pybind11::class_<vision::detection::YOLOv5Preprocessor>( pybind11::class_<vision::detection::YOLOv5Preprocessor>(m,
m, "YOLOv5Preprocessor") "YOLOv5Preprocessor")
.def(pybind11::init<>()) .def(pybind11::init<>())
.def("run", [](vision::detection::YOLOv5Preprocessor& self, std::vector<pybind11::array>& im_list) { .def(
std::vector<vision::FDMat> images; "run",
for (size_t i = 0; i < im_list.size(); ++i) { [](vision::detection::YOLOv5Preprocessor& self,
images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i]))); std::vector<pybind11::array>& im_list) {
} std::vector<vision::FDMat> images;
std::vector<FDTensor> outputs; for (size_t i = 0; i < im_list.size(); ++i) {
std::vector<std::map<std::string, std::array<float, 2>>> ims_info; images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i])));
if (!self.Run(&images, &outputs, &ims_info)) { }
throw std::runtime_error("Failed to preprocess the input data in PaddleClasPreprocessor."); std::vector<FDTensor> outputs;
} std::vector<std::map<std::string, std::array<float, 2>>> ims_info;
for (size_t i = 0; i < outputs.size(); ++i) { if (!self.Run(&images, &outputs, &ims_info)) {
outputs[i].StopSharing(); throw std::runtime_error(
} "Failed to preprocess the input data in YOLOv5Preprocessor.");
return make_pair(outputs, ims_info); }
}) for (size_t i = 0; i < outputs.size(); ++i) {
.def_property("size", &vision::detection::YOLOv5Preprocessor::GetSize, &vision::detection::YOLOv5Preprocessor::SetSize) outputs[i].StopSharing();
.def_property("padding_value", &vision::detection::YOLOv5Preprocessor::GetPaddingValue, &vision::detection::YOLOv5Preprocessor::SetPaddingValue) }
.def_property("is_scale_up", &vision::detection::YOLOv5Preprocessor::GetScaleUp, &vision::detection::YOLOv5Preprocessor::SetScaleUp) return make_pair(outputs, ims_info);
.def_property("is_mini_pad", &vision::detection::YOLOv5Preprocessor::GetMiniPad, &vision::detection::YOLOv5Preprocessor::SetMiniPad) })
.def_property("stride", &vision::detection::YOLOv5Preprocessor::GetStride, &vision::detection::YOLOv5Preprocessor::SetStride); .def_property("size", &vision::detection::YOLOv5Preprocessor::GetSize,
&vision::detection::YOLOv5Preprocessor::SetSize)
.def_property("padding_value",
&vision::detection::YOLOv5Preprocessor::GetPaddingValue,
&vision::detection::YOLOv5Preprocessor::SetPaddingValue)
.def_property("is_scale_up",
&vision::detection::YOLOv5Preprocessor::GetScaleUp,
&vision::detection::YOLOv5Preprocessor::SetScaleUp)
.def_property("is_mini_pad",
&vision::detection::YOLOv5Preprocessor::GetMiniPad,
&vision::detection::YOLOv5Preprocessor::SetMiniPad)
.def_property("stride", &vision::detection::YOLOv5Preprocessor::GetStride,
&vision::detection::YOLOv5Preprocessor::SetStride);
pybind11::class_<vision::detection::YOLOv5Postprocessor>( pybind11::class_<vision::detection::YOLOv5Postprocessor>(
m, "YOLOv5Postprocessor") m, "YOLOv5Postprocessor")
.def(pybind11::init<>()) .def(pybind11::init<>())
.def("run", [](vision::detection::YOLOv5Postprocessor& self, std::vector<FDTensor>& inputs, .def("run",
const std::vector<std::map<std::string, std::array<float, 2>>>& ims_info) { [](vision::detection::YOLOv5Postprocessor& self,
std::vector<vision::DetectionResult> results; std::vector<FDTensor>& inputs,
if (!self.Run(inputs, &results, ims_info)) { const std::vector<std::map<std::string, std::array<float, 2>>>&
throw std::runtime_error("Failed to postprocess the runtime result in YOLOv5Postprocessor."); ims_info) {
} std::vector<vision::DetectionResult> results;
return results; if (!self.Run(inputs, &results, ims_info)) {
}) throw std::runtime_error(
.def("run", [](vision::detection::YOLOv5Postprocessor& self, std::vector<pybind11::array>& input_array, "Failed to postprocess the runtime result in "
const std::vector<std::map<std::string, std::array<float, 2>>>& ims_info) { "YOLOv5Postprocessor.");
std::vector<vision::DetectionResult> results; }
std::vector<FDTensor> inputs; return results;
PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true); })
if (!self.Run(inputs, &results, ims_info)) { .def("run",
throw std::runtime_error("Failed to postprocess the runtime result in YOLOv5Postprocessor."); [](vision::detection::YOLOv5Postprocessor& self,
} std::vector<pybind11::array>& input_array,
return results; const std::vector<std::map<std::string, std::array<float, 2>>>&
}) ims_info) {
.def_property("conf_threshold", &vision::detection::YOLOv5Postprocessor::GetConfThreshold, &vision::detection::YOLOv5Postprocessor::SetConfThreshold) std::vector<vision::DetectionResult> results;
.def_property("nms_threshold", &vision::detection::YOLOv5Postprocessor::GetNMSThreshold, &vision::detection::YOLOv5Postprocessor::SetNMSThreshold) std::vector<FDTensor> inputs;
.def_property("multi_label", &vision::detection::YOLOv5Postprocessor::GetMultiLabel, &vision::detection::YOLOv5Postprocessor::SetMultiLabel); PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true);
if (!self.Run(inputs, &results, ims_info)) {
throw std::runtime_error(
"Failed to postprocess the runtime result in "
"YOLOv5Postprocessor.");
}
return results;
})
.def_property("conf_threshold",
&vision::detection::YOLOv5Postprocessor::GetConfThreshold,
&vision::detection::YOLOv5Postprocessor::SetConfThreshold)
.def_property("nms_threshold",
&vision::detection::YOLOv5Postprocessor::GetNMSThreshold,
&vision::detection::YOLOv5Postprocessor::SetNMSThreshold)
.def_property("multi_label",
&vision::detection::YOLOv5Postprocessor::GetMultiLabel,
&vision::detection::YOLOv5Postprocessor::SetMultiLabel);
pybind11::class_<vision::detection::YOLOv5, FastDeployModel>(m, "YOLOv5") pybind11::class_<vision::detection::YOLOv5, FastDeployModel>(m, "YOLOv5")
.def(pybind11::init<std::string, std::string, RuntimeOption, .def(pybind11::init<std::string, std::string, RuntimeOption,
@@ -75,16 +103,20 @@ void BindYOLOv5(pybind11::module& m) {
self.Predict(mat, &res); self.Predict(mat, &res);
return res; return res;
}) })
.def("batch_predict", [](vision::detection::YOLOv5& self, std::vector<pybind11::array>& data) { .def("batch_predict",
std::vector<cv::Mat> images; [](vision::detection::YOLOv5& self,
for (size_t i = 0; i < data.size(); ++i) { std::vector<pybind11::array>& data) {
images.push_back(PyArrayToCvMat(data[i])); std::vector<cv::Mat> images;
} for (size_t i = 0; i < data.size(); ++i) {
std::vector<vision::DetectionResult> results; images.push_back(PyArrayToCvMat(data[i]));
self.BatchPredict(images, &results); }
return results; std::vector<vision::DetectionResult> results;
}) self.BatchPredict(images, &results);
.def_property_readonly("preprocessor", &vision::detection::YOLOv5::GetPreprocessor) return results;
.def_property_readonly("postprocessor", &vision::detection::YOLOv5::GetPostprocessor); })
.def_property_readonly("preprocessor",
&vision::detection::YOLOv5::GetPreprocessor)
.def_property_readonly("postprocessor",
&vision::detection::YOLOv5::GetPostprocessor);
} }
} // namespace fastdeploy } // namespace fastdeploy

143
fastdeploy/vision/detection/contrib/yolov8/postprocessor.cc Executable file
View File

@@ -0,0 +1,143 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision/detection/contrib/yolov8/postprocessor.h"
#include "fastdeploy/vision/utils/utils.h"
namespace fastdeploy {
namespace vision {
namespace detection {
YOLOv8Postprocessor::YOLOv8Postprocessor() {
conf_threshold_ = 0.25;
nms_threshold_ = 0.5;
multi_label_ = true;
max_wh_ = 7680.0;
}
bool YOLOv8Postprocessor::Run(
const std::vector<FDTensor>& tensors, std::vector<DetectionResult>* results,
const std::vector<std::map<std::string, std::array<float, 2>>>& ims_info) {
int batch = tensors[0].shape[0];
// transpose
std::vector<int64_t> dim{0, 2, 1};
FDTensor tensor_transpose;
function::Transpose(tensors[0], &tensor_transpose, dim);
results->resize(batch);
for (size_t bs = 0; bs < batch; ++bs) {
(*results)[bs].Clear();
if (multi_label_) {
(*results)[bs].Reserve(tensor_transpose.shape[1] *
(tensor_transpose.shape[2] - 4));
} else {
(*results)[bs].Reserve(tensor_transpose.shape[1]);
}
if (tensor_transpose.dtype != FDDataType::FP32) {
FDERROR << "Only support post process with float32 data." << std::endl;
return false;
}
const float* data =
reinterpret_cast<const float*>(tensor_transpose.Data()) +
bs * tensor_transpose.shape[1] * tensor_transpose.shape[2];
for (size_t i = 0; i < tensor_transpose.shape[1]; ++i) {
int s = i * tensor_transpose.shape[2];
if (multi_label_) {
for (size_t j = 4; j < tensor_transpose.shape[2]; ++j) {
float confidence = data[s + j];
// filter boxes by conf_threshold
if (confidence <= conf_threshold_) {
continue;
}
int32_t label_id = j - 4;
// convert from [x, y, w, h] to [x1, y1, x2, y2]
(*results)[bs].boxes.emplace_back(std::array<float, 4>{
data[s] - data[s + 2] / 2.0f + label_id * max_wh_,
data[s + 1] - data[s + 3] / 2.0f + label_id * max_wh_,
data[s + 0] + data[s + 2] / 2.0f + label_id * max_wh_,
data[s + 1] + data[s + 3] / 2.0f + label_id * max_wh_});
(*results)[bs].label_ids.push_back(label_id);
(*results)[bs].scores.push_back(confidence);
}
} else {
const float* max_class_score = std::max_element(
data + s + 4, data + s + tensor_transpose.shape[2]);
float confidence = *max_class_score;
// filter boxes by conf_threshold
if (confidence <= conf_threshold_) {
continue;
}
int32_t label_id = std::distance(data + s + 4, max_class_score);
// convert from [x, y, w, h] to [x1, y1, x2, y2]
(*results)[bs].boxes.emplace_back(std::array<float, 4>{
data[s] - data[s + 2] / 2.0f + label_id * max_wh_,
data[s + 1] - data[s + 3] / 2.0f + label_id * max_wh_,
data[s + 0] + data[s + 2] / 2.0f + label_id * max_wh_,
data[s + 1] + data[s + 3] / 2.0f + label_id * max_wh_});
(*results)[bs].label_ids.push_back(label_id);
(*results)[bs].scores.push_back(confidence);
}
}
if ((*results)[bs].boxes.size() == 0) {
return true;
}
utils::NMS(&((*results)[bs]), nms_threshold_);
// scale the boxes to the origin image shape
auto iter_out = ims_info[bs].find("output_shape");
auto iter_ipt = ims_info[bs].find("input_shape");
FDASSERT(iter_out != ims_info[bs].end() && iter_ipt != ims_info[bs].end(),
"Cannot find input_shape or output_shape from im_info.");
float out_h = iter_out->second[0];
float out_w = iter_out->second[1];
float ipt_h = iter_ipt->second[0];
float ipt_w = iter_ipt->second[1];
float scale = std::min(out_h / ipt_h, out_w / ipt_w);
float pad_h = (out_h - ipt_h * scale) / 2;
float pad_w = (out_w - ipt_w * scale) / 2;
for (size_t i = 0; i < (*results)[bs].boxes.size(); ++i) {
int32_t label_id = ((*results)[bs].label_ids)[i];
// clip box
(*results)[bs].boxes[i][0] =
(*results)[bs].boxes[i][0] - max_wh_ * label_id;
(*results)[bs].boxes[i][1] =
(*results)[bs].boxes[i][1] - max_wh_ * label_id;
(*results)[bs].boxes[i][2] =
(*results)[bs].boxes[i][2] - max_wh_ * label_id;
(*results)[bs].boxes[i][3] =
(*results)[bs].boxes[i][3] - max_wh_ * label_id;
(*results)[bs].boxes[i][0] =
std::max(((*results)[bs].boxes[i][0] - pad_w) / scale, 0.0f);
(*results)[bs].boxes[i][1] =
std::max(((*results)[bs].boxes[i][1] - pad_h) / scale, 0.0f);
(*results)[bs].boxes[i][2] =
std::max(((*results)[bs].boxes[i][2] - pad_w) / scale, 0.0f);
(*results)[bs].boxes[i][3] =
std::max(((*results)[bs].boxes[i][3] - pad_h) / scale, 0.0f);
(*results)[bs].boxes[i][0] = std::min((*results)[bs].boxes[i][0], ipt_w);
(*results)[bs].boxes[i][1] = std::min((*results)[bs].boxes[i][1], ipt_h);
(*results)[bs].boxes[i][2] = std::min((*results)[bs].boxes[i][2], ipt_w);
(*results)[bs].boxes[i][3] = std::min((*results)[bs].boxes[i][3], ipt_h);
}
}
return true;
}
} // namespace detection
} // namespace vision
} // namespace fastdeploy

75
fastdeploy/vision/detection/contrib/yolov8/postprocessor.h Executable file
View File

@@ -0,0 +1,75 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "fastdeploy/vision/common/processors/transform.h"
#include "fastdeploy/vision/common/result.h"
namespace fastdeploy {
namespace vision {
namespace detection {
/*! @brief Postprocessor object for YOLOv8 serials model.
*/
class FASTDEPLOY_DECL YOLOv8Postprocessor {
public:
/** \brief Create a postprocessor instance for YOLOv8 serials model
*/
YOLOv8Postprocessor();
/** \brief Process the result of runtime and fill to DetectionResult structure
*
* \param[in] tensors The inference result from runtime
* \param[in] result The output result of detection
* \param[in] ims_info The shape info list, record input_shape and output_shape
* \return true if the postprocess successed, otherwise false
*/
bool Run(const std::vector<FDTensor>& tensors,
std::vector<DetectionResult>* results,
const std::vector<std::map<std::string, std::array<float, 2>>>& ims_info);
/// Set conf_threshold, default 0.25
void SetConfThreshold(const float& conf_threshold) {
conf_threshold_ = conf_threshold;
}
/// Get conf_threshold, default 0.25
float GetConfThreshold() const { return conf_threshold_; }
/// Set nms_threshold, default 0.5
void SetNMSThreshold(const float& nms_threshold) {
nms_threshold_ = nms_threshold;
}
/// Get nms_threshold, default 0.5
float GetNMSThreshold() const { return nms_threshold_; }
/// Set multi_label, set true for eval, default true
void SetMultiLabel(bool multi_label) {
multi_label_ = multi_label;
}
/// Get multi_label, default true
bool GetMultiLabel() const { return multi_label_; }
protected:
float conf_threshold_;
float nms_threshold_;
bool multi_label_;
float max_wh_;
};
} // namespace detection
} // namespace vision
} // namespace fastdeploy

119
fastdeploy/vision/detection/contrib/yolov8/preprocessor.cc Normal file
View File

@@ -0,0 +1,119 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision/detection/contrib/yolov8/preprocessor.h"
#include "fastdeploy/function/concat.h"
namespace fastdeploy {
namespace vision {
namespace detection {
YOLOv8Preprocessor::YOLOv8Preprocessor() {
size_ = {640, 640};
padding_value_ = {114.0, 114.0, 114.0};
is_mini_pad_ = false;
is_no_pad_ = false;
is_scale_up_ = true;
stride_ = 32;
max_wh_ = 7680.0;
}
void YOLOv8Preprocessor::LetterBox(FDMat* mat) {
float scale =
std::min(size_[1] * 1.0 / mat->Height(), size_[0] * 1.0 / mat->Width());
if (!is_scale_up_) {
scale = std::min(scale, 1.0f);
}
int resize_h = int(round(mat->Height() * scale));
int resize_w = int(round(mat->Width() * scale));
int pad_w = size_[0] - resize_w;
int pad_h = size_[1] - resize_h;
if (is_mini_pad_) {
pad_h = pad_h % stride_;
pad_w = pad_w % stride_;
} else if (is_no_pad_) {
pad_h = 0;
pad_w = 0;
resize_h = size_[1];
resize_w = size_[0];
}
if (std::fabs(scale - 1.0f) > 1e-06) {
Resize::Run(mat, resize_w, resize_h);
}
if (pad_h > 0 || pad_w > 0) {
float half_h = pad_h * 1.0 / 2;
int top = int(round(half_h - 0.1));
int bottom = int(round(half_h + 0.1));
float half_w = pad_w * 1.0 / 2;
int left = int(round(half_w - 0.1));
int right = int(round(half_w + 0.1));
Pad::Run(mat, top, bottom, left, right, padding_value_);
}
}
bool YOLOv8Preprocessor::Preprocess(
FDMat* mat, FDTensor* output,
std::map<std::string, std::array<float, 2>>* im_info) {
// Record the shape of image and the shape of preprocessed image
(*im_info)["input_shape"] = {static_cast<float>(mat->Height()),
static_cast<float>(mat->Width())};
// yolov8's preprocess steps
// 1. letterbox
// 2. convert_and_permute(swap_rb=true)
LetterBox(mat);
std::vector<float> alpha = {1.0f / 255.0f, 1.0f / 255.0f, 1.0f / 255.0f};
std::vector<float> beta = {0.0f, 0.0f, 0.0f};
ConvertAndPermute::Run(mat, alpha, beta, true);
// Record output shape of preprocessed image
(*im_info)["output_shape"] = {static_cast<float>(mat->Height()),
static_cast<float>(mat->Width())};
mat->ShareWithTensor(output);
output->ExpandDim(0); // reshape to n, h, w, c
return true;
}
bool YOLOv8Preprocessor::Run(
std::vector<FDMat>* images, std::vector<FDTensor>* outputs,
std::vector<std::map<std::string, std::array<float, 2>>>* ims_info) {
if (images->size() == 0) {
FDERROR << "The size of input images should be greater than 0."
<< std::endl;
return false;
}
ims_info->resize(images->size());
outputs->resize(1);
// Concat all the preprocessed data to a batch tensor
std::vector<FDTensor> tensors(images->size());
for (size_t i = 0; i < images->size(); ++i) {
if (!Preprocess(&(*images)[i], &tensors[i], &(*ims_info)[i])) {
FDERROR << "Failed to preprocess input image." << std::endl;
return false;
}
}
if (tensors.size() == 1) {
(*outputs)[0] = std::move(tensors[0]);
} else {
function::Concat(tensors, &((*outputs)[0]), 0);
}
return true;
}
} // namespace detection
} // namespace vision
} // namespace fastdeploy

113
fastdeploy/vision/detection/contrib/yolov8/preprocessor.h Executable file
View File

@@ -0,0 +1,113 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "fastdeploy/vision/common/processors/transform.h"
#include "fastdeploy/vision/common/result.h"
namespace fastdeploy {
namespace vision {
namespace detection {
/*! @brief Preprocessor object for YOLOv8 serials model.
*/
class FASTDEPLOY_DECL YOLOv8Preprocessor {
public:
/** \brief Create a preprocessor instance for YOLOv8 serials model
*/
YOLOv8Preprocessor();
/** \brief Process the input image and prepare input tensors for runtime
*
* \param[in] images The input image data list, all the elements are returned by cv::imread()
* \param[in] outputs The output tensors which will feed in runtime
* \param[in] ims_info The shape info list, record input_shape and output_shape
* \return true if the preprocess successed, otherwise false
*/
bool Run(std::vector<FDMat>* images, std::vector<FDTensor>* outputs,
std::vector<std::map<std::string, std::array<float, 2>>>* ims_info);
/// Set target size, tuple of (width, height), default size = {640, 640}
void SetSize(const std::vector<int>& size) { size_ = size; }
/// Get target size, tuple of (width, height), default size = {640, 640}
std::vector<int> GetSize() const { return size_; }
/// Set padding value, size should be the same as channels
void SetPaddingValue(const std::vector<float>& padding_value) {
padding_value_ = padding_value;
}
/// Get padding value, size should be the same as channels
std::vector<float> GetPaddingValue() const { return padding_value_; }
/// Set is_scale_up, if is_scale_up is false, the input image only
/// can be zoom out, the maximum resize scale cannot exceed 1.0, default true
void SetScaleUp(bool is_scale_up) {
is_scale_up_ = is_scale_up;
}
/// Get is_scale_up, default true
bool GetScaleUp() const { return is_scale_up_; }
/// Set is_mini_pad, pad to the minimum rectange
/// which height and width is times of stride
void SetMiniPad(bool is_mini_pad) {
is_mini_pad_ = is_mini_pad;
}
/// Get is_mini_pad, default false
bool GetMiniPad() const { return is_mini_pad_; }
/// Set padding stride, only for mini_pad mode
void SetStride(int stride) {
stride_ = stride;
}
/// Get padding stride, default 32
bool GetStride() const { return stride_; }
protected:
bool Preprocess(FDMat* mat, FDTensor* output,
std::map<std::string, std::array<float, 2>>* im_info);
void LetterBox(FDMat* mat);
// target size, tuple of (width, height), default size = {640, 640}
std::vector<int> size_;
// padding value, size should be the same as channels
std::vector<float> padding_value_;
// only pad to the minimum rectange which height and width is times of stride
bool is_mini_pad_;
// while is_mini_pad = false and is_no_pad = true,
// will resize the image to the set size
bool is_no_pad_;
// if is_scale_up is false, the input image only can be zoom out,
// the maximum resize scale cannot exceed 1.0
bool is_scale_up_;
// padding stride, for is_mini_pad
int stride_;
// for offseting the boxes by classes when using NMS
float max_wh_;
};
} // namespace detection
} // namespace vision
} // namespace fastdeploy

82
fastdeploy/vision/detection/contrib/yolov8/yolov8.cc Normal file
View File

@@ -0,0 +1,82 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision/detection/contrib/yolov8/yolov8.h"
namespace fastdeploy {
namespace vision {
namespace detection {
YOLOv8::YOLOv8(const std::string& model_file, const std::string& params_file,
const RuntimeOption& custom_option,
const ModelFormat& model_format) {
if (model_format == ModelFormat::ONNX) {
valid_cpu_backends = {Backend::OPENVINO, Backend::ORT};
valid_gpu_backends = {Backend::ORT, Backend::TRT};
} else {
valid_cpu_backends = {Backend::PDINFER, Backend::ORT, Backend::LITE};
valid_gpu_backends = {Backend::PDINFER, Backend::ORT, Backend::TRT};
}
runtime_option = custom_option;
runtime_option.model_format = model_format;
runtime_option.model_file = model_file;
runtime_option.params_file = params_file;
initialized = Initialize();
}
bool YOLOv8::Initialize() {
if (!InitRuntime()) {
FDERROR << "Failed to initialize fastdeploy backend." << std::endl;
return false;
}
return true;
}
bool YOLOv8::Predict(const cv::Mat& im, DetectionResult* result) {
std::vector<DetectionResult> results;
if (!BatchPredict({im}, &results)) {
return false;
}
*result = std::move(results[0]);
return true;
}
bool YOLOv8::BatchPredict(const std::vector<cv::Mat>& images,
std::vector<DetectionResult>* results) {
std::vector<std::map<std::string, std::array<float, 2>>> ims_info;
std::vector<FDMat> fd_images = WrapMat(images);
if (!preprocessor_.Run(&fd_images, &reused_input_tensors_, &ims_info)) {
FDERROR << "Failed to preprocess the input image." << std::endl;
return false;
}
reused_input_tensors_[0].name = InputInfoOfRuntime(0).name;
if (!Infer(reused_input_tensors_, &reused_output_tensors_)) {
FDERROR << "Failed to inference by runtime." << std::endl;
return false;
}
if (!postprocessor_.Run(reused_output_tensors_, results, ims_info)) {
FDERROR << "Failed to postprocess the inference results by runtime."
<< std::endl;
return false;
}
return true;
}
} // namespace detection
} // namespace vision
} // namespace fastdeploy

76
fastdeploy/vision/detection/contrib/yolov8/yolov8.h Executable file
View File

@@ -0,0 +1,76 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. //NOLINT
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "fastdeploy/fastdeploy_model.h"
#include "fastdeploy/vision/detection/contrib/yolov8/preprocessor.h"
#include "fastdeploy/vision/detection/contrib/yolov8/postprocessor.h"
namespace fastdeploy {
namespace vision {
namespace detection {
/*! @brief YOLOv8 model object used when to load a YOLOv8 model exported by YOLOv8.
*/
class FASTDEPLOY_DECL YOLOv8 : public FastDeployModel {
public:
/** \brief Set path of model file and the configuration of runtime.
*
* \param[in] model_file Path of model file, e.g ./yolov8.onnx
* \param[in] params_file Path of parameter file, e.g ppyoloe/model.pdiparams, if the model format is ONNX, this parameter will be ignored
* \param[in] custom_option RuntimeOption for inference, the default will use cpu, and choose the backend defined in "valid_cpu_backends"
* \param[in] model_format Model format of the loaded model, default is ONNX format
*/
YOLOv8(const std::string& model_file, const std::string& params_file = "",
const RuntimeOption& custom_option = RuntimeOption(),
const ModelFormat& model_format = ModelFormat::ONNX);
std::string ModelName() const { return "yolov8"; }
/** \brief Predict the detection result for an input image
*
* \param[in] img The input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format
* \param[in] result The output detection result will be writen to this structure
* \return true if the prediction successed, otherwise false
*/
virtual bool Predict(const cv::Mat& img, DetectionResult* result);
/** \brief Predict the detection results for a batch of input images
*
* \param[in] imgs, The input image list, each element comes from cv::imread()
* \param[in] results The output detection result list
* \return true if the prediction successed, otherwise false
*/
virtual bool BatchPredict(const std::vector<cv::Mat>& imgs,
std::vector<DetectionResult>* results);
/// Get preprocessor reference of YOLOv8
virtual YOLOv8Preprocessor& GetPreprocessor() {
return preprocessor_;
}
/// Get postprocessor reference of YOLOv8
virtual YOLOv8Postprocessor& GetPostprocessor() {
return postprocessor_;
}
protected:
bool Initialize();
YOLOv8Preprocessor preprocessor_;
YOLOv8Postprocessor postprocessor_;
};
} // namespace detection
} // namespace vision
} // namespace fastdeploy

122
fastdeploy/vision/detection/contrib/yolov8/yolov8_pybind.cc Normal file
View File

@@ -0,0 +1,122 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/pybind/main.h"
namespace fastdeploy {
void BindYOLOv8(pybind11::module& m) {
pybind11::class_<vision::detection::YOLOv8Preprocessor>(m,
"YOLOv8Preprocessor")
.def(pybind11::init<>())
.def(
"run",
[](vision::detection::YOLOv8Preprocessor& self,
std::vector<pybind11::array>& im_list) {
std::vector<vision::FDMat> images;
for (size_t i = 0; i < im_list.size(); ++i) {
images.push_back(vision::WrapMat(PyArrayToCvMat(im_list[i])));
}
std::vector<FDTensor> outputs;
std::vector<std::map<std::string, std::array<float, 2>>> ims_info;
if (!self.Run(&images, &outputs, &ims_info)) {
throw std::runtime_error(
"Failed to preprocess the input data in YOLOv8Preprocessor.");
}
for (size_t i = 0; i < outputs.size(); ++i) {
outputs[i].StopSharing();
}
return make_pair(outputs, ims_info);
})
.def_property("size", &vision::detection::YOLOv8Preprocessor::GetSize,
&vision::detection::YOLOv8Preprocessor::SetSize)
.def_property("padding_value",
&vision::detection::YOLOv8Preprocessor::GetPaddingValue,
&vision::detection::YOLOv8Preprocessor::SetPaddingValue)
.def_property("is_scale_up",
&vision::detection::YOLOv8Preprocessor::GetScaleUp,
&vision::detection::YOLOv8Preprocessor::SetScaleUp)
.def_property("is_mini_pad",
&vision::detection::YOLOv8Preprocessor::GetMiniPad,
&vision::detection::YOLOv8Preprocessor::SetMiniPad)
.def_property("stride", &vision::detection::YOLOv8Preprocessor::GetStride,
&vision::detection::YOLOv8Preprocessor::SetStride);
pybind11::class_<vision::detection::YOLOv8Postprocessor>(
m, "YOLOv8Postprocessor")
.def(pybind11::init<>())
.def("run",
[](vision::detection::YOLOv8Postprocessor& self,
std::vector<FDTensor>& inputs,
const std::vector<std::map<std::string, std::array<float, 2>>>&
ims_info) {
std::vector<vision::DetectionResult> results;
if (!self.Run(inputs, &results, ims_info)) {
throw std::runtime_error(
"Failed to postprocess the runtime result in "
"YOLOv8Postprocessor.");
}
return results;
})
.def("run",
[](vision::detection::YOLOv8Postprocessor& self,
std::vector<pybind11::array>& input_array,
const std::vector<std::map<std::string, std::array<float, 2>>>&
ims_info) {
std::vector<vision::DetectionResult> results;
std::vector<FDTensor> inputs;
PyArrayToTensorList(input_array, &inputs, /*share_buffer=*/true);
if (!self.Run(inputs, &results, ims_info)) {
throw std::runtime_error(
"Failed to postprocess the runtime result in "
"YOLOv8Postprocessor.");
}
return results;
})
.def_property("conf_threshold",
&vision::detection::YOLOv8Postprocessor::GetConfThreshold,
&vision::detection::YOLOv8Postprocessor::SetConfThreshold)
.def_property("nms_threshold",
&vision::detection::YOLOv8Postprocessor::GetNMSThreshold,
&vision::detection::YOLOv8Postprocessor::SetNMSThreshold)
.def_property("multi_label",
&vision::detection::YOLOv8Postprocessor::GetMultiLabel,
&vision::detection::YOLOv8Postprocessor::SetMultiLabel);
pybind11::class_<vision::detection::YOLOv8, FastDeployModel>(m, "YOLOv8")
.def(pybind11::init<std::string, std::string, RuntimeOption,
ModelFormat>())
.def("predict",
[](vision::detection::YOLOv8& self, pybind11::array& data) {
auto mat = PyArrayToCvMat(data);
vision::DetectionResult res;
self.Predict(mat, &res);
return res;
})
.def("batch_predict",
[](vision::detection::YOLOv8& self,
std::vector<pybind11::array>& data) {
std::vector<cv::Mat> images;
for (size_t i = 0; i < data.size(); ++i) {
images.push_back(PyArrayToCvMat(data[i]));
}
std::vector<vision::DetectionResult> results;
self.BatchPredict(images, &results);
return results;
})
.def_property_readonly("preprocessor",
&vision::detection::YOLOv8::GetPreprocessor)
.def_property_readonly("postprocessor",
&vision::detection::YOLOv8::GetPostprocessor);
}
} // namespace fastdeploy

2
fastdeploy/vision/detection/detection_pybind.cc
View File

@@ -29,6 +29,7 @@ void BindNanoDetPlus(pybind11::module& m);
void BindPPDet(pybind11::module& m); void BindPPDet(pybind11::module& m);
void BindYOLOv7End2EndTRT(pybind11::module& m); void BindYOLOv7End2EndTRT(pybind11::module& m);
void BindYOLOv7End2EndORT(pybind11::module& m); void BindYOLOv7End2EndORT(pybind11::module& m);
void BindYOLOv8(pybind11::module& m);
void BindRKYOLO(pybind11::module& m); void BindRKYOLO(pybind11::module& m);
void BindDetection(pybind11::module& m) { void BindDetection(pybind11::module& m) {
@@ -47,6 +48,7 @@ void BindDetection(pybind11::module& m) {
BindNanoDetPlus(detection_module); BindNanoDetPlus(detection_module);
BindYOLOv7End2EndTRT(detection_module); BindYOLOv7End2EndTRT(detection_module);
BindYOLOv7End2EndORT(detection_module); BindYOLOv7End2EndORT(detection_module);
BindYOLOv8(detection_module);
BindRKYOLO(detection_module); BindRKYOLO(detection_module);
} }
} // namespace fastdeploy } // namespace fastdeploy

1
python/fastdeploy/vision/detection/__init__.py
View File

@@ -25,5 +25,6 @@ from .contrib.yolov5lite import YOLOv5Lite
from .contrib.yolov6 import YOLOv6 from .contrib.yolov6 import YOLOv6
from .contrib.yolov7end2end_trt import YOLOv7End2EndTRT from .contrib.yolov7end2end_trt import YOLOv7End2EndTRT
from .contrib.yolov7end2end_ort import YOLOv7End2EndORT from .contrib.yolov7end2end_ort import YOLOv7End2EndORT
from .contrib.yolov8 import *
from .ppdet import * from .ppdet import *
from .contrib.rkyolo import * from .contrib.rkyolo import *

4
python/fastdeploy/vision/detection/contrib/yolov5seg.py Normal file → Executable file
View File

@@ -180,7 +180,7 @@ class YOLOv5Seg(FastDeployModel):
model_file, params_file, self._runtime_option, model_format) model_file, params_file, self._runtime_option, model_format)
assert self.initialized, "YOLOv5Seg initialize failed." assert self.initialized, "YOLOv5Seg initialize failed."
def predict(self, input_image, conf_threshold=0.25, nms_iou_threshold=0.5): def predict(self, input_image):
"""Detect an input image """Detect an input image
:param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format :param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format
@@ -189,8 +189,6 @@ class YOLOv5Seg(FastDeployModel):
:return: DetectionResult :return: DetectionResult
""" """
self.postprocessor.conf_threshold = conf_threshold
self.postprocessor.nms_threshold = nms_iou_threshold
return self._model.predict(input_image) return self._model.predict(input_image)
def batch_predict(self, images): def batch_predict(self, images):

217
python/fastdeploy/vision/detection/contrib/yolov8.py Executable file
View File

@@ -0,0 +1,217 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import absolute_import
import logging
from .... import FastDeployModel, ModelFormat
from .... import c_lib_wrap as C
class YOLOv8Preprocessor:
def __init__(self):
"""Create a preprocessor for YOLOv8
"""
self._preprocessor = C.vision.detection.YOLOv8Preprocessor()
def run(self, input_ims):
"""Preprocess input images for YOLOv8
:param: input_ims: (list of numpy.ndarray)The input image
:return: list of FDTensor
"""
return self._preprocessor.run(input_ims)
@property
def size(self):
"""
Argument for image preprocessing step, the preprocess image size, tuple of (width, height), default size = [640, 640]
"""
return self._preprocessor.size
@property
def padding_value(self):
"""
padding value for preprocessing, default [114.0, 114.0, 114.0]
"""
# padding value, size should be the same as channels
return self._preprocessor.padding_value
@property
def is_scale_up(self):
"""
is_scale_up for preprocessing, the input image only can be zoom out, the maximum resize scale cannot exceed 1.0, default true
"""
return self._preprocessor.is_scale_up
@property
def is_mini_pad(self):
"""
is_mini_pad for preprocessing, pad to the minimum rectange which height and width is times of stride, default false
"""
return self._preprocessor.is_mini_pad
@property
def stride(self):
"""
stride for preprocessing, only for mini_pad mode, default 32
"""
return self._preprocessor.stride
@size.setter
def size(self, wh):
assert isinstance(wh, (list, tuple)),\
"The value to set `size` must be type of tuple or list."
assert len(wh) == 2,\
"The value to set `size` must contatins 2 elements means [width, height], but now it contains {} elements.".format(
len(wh))
self._preprocessor.size = wh
@padding_value.setter
def padding_value(self, value):
assert isinstance(
value,
list), "The value to set `padding_value` must be type of list."
self._preprocessor.padding_value = value
@is_scale_up.setter
def is_scale_up(self, value):
assert isinstance(
value,
bool), "The value to set `is_scale_up` must be type of bool."
self._preprocessor.is_scale_up = value
@is_mini_pad.setter
def is_mini_pad(self, value):
assert isinstance(
value,
bool), "The value to set `is_mini_pad` must be type of bool."
self._preprocessor.is_mini_pad = value
@stride.setter
def stride(self, value):
assert isinstance(
stride, int), "The value to set `stride` must be type of int."
self._preprocessor.stride = value
class YOLOv8Postprocessor:
def __init__(self):
"""Create a postprocessor for YOLOv8
"""
self._postprocessor = C.vision.detection.YOLOv8Postprocessor()
def run(self, runtime_results, ims_info):
"""Postprocess the runtime results for YOLOv8
:param: runtime_results: (list of FDTensor)The output FDTensor results from runtime
:param: ims_info: (list of dict)Record input_shape and output_shape
:return: list of DetectionResult(If the runtime_results is predict by batched samples, the length of this list equals to the batch size)
"""
return self._postprocessor.run(runtime_results, ims_info)
@property
def conf_threshold(self):
"""
confidence threshold for postprocessing, default is 0.25
"""
return self._postprocessor.conf_threshold
@property
def nms_threshold(self):
"""
nms threshold for postprocessing, default is 0.5
"""
return self._postprocessor.nms_threshold
@property
def multi_label(self):
"""
multi_label for postprocessing, set true for eval, default is True
"""
return self._postprocessor.multi_label
@conf_threshold.setter
def conf_threshold(self, conf_threshold):
assert isinstance(conf_threshold, float),\
"The value to set `conf_threshold` must be type of float."
self._postprocessor.conf_threshold = conf_threshold
@nms_threshold.setter
def nms_threshold(self, nms_threshold):
assert isinstance(nms_threshold, float),\
"The value to set `nms_threshold` must be type of float."
self._postprocessor.nms_threshold = nms_threshold
@multi_label.setter
def multi_label(self, value):
assert isinstance(
value,
bool), "The value to set `multi_label` must be type of bool."
self._postprocessor.multi_label = value
class YOLOv8(FastDeployModel):
def __init__(self,
model_file,
params_file="",
runtime_option=None,
model_format=ModelFormat.ONNX):
"""Load a YOLOv8 model exported by YOLOv8.
:param model_file: (str)Path of model file, e.g ./yolov8s.onnx
:param params_file: (str)Path of parameters file, e.g yolox/model.pdiparams, if the model_fomat is ModelFormat.ONNX, this param will be ignored, can be set as empty string
:param runtime_option: (fastdeploy.RuntimeOption)RuntimeOption for inference this model, if it's None, will use the default backend on CPU
:param model_format: (fastdeploy.ModelForamt)Model format of the loaded model
"""
super(YOLOv8, self).__init__(runtime_option)
self._model = C.vision.detection.YOLOv8(
model_file, params_file, self._runtime_option, model_format)
assert self.initialized, "YOLOv8 initialize failed."
def predict(self, input_image):
"""Detect an input image
:param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format
:param conf_threshold: confidence threshold for postprocessing, default is 0.25
:param nms_iou_threshold: iou threshold for NMS, default is 0.5
:return: DetectionResult
"""
return self._model.predict(input_image)
def batch_predict(self, images):
"""Classify a batch of input image
:param im: (list of numpy.ndarray) The input image list, each element is a 3-D array with layout HWC, BGR format
:return list of DetectionResult
"""
return self._model.batch_predict(images)
@property
def preprocessor(self):
"""Get YOLOv8Preprocessor object of the loaded model
:return YOLOv8Preprocessor
"""
return self._model.preprocessor
@property
def postprocessor(self):
"""Get YOLOv8Postprocessor object of the loaded model
:return YOLOv8Postprocessor
"""
return self._model.postprocessor