diff --git a/README_EN.md b/README_EN.md old mode 100755 new mode 100644 index 8f838b5fb..2ec5f6bcc --- a/README_EN.md +++ b/README_EN.md @@ -53,9 +53,10 @@ Including [image classification](examples/vision/classification), [object detect ## 🌠 Recent updates - ✨✨✨ In **2023.01.17** we released [**YOLOv8**](./examples/vision/detection/paddledetection/) for deployment on FastDeploy series hardware, which includes [**Paddle YOLOv8**](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8) and [**ultralytics YOLOv8**](https://github.com/ultralytics/ultralytics) - - Deployable hardware for [**Paddle YOLOv8**](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8)：[**Intel CPU**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**NVIDIA GPU**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Jetson**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Phytium**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Kunlunxin**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**HUAWEI Ascend**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**ARM CPU**](./examples/vision/detection/paddledetection/cpp/infer_yolov8.cc),[**RK3588**](./examples/vision/detection/paddledetection/rknpu2) and [**Sophgo TPU**](./examples/vision/detection/paddledetection/sophgo),including **Python** deployments and **C++** deployments. - - Deployable hardware for [**ultralytics YOLOv8**](https://github.com/ultralytics/ultralytics): [**Intel CPU**](./examples/vision/detection/yolov8), [**NVIDIA GPU**](./examples/vision/detection/yolov8), [**Jetson**](./examples/vision/detection/yolov8), both **Python** deployments and **C++** deployments are included - - FastDeploy one-line model API switch, which can achieve **YOLOv8**, **PP-YOLOE+**, **YOLOv5** and other model performance comparison + + - You can deploy [**Paddle YOLOv8**](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8) on [**Intel CPU**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**NVIDIA GPU**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Jetson**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Phytium**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**Kunlunxin**](./examples/vision/detection/paddledetection/python/infer_yolov8.py), [**HUAWEI Ascend**](./examples/vision/detection/paddledetection/python/infer_yolov8.py) ,[**ARM CPU**](./examples/vision/detection/paddledetection/cpp/infer_yolov8.cc) [**RK3588**](./examples/vision/detection/paddledetection/rknpu2) and [**Sophgo TPU**](./examples/vision/detection/paddledetection/sophgo). Both **Python** deployments and **C++** deployments are included. + - You can deploy [**ultralytics YOLOv8**](https://github.com/ultralytics/ultralytics) on [**Intel CPU**](./examples/vision/detection/yolov8), [**NVIDIA GPU**](./examples/vision/detection/yolov8), [**Jetson**](./examples/vision/detection/yolov8). Both **Python** deployments and **C++** deployments are included + - Fastdeploy supports quick deployment of multiple models, including **YOLOv8**, **PP-YOLOE+**, **YOLOv5** and other models - **✨👥✨ Community** diff --git a/docs/en/build_and_install/README.md b/docs/en/build_and_install/README.md index 05b05e3a6..29712a883 100755 --- a/docs/en/build_and_install/README.md +++ b/docs/en/build_and_install/README.md @@ -17,7 +17,7 @@ English | [中文](../../cn/build_and_install/README.md) - [Build and Install on A311D Platform](a311d.md) - [Build and Install on KunlunXin XPU Platform](kunlunxin.md) - [Build and Install on Huawei Ascend Platform](huawei_ascend.md) - +- [Build and Install on SOPHGO Platform](sophgo.md.md) ## Build options diff --git a/docs/en/faq/rknpu2/rknpu2.md b/docs/en/faq/rknpu2/rknpu2.md index a9afe3b96..f5c195d5a 100644 --- a/docs/en/faq/rknpu2/rknpu2.md +++ b/docs/en/faq/rknpu2/rknpu2.md @@ -14,61 +14,22 @@ The following tests are at end-to-end speed, and the test environment is as foll * with single-core NPU -| Mission Scenario | Model | Model Version(tested version) | ARM CPU/RKNN speed(ms) | -|------------------|-------------------|-------------------------------|--------------------| -| Detection | Picodet | Picodet-s | 162/112 | -| Detection | RKYOLOV5 | YOLOV5-S-Relu(int8) | -/57 | -| Detection | RKYOLOX | - | -/- | -| Detection | RKYOLOV7 | - | -/- | -| Segmentation | Unet | Unet-cityscapes | -/- | -| Segmentation | PP-HumanSegV2Lite | portrait | 133/43 | -| Segmentation | PP-HumanSegV2Lite | human | 133/43 | -| Face Detection | SCRFD | SCRFD-2.5G-kps-640 | 108/42 | +| Mission Scenario | Model | Model Version(tested version) | ARM CPU/RKNN speed(ms) | +|----------------------|------------------------------------------------------------------------------------------|--------------------------|--------------------| +| Detection | [Picodet](../../../../examples/vision/detection/paddledetection/rknpu2/README.md) | Picodet-s | 162/112 | +| Detection | [RKYOLOV5](../../../../examples/vision/detection/rkyolo/README.md) | YOLOV5-S-Relu(int8) | -/57 | +| Detection | [RKYOLOX](../../../../examples/vision/detection/rkyolo/README.md) | - | -/- | +| Detection | [RKYOLOV7](../../../../examples/vision/detection/rkyolo/README.md) | - | -/- | +| Segmentation | [Unet](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | Unet-cityscapes | -/- | +| Segmentation | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | portrait(int8) | 133/43 | +| Segmentation | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | human(int8) | 133/43 | +| Face Detection | [SCRFD](../../../../examples/vision/facedet/scrfd/rknpu2/README.md) | SCRFD-2.5G-kps-640(int8) | 108/42 | +| Face FaceRecognition | [InsightFace](../../../../examples/vision/faceid/insightface/rknpu2/README_CN.md) | ms1mv3_arcface_r18(int8) | 81/12 | +| Classification | [ResNet](../../../../examples/vision/classification/paddleclas/rknpu2/README.md) | ResNet50_vd | -/33 | +## Download Pre-trained library -## How to use RKNPU2 Backend to Infer Models +For convenience, here we provide the 1.0.2 version of FastDeploy. -We provide an example on Scrfd model here to show how to use RKNPU2 Backend for model inference. The modifications mentioned in the annotations below are in comparison to the ONNX CPU. - -```c++ -int infer_scrfd_npu() { - char model_path[] = "./model/scrfd_2.5g_bnkps_shape640x640.rknn"; - char image_file[] = "./image/test_lite_face_detector_3.jpg"; - auto option = fastdeploy::RuntimeOption(); - // Modification1: option.UseRKNPU2 function should be called - option.UseRKNPU2(); - - // Modification2: The parameter 'fastdeploy::ModelFormat::RKNN' should be transferred when loading the model - auto *model = new fastdeploy::vision::facedet::SCRFD(model_path,"",option,fastdeploy::ModelFormat::RKNN); - if (!model->Initialized()) { - std::cerr << "Failed to initialize." << std::endl; - return 0; - } - - // Modification3(optional): RKNPU2 supports to normalize using NPU and the input format is nhwc format. - // The action of DisableNormalizeAndPermute will block the nor action and hwc to chw converting action during preprocessing. - // If you use an already supported model list, please call its method before Predict. - model->DisableNormalizeAndPermute(); - auto im = cv::imread(image_file); - auto im_bak = im.clone(); - fastdeploy::vision::FaceDetectionResult res; - clock_t start = clock(); - if (!model->Predict(&im, &res, 0.8, 0.8)) { - std::cerr << "Failed to predict." << std::endl; - return 0; - } - clock_t end = clock(); - double dur = (double) (end - start); - printf("infer_scrfd_npu use time:%f\n", (dur / CLOCKS_PER_SEC)); - auto vis_im = fastdeploy::vision::Visualize::VisFaceDetection(im_bak, res); - cv::imwrite("scrfd_rknn_vis_result.jpg", vis_im); - std::cout << "Visualized result saved in ./scrfd_rknn_vis_result.jpg" << std::endl; - return 0; -} -``` - - -## Other related Documents -- [How to Build RKNPU2 Deployment Environment](../../build_and_install/rknpu2.md) -- [RKNN-Toolkit2 Installation Document](./install_rknn_toolkit2.md) -- [How to convert ONNX to RKNN](./export.md) +- [FastDeploy RK356X c++ SDK](https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-aarch64-rk356X-1.0.2.tgz) +- [FastDeploy RK3588 c++ SDK](https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-aarch64-rk3588-1.0.2.tgz) diff --git a/examples/vision/detection/yolov6/README.md b/examples/vision/detection/yolov6/README.md index 6e4ede6ee..4e2af9e8e 100755 --- a/examples/vision/detection/yolov6/README.md +++ b/examples/vision/detection/yolov6/README.md @@ -7,6 +7,11 @@ English | [简体中文](README_CN.md) - （1）The *.onnx provided by [Official Repository](https://github.com/meituan/YOLOv6/releases/tag/0.1.0) can directly conduct deployemnt； - （2）Personal models trained by developers should export the ONNX model. Refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. +<<<<<<< HEAD + + +======= +>>>>>>> 30def02a8969f52f40b5e3e305271ef8662126f2 ## Download Pre-trained ONNX Model diff --git a/examples/vision/detection/yolov7end2end_trt/README.md b/examples/vision/detection/yolov7end2end_trt/README.md index 44e3c0b3a..13ddea43a 100755 --- a/examples/vision/detection/yolov7end2end_trt/README.md +++ b/examples/vision/detection/yolov7end2end_trt/README.md @@ -5,6 +5,11 @@ The YOLOv7End2EndTRT deployment is based on [YOLOv7](https://github.com/WongKinY - （1）*.pt provided by [Official Repository](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1) should [Export the ONNX Model](#Export-the-ONNX-Model) to complete the deployment. The deployment of *.trt and *.pose models is not supported. - （2）The YOLOv7 model trained by personal data should [Export the ONNX Model](#%E5%AF%BC%E5%87%BAONNX%E6%A8%A1%E5%9E%8B). Please refer to [Detailed Deployment Documents](#Detailed-Deployment-Documents) to complete the deployment. +<<<<<<< HEAD + + +======= +>>>>>>> 30def02a8969f52f40b5e3e305271ef8662126f2 ## Export the ONNX Model diff --git a/examples/vision/faceid/insightface/cpp/README.md b/examples/vision/faceid/insightface/cpp/README.md index 2960159b1..2be0b3786 100644 --- a/examples/vision/faceid/insightface/cpp/README.md +++ b/examples/vision/faceid/insightface/cpp/README.md @@ -101,7 +101,7 @@ VPL model loading and initialization, among which model_file is the exported ONN #### Predict function > ```c++ -> ArcFace::Predict(cv::Mat* im, FaceRecognitionResult* result) +> ArcFace::Predict(const cv::Mat& im, FaceRecognitionResult* result) > ``` > > Model prediction interface. Input images and output detection results. @@ -121,8 +121,6 @@ Pre-processing and post-processing parameters can be changed by modifying the me Revise through InsightFaceRecognitionPreprocessor::SetAlpha(std::vector& alpha) > > * **beta**(vector<float>): Preprocess normalized beta, and calculated as `x'=x*alpha+beta`. beta defaults to [-1.f, -1.f, -1.f]. Revise through InsightFaceRecognitionPreprocessor::SetBeta(std::vector& beta) -> > * **permute**(bool): Whether to convert BGR to RGB in pre-processing. Default true - Revise through InsightFaceRecognitionPreprocessor::SetPermute(bool permute) #### InsightFaceRecognitionPostprocessor member variables (post-processing parameters) > > * **l2_normalize**(bool): Whether to perform l2 normalization before outputting the face vector. Default false. diff --git a/examples/vision/faceid/insightface/python/README.md b/examples/vision/faceid/insightface/python/README.md index 70f2eee6b..ed5cc8bf7 100644 --- a/examples/vision/faceid/insightface/python/README.md +++ b/examples/vision/faceid/insightface/python/README.md @@ -99,7 +99,6 @@ Member variables of AdaFacePreprocessor are as follows > > * **size**(list[int]): This parameter changes the size of the resize during preprocessing, containing two integer elements for [width, height] with default value [112, 112] > > * **alpha**(list[float]): Preprocess normalized alpha, and calculated as `x'=x*alpha+beta`. alpha defaults to [1. / 127.5, 1.f / 127.5, 1. / 127.5] > > * **beta**(list[float]): Preprocess normalized beta, and calculated as `x'=x*alpha+beta`，beta defaults to [-1.f, -1.f, -1.f] -> > * **swap_rb**(bool): Whether to convert BGR to RGB in pre-processing. Default True #### Member variables of AdaFacePostprocessor Member variables of AdaFacePostprocessor are as follows diff --git a/examples/vision/faceid/insightface/rknpu2/README.md b/examples/vision/faceid/insightface/rknpu2/README.md index 01bee2e28..25e8080b4 100644 --- a/examples/vision/faceid/insightface/rknpu2/README.md +++ b/examples/vision/faceid/insightface/rknpu2/README.md @@ -1,7 +1,7 @@ -[English](README.md) | 简体中文 -# InsightFace RKNPU准备部署模型 +English | [简体中文](README_CN.md) +# InsightFace RKNPU Deployment Example -本教程提供InsightFace模型在RKNPU2环境下的部署，模型的详细介绍已经ONNX模型的下载请查看[模型介绍文档](../README.md)。 +This document provides the deployment of the InsightFace model in the RKNPU2 environment. For details, please refer to [Model Introduction Document].本教程提供InsightFace模型在RKNPU2环境下的部署，模型的详细介绍已经ONNX模型的下载请查看[模型介绍文档](../README.md)。 ## 支持模型列表 目前FastDeploy支持如下模型的部署 diff --git a/examples/vision/faceid/insightface/rknpu2/README_CN.md b/examples/vision/faceid/insightface/rknpu2/README_CN.md new file mode 100644 index 000000000..01bee2e28 --- /dev/null +++ b/examples/vision/faceid/insightface/rknpu2/README_CN.md @@ -0,0 +1,54 @@ +[English](README.md) | 简体中文 +# InsightFace RKNPU准备部署模型 + +本教程提供InsightFace模型在RKNPU2环境下的部署，模型的详细介绍已经ONNX模型的下载请查看[模型介绍文档](../README.md)。 + +## 支持模型列表 +目前FastDeploy支持如下模型的部署 +- ArcFace +- CosFace +- PartialFC +- VPL + +## 下载预训练ONNX模型 + +为了方便开发者的测试，下面提供了InsightFace导出的各系列模型，开发者可直接下载使用。（下表中模型的精度来源于源官方库）其中精度指标来源于InsightFace中对各模型的介绍，详情各参考InsightFace中的说明 + +| 模型 | 大小 | 精度 (AgeDB_30) | +|:-------------------------------------------------------------------------------------------|:------|:--------------| +| [CosFace-r18](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r18.onnx) | 92MB | 97.7 | +| [CosFace-r34](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r34.onnx) | 131MB | 98.3 | +| [CosFace-r50](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r50.onnx) | 167MB | 98.3 | +| [CosFace-r100](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r100.onnx) | 249MB | 98.4 | +| [ArcFace-r18](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx) | 92MB | 97.7 | +| [ArcFace-r34](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r34.onnx) | 131MB | 98.1 | +| [ArcFace-r50](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r50.onnx) | 167MB | - | +| [ArcFace-r100](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r100.onnx) | 249MB | 98.4 | +| [ArcFace-r100_lr0.1](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_r100_lr01.onnx) | 249MB | 98.4 | +| [PartialFC-r34](https://bj.bcebos.com/paddlehub/fastdeploy/partial_fc_glint360k_r50.onnx) | 167MB | - | +| [PartialFC-r50](https://bj.bcebos.com/paddlehub/fastdeploy/partial_fc_glint360k_r100.onnx) | 249MB | - | + + +## 转换为RKNPU模型 + +```bash +wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx + +python -m paddle2onnx.optimize --input_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \ + --output_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \ + --input_shape_dict "{'data':[1,3,112,112]}" + +python /Path/To/FastDeploy/tools/rknpu2/export.py \ + --config_path tools/rknpu2/config/arcface_unquantized.yaml \ + --target_platform rk3588 +``` + +## 详细部署文档 + +- [Python部署](python) +- [C++部署](cpp) + + +## 版本说明 + +- 本版本文档和代码基于[InsightFace CommitID:babb9a5](https://github.com/deepinsight/insightface/commit/babb9a5) 编写 diff --git a/examples/vision/ocr/PP-OCRv3/python/README.md b/examples/vision/ocr/PP-OCRv3/python/README.md index 1661f4770..977c99b9f 100755 --- a/examples/vision/ocr/PP-OCRv3/python/README.md +++ b/examples/vision/ocr/PP-OCRv3/python/README.md @@ -26,7 +26,7 @@ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/ppocr/utils/ppocr_ # Download the example code for deployment git clone https://github.com/PaddlePaddle/FastDeploy.git -cd examples/vison/ocr/PP-OCRv3/python/ +cd examples/vision/ocr/PP-OCRv3/python/ # CPU inference python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image 12.jpg --device cpu diff --git a/examples/vision/ocr/PP-OCRv3/python/README_CN.md b/examples/vision/ocr/PP-OCRv3/python/README_CN.md index 845cc91ab..ae1f59954 100644 --- a/examples/vision/ocr/PP-OCRv3/python/README_CN.md +++ b/examples/vision/ocr/PP-OCRv3/python/README_CN.md @@ -26,7 +26,7 @@ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/ppocr/utils/ppocr_ #下载部署示例代码 git clone https://github.com/PaddlePaddle/FastDeploy.git -cd examples/vison/ocr/PP-OCRv3/python/ +cd examples/vision/ocr/PP-OCRv3/python/ # CPU推理 python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image 12.jpg --device cpu