mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-05 16:48:03 +08:00
[Docs] Pick paddleclas fastdeploy docs from PaddleClas (#1654)
* Adjust folders structures in paddleclas * remove useless files * Update sophgo * improve readme
This commit is contained in:
yunyaoXYY
140
examples/vision/classification/paddleclas/README.md
Executable file → Normal file
140
examples/vision/classification/paddleclas/README.md
Executable file → Normal file
@@ -1,69 +1,99 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas高性能全场景模型部署方案—FastDeploy
|
||||
|
||||
# PaddleClas Model Deployment
|
||||
## 目录
|
||||
- [FastDeploy介绍](#FastDeploy介绍)
|
||||
- [图像分类模型部署](#图像分类模型部署)
|
||||
- [常见问题](#常见问题)
|
||||
|
||||
## Model Description
|
||||
## 1. FastDeploy介绍
|
||||
<div id="FastDeploy介绍"></div>
|
||||
|
||||
- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4)
|
||||
**[⚡️FastDeploy](https://github.com/PaddlePaddle/FastDeploy)**是一款**全场景**、**易用灵活**、**极致高效**的AI推理部署工具，支持**云边端**部署。使用FastDeploy可以简单高效的在X86 CPU、NVIDIA GPU、飞腾CPU、ARM CPU、Intel GPU、Graphcore IPU、昆仑、昇腾、瑞芯微、晶晨、算能等10+款硬件上对PaddleClas模型进行快速部署，并且支持Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNXRuntime、RKNPU2、SOPHGO等多种推理后端。
|
||||
|
||||
Now FastDeploy supports the deployment of the following models
|
||||
<div align="center">
|
||||
|
||||
- [PP-LCNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNet.md)
|
||||
- [PP-LCNetV2 Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- [EfficientNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md)
|
||||
- [GhostNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [MobileNet Models(including v1,v2,v3)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [ShuffleNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [SqueezeNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Others.md)
|
||||
- [Inception Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Inception.md)
|
||||
- [PP-HGNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md)
|
||||
- [ResNet Models（including vd series）](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
<img src="https://user-images.githubusercontent.com/31974251/224941235-d5ea4ed0-7626-4c62-8bbd-8e4fad1e72ad.png" >
|
||||
|
||||
## Prepare PaddleClas Deployment Model
|
||||
</div>
|
||||
|
||||
For PaddleClas model export, refer to [Model Export](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA).
|
||||
## 2. 图像分类模型部署
|
||||
<div id="图像分类模型部署"></div>
|
||||
|
||||
Attention：The model exported by PaddleClas contains two files, including `inference.pdmodel` and `inference.pdiparams`. However, it is necessary to prepare the generic [inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml) file provided by PaddleClas to meet the requirements of deployment. FastDeploy will obtain from the yaml file the preprocessing information required during inference. FastDeploy will get the preprocessing information needed by the model from the yaml file. Developers can directly download this file. But they need to modify the configuration parameters in the yaml file based on personalized needs. Refer to the configuration information in the infer section of the PaddleClas model training [config.](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet)
|
||||
### 2.1 硬件支持列表
|
||||
|
||||
|硬件类型|该硬件是否支持|使用指南|Python|C++|
|
||||
|:---:|:---:|:---:|:---:|:---:|
|
||||
|X86 CPU|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|NVIDIA GPU|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|飞腾CPU|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|ARM CPU|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|Intel GPU(集成显卡)|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|Intel GPU(独立显卡)|✅|[链接](./cpu-gpu)|✅|✅|
|
||||
|Graphcore IPU|✅|[链接](./graphcore)|✅|✅|
|
||||
|昆仑|✅|[链接](./kunlunxin)|✅|✅|
|
||||
|昇腾|✅|[链接](./ascend)|✅|✅|
|
||||
|瑞芯微|✅|[链接](./rockchip)|✅|✅|
|
||||
|晶晨|✅|[链接](./amlogic)|--|✅|✅|
|
||||
|算能|✅|[链接](./sophgo)|✅|✅|
|
||||
|
||||
|
||||
## Download Pre-trained Model
|
||||
### 2.2. 详细使用文档
|
||||
- X86 CPU
|
||||
- [部署模型准备](./cpu-gpu)
|
||||
- [Python部署示例](./cpu-gpu/python/)
|
||||
- [C++部署示例](./cpu-gpu/cpp/)
|
||||
- NVIDIA GPU
|
||||
- [部署模型准备](./cpu-gpu)
|
||||
- [Python部署示例](./cpu-gpu/python/)
|
||||
- [C++部署示例](./cpu-gpu/cpp/)
|
||||
- 飞腾CPU
|
||||
- [部署模型准备](./cpu-gpu)
|
||||
- [Python部署示例](./cpu-gpu/python/)
|
||||
- [C++部署示例](./cpu-gpu/cpp/)
|
||||
- ARM CPU
|
||||
- [部署模型准备](./cpu-gpu)
|
||||
- [Python部署示例](./cpu-gpu/python/)
|
||||
- [C++部署示例](./cpu-gpu/cpp/)
|
||||
- Intel GPU
|
||||
- [部署模型准备](./cpu-gpu)
|
||||
- [Python部署示例](./cpu-gpu/python/)
|
||||
- [C++部署示例](./cpu-gpu/cpp/)
|
||||
- Graphcore IPU
|
||||
- [部署模型准备](./graphcore)
|
||||
- [Python部署示例](./graphcore/python/)
|
||||
- [C++部署示例](./graphcore/cpp/)
|
||||
- 昆仑 XPU
|
||||
- [部署模型准备](./kunlunxin)
|
||||
- [Python部署示例](./kunlunxin/python/)
|
||||
- [C++部署示例](./kunlunxin/cpp/)
|
||||
- 昇腾 Ascend
|
||||
- [部署模型准备](./ascend)
|
||||
- [Python部署示例](./ascend/python/)
|
||||
- [C++部署示例](./ascend/cpp/)
|
||||
- 瑞芯微 Rockchip
|
||||
- [部署模型准备](./rockchip/)
|
||||
- [Python部署示例](./rockchip/rknpu2/)
|
||||
- [C++部署示例](./rockchip/rknpu2/)
|
||||
- 晶晨 Amlogic
|
||||
- [部署模型准备](./amlogic/a311d/)
|
||||
- [C++部署示例](./amlogic/a311d/cpp/)
|
||||
- 算能 Sophgo
|
||||
- [部署模型准备](./sophgo/)
|
||||
- [Python部署示例](./sophgo/python/)
|
||||
- [C++部署示例](./sophgo/cpp/)
|
||||
|
||||
For developers' testing, some models exported by PaddleClas (including the inference_cls.yaml file) are provided below. Developers can download them directly.
|
||||
### 2.3 更多部署方式
|
||||
|
||||
| Model | Parameter File Size |Input Shape | Top1 | Top5 |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- |
|
||||
| [PPLCNet_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNet_x1_0_infer.tgz) | 12MB | 224x224 |71.32% | 90.03% |
|
||||
| [PPLCNetV2_base](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNetV2_base_infer.tgz) | 26MB | 224x224 |77.04% | 93.27% |
|
||||
| [EfficientNetB7](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB7_infer.tgz) | 255MB | 600x600 | 84.3% | 96.9% |
|
||||
| [EfficientNetB0](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_infer.tgz)| 19MB | 224x224 | 77.38% | 93.31% |
|
||||
| [EfficientNetB0_small](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_small_infer.tgz)| 18MB | 224x224 | 75.8% | 92.58% |
|
||||
| [GhostNet_x1_3](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_infer.tgz) | 27MB | 224x224 | 75.79% | 92.54% |
|
||||
| [GhostNet_x1_3_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_ssld_infer.tgz) | 29MB | 224x224 | 79.3% | 94.49% |
|
||||
| [GhostNet_x0_5](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x0_5_infer.tgz) | 10MB | 224x224 | 66.8% | 86.9% |
|
||||
| [MobileNetV1_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_x0_25_infer.tgz) | 1.9MB | 224x224 | 51.4% | 75.5% |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_ssld_infer.tgz) | 17MB | 224x224 | 77.9% | 93.9% |
|
||||
| [MobileNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_x0_25_infer.tgz) | 5.9MB | 224x224 | 53.2% | 76.5% |
|
||||
| [MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_infer.tgz) | 13MB | 224x224 | 72.15% | 90.65% |
|
||||
| [MobileNetV2_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_ssld_infer.tgz) | 14MB | 224x224 | 76.74% | 93.39% |
|
||||
| [MobileNetV3_small_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x1_0_infer.tgz) | 11MB | 224x224 | 68.24% | 88.06% |
|
||||
| [MobileNetV3_small_x0_35_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x0_35_ssld_infer.tgz) | 6.4MB | 224x224 | 55.55% | 77.71% |
|
||||
| [MobileNetV3_large_x1_0_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_large_x1_0_ssld_infer.tgz) | 22MB | 224x224 | 78.96% | 94.48% |
|
||||
| [ShuffleNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x0_25_infer.tgz) | 2.4MB | 224x224 | 49.9% | 73.79% |
|
||||
| [ShuffleNetV2_x2_0](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x2_0_infer.tgz) | 29MB | 224x224 | 73.15% | 91.2% |
|
||||
| [SqueezeNet1_1](https://bj.bcebos.com/paddlehub/fastdeploy/SqueezeNet1_1_infer.tgz) | 4.8MB | 224x224 | 60.1% | 81.9% |
|
||||
| [InceptionV3](https://bj.bcebos.com/paddlehub/fastdeploy/InceptionV3_infer.tgz) | 92MB | 299x299 | 79.14% | 94.59% |
|
||||
| [PPHGNet_tiny_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_tiny_ssld_infer.tgz) | 57MB | 224x224 | 81.95% | 96.12% |
|
||||
| [PPHGNet_small](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_small_infer.tgz) | 87MB | 224x224 | 81.51% | 95.82% |
|
||||
| [PPHGNet_base_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_base_ssld_infer.tgz) | 274MB | 224x224 | 85.0% | 97.35% |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) | 98MB | 224x224 | 79.12% | 94.44% |
|
||||
| [ResNet50](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_infer.tgz) | 91MB | 224x224 | 76.5% | 93% |
|
||||
| [ResNeXt50_32x4d](https://bj.bcebos.com/paddlehub/fastdeploy/ResNeXt50_32x4d_infer.tgz) | 89MB | 224x224 | 77.75% | 93.82% |
|
||||
| [DenseNet121](https://bj.bcebos.com/paddlehub/fastdeploy/DenseNet121_infer.tgz) | 29MB | 224x224 | 75.66% | 92.58% |
|
||||
| [PULC_person_exists](https://bj.bcebos.com/paddlehub/fastdeploy/person_exists_infer.tgz) | 6MB | 224x224 | | |
|
||||
| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%|
|
||||
- [Android ARM CPU部署](./android)
|
||||
- [服务化Serving部署](./serving)
|
||||
- [web部署](./web)
|
||||
- [模型自动化压缩工具](./quantize)
|
||||
|
||||
## Detailed Deployment Documents
|
||||
|
||||
- [Python Deployment](python)
|
||||
- [C++ Deployment](cpp)
|
||||
- [Serving Deployment](serving)
|
||||
## 4. 常见问题
|
||||
<div id="常见问题"></div>
|
||||
|
||||
遇到问题可查看常见问题集合，搜索FastDeploy issue，*或给FastDeploy提交[issue](https://github.com/PaddlePaddle/FastDeploy/issues)*:
|
||||
|
||||
[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq)
|
||||
[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues)
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# Deploy PaddleClas Quantification Model on A311D
|
||||
Now FastDeploy supports the deployment of PaddleClas quantification model to A311D based on Paddle Lite.
|
||||
|
||||
For model quantification and download, refer to [model quantification](../quantize/README.md).
|
||||
|
||||
|
||||
## Detailed Deployment Tutorials
|
||||
|
||||
Only C++ deployment is supported on A311D.
|
||||
|
||||
- [C++ deployment](cpp)
|
||||
@@ -1,12 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 量化模型在 A311D 上的部署
|
||||
目前 FastDeploy 已经支持基于 Paddle Lite 部署 PaddleClas 量化模型到 A311D 上。
|
||||
|
||||
模型的量化和量化模型的下载请参考：[模型量化](../quantize/README.md)
|
||||
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
在 A311D 上只支持 C++ 的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -1,57 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas A311D Development Board C++ Deployment Example
|
||||
`infer.cc` in this directory can help you quickly complete the inference acceleration of PaddleClas quantization model deployment on A311D.
|
||||
|
||||
## Deployment Preparations
|
||||
### FastDeploy Cross-compile Environment Preparations
|
||||
1. For the software and hardware environment, and the cross-compile environment, please refer to [FastDeploy Cross-compile environment](../../../../../../docs/en/build_and_install/a311d.md#Cross-compilation-environment-construction).
|
||||
|
||||
### Quantization Model Preparations
|
||||
1. You can directly use the quantized model provided by FastDeploy for deployment.
|
||||
2. You can use [one-click automatical compression tool](../../../../../../tools/common_tools/auto_compression/) provided by FastDeploy to quantize model by yourself, and use the generated quantized model for deployment.(Note: The quantized classification model still needs the inference_cls.yaml file in the FP32 model folder. Self-quantized model folder does not contain this yaml file, you can copy it from the FP32 model folder to the quantized model folder.)
|
||||
|
||||
For more information, please refer to [Model Quantization](../../quantize/README.md).
|
||||
|
||||
## Deploying the Quantized ResNet50_Vd Segmentation model on A311D
|
||||
Please follow these steps to complete the deployment of the ResNet50_Vd quantization model on A311D.
|
||||
1. Cross-compile the FastDeploy library as described in [Cross-compile FastDeploy](../../../../../../docs/en/build_and_install/a311d.md#FastDeploy-cross-compilation-library-compilation-based-on-Paddle-Lite).
|
||||
|
||||
2. Copy the compiled library to the current directory. You can run this line:
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
```
|
||||
|
||||
3. Download the model and example images required for deployment in current path.
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. Compile the deployment example. You can run the following lines:
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=arm64 ..
|
||||
make -j8
|
||||
make install
|
||||
# After success, an install folder will be created with a running demo and libraries required for deployment.
|
||||
```
|
||||
|
||||
5. Deploy the ResNet50 segmentation model to A311D based on adb. You can run the following lines:
|
||||
```bash
|
||||
# Go to the install directory.
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/build/install/
|
||||
# The following line represents: bash run_with_adb.sh, demo needed to run, model path, image path, DEVICE ID.
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
The output is:
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
|
||||
Please note that the model deployed on A311D needs to be quantized. You can refer to [Model Quantization](../../../../../../docs/en/quantize.md).
|
||||
@@ -1,57 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas A311D 开发板 C++ 部署示例
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PaddleClas 量化模型在 A311D 上的部署推理加速。
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy 交叉编译环境准备
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](../../../../../../docs/cn/build_and_install/a311d.md#交叉编译环境搭建)
|
||||
|
||||
### 量化模型准备
|
||||
1. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户可以使用 FastDeploy 提供的[一键模型自动化压缩工具](../../../../../../tools/common_tools/auto_compression/)，自行进行模型量化, 并使用产出的量化模型进行部署。(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从 FP32 模型文件夹下复制此 yaml 文件到量化后的模型文件夹内即可.)
|
||||
|
||||
更多量化相关相关信息可查阅[模型量化](../../quantize/README.md)
|
||||
|
||||
## 在 A311D 上部署量化后的 ResNet50_Vd 分类模型
|
||||
请按照以下步骤完成在 A311D 上部署 ResNet50_Vd 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](../../../../../../docs/cn/build_and_install/a311d.md#基于-paddlelite-的-fastdeploy-交叉编译库编译)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=arm64 ..
|
||||
make -j8
|
||||
make install
|
||||
# 成功编译之后，会生成 install 文件夹，里面有一个运行 demo 和部署所需的库
|
||||
```
|
||||
|
||||
5. 基于 adb 工具部署 ResNet50 分类模型到晶晨 A311D，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/a311d/cpp/build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
部署成功后运行结果如下：
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
|
||||
需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../../../../docs/cn/quantize.md)
|
||||
25
examples/vision/classification/paddleclas/amlogic/a311d/README.md
Executable file
25
examples/vision/classification/paddleclas/amlogic/a311d/README.md
Executable file
@@ -0,0 +1,25 @@
|
||||
# PaddleClas 模型在晶晨NPU上的部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
|
||||
晶晨A311D是一款先进的AI应用处理器。PaddleClas支持通过FastDeploy在A311D上基于Paddle-Lite部署相关PaddleClas模型。**注意**：需要注意的是，芯原（verisilicon）作为 IP 设计厂商，本身并不提供实体SoC产品，而是授权其 IP 给芯片厂商，如：晶晨（Amlogic），瑞芯微（Rockchip）等。因此本文是适用于被芯原授权了 NPU IP 的芯片产品。只要芯片产品没有大副修改芯原的底层库，则该芯片就可以使用本文档作为 Paddle Lite 推理部署的参考和教程。在本文中，晶晨 SoC 中的 NPU 和 瑞芯微 SoC 中的 NPU 统称为芯原 NPU。目前支持如下芯片的部署：
|
||||
- Amlogic A311D
|
||||
- Amlogic C308X
|
||||
- Amlogic S905D3
|
||||
|
||||
本示例基于晶晨A311D来介绍如何使用FastDeploy部署PaddleClas的量化模型。
|
||||
|
||||
## 2. 使用预导出的模型列表
|
||||
|
||||
FastDeploy提供预先量化好的模型进行部署. 更多模型, 欢迎用户参考[FastDeploy 一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression) 来实现模型量化, 并完成部署.
|
||||
|
||||
|
||||
| 模型 | 量化方式 |
|
||||
|:---------------| :----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | 离线量化 |
|
||||
|
||||
## 3. 详细部署示例
|
||||
|
||||
目前，A311D上只支持C++的部署。
|
||||
- [C++部署](cpp)
|
||||
61
examples/vision/classification/paddleclas/amlogic/a311d/cpp/README.md
Executable file
61
examples/vision/classification/paddleclas/amlogic/a311d/cpp/README.md
Executable file
@@ -0,0 +1,61 @@
|
||||
# PaddleClas A311D 开发板 C++ 部署示例
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PaddleClas 量化模型在 A311D 上的部署推理加速。
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需确认以下两个步骤
|
||||
- 1. 在部署前，需自行编译基于A311D的预测库，参考文档[A311D部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
|
||||
## 2. 量化模型准备
|
||||
1. 需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户也可以使用 FastDeploy 提供的[一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression/)，自行进行模型量化, 并使用产出的量化模型进行部署。(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从 FP32 模型文件夹下复制此 yaml 文件到量化后的模型文件夹内即可.)
|
||||
|
||||
## 3. 在 A311D 上部署量化后的 ResNet50_Vd 分类模型
|
||||
请按照以下步骤完成在 A311D 上部署 ResNet50_Vd 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/a311d.md)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/amlogic/a311d/cpp/
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/amlogic/a311d/cpp/
|
||||
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ ./
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=arm64 ..
|
||||
make -j8
|
||||
make install
|
||||
# 成功编译之后，会生成 install 文件夹，里面有一个运行 demo 和部署所需的库
|
||||
```
|
||||
|
||||
5. 基于 adb 工具部署 ResNet50 分类模型到晶晨 A311D，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
部署成功后运行结果如下：
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
@@ -12,6 +12,7 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
#include <string>
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
@@ -19,7 +20,7 @@ const char sep = '\\';
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
void InitAndInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
@@ -41,10 +42,9 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/quant_model "
|
||||
"path/to/image "
|
||||
@@ -1,82 +1,80 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
## PaddleClas Android Demo Tutorial
|
||||
# PaddleClas Android Demo 使用文档
|
||||
|
||||
Real-time image classification on Android. This demo is easy to use for everyone. For example, you can run your own trained model in the Demo.
|
||||
在 Android 上实现实时的PaddleClas图像分类功能，此 Demo 有很好的的易用性和开放性，如在 Demo 中跑自己训练好的模型等。
|
||||
|
||||
## Prepare the Environment
|
||||
## 环境准备
|
||||
|
||||
1. Install Android Studio in your local environment. Refer to [Android Studio Official Website](https://developer.android.com/studio) for detailed tutorial.
|
||||
2. Prepare an Android phone and turn on the USB debug mode: `Settings -> Find developer options -> Open developer options and USB debug mode`
|
||||
1. 在本地环境安装好 Android Studio 工具，详细安装方法请见[Android Stuido 官网](https://developer.android.com/studio)。
|
||||
2. 准备一部 Android 手机，并开启 USB 调试模式。开启方法: `手机设置 -> 查找开发者选项 -> 打开开发者选项和 USB 调试模式`
|
||||
|
||||
## Deployment steps
|
||||
## 部署步骤
|
||||
|
||||
1. The target detection PaddleClas Demo is located in the `fastdeploy/examples/vision/classification/paddleclas/android`
|
||||
2. Open paddleclas/android project with Android Studio
|
||||
3. Connect the phone to the computer, turn on USB debug mode and file transfer mode, and connect your phone to Android Studio (allow the phone to install software from USB)
|
||||
1. 用 Android Studio 打开 paddleclas/android 工程
|
||||
2. 手机连接电脑，打开 USB 调试和文件传输模式，并在 Android Studio 上连接自己的手机设备（手机需要开启允许从 USB 安装软件权限）
|
||||
|
||||
<p align="center">
|
||||
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/31974251/197338597-2c9e1cf0-569b-49b9-a7fb-cdec71921af8.png">
|
||||
</p>
|
||||
|
||||
> **Attention：**
|
||||
>> If you encounter an NDK configuration error during import, compilation or running, open ` File > Project Structure > SDK Location` and change the path of SDK configured by the `Andriod SDK location`.
|
||||
> **注意：**
|
||||
>> 如果您在导入项目、编译或者运行过程中遇到 NDK 配置错误的提示，请打开 ` File > Project Structure > SDK Location`，修改 `Andriod SDK location` 为您本机配置的 SDK 所在路径。
|
||||
|
||||
4. Click the Run button to automatically compile the APP and install it to the phone. (The process will automatically download the pre-compiled FastDeploy Android library and model files. Internet is required).
|
||||
The final effect is as follows. Figure 1: Install the APP on the phone; Figure 2: The effect when opening the APP. It will automatically recognize and mark the objects in the image; Figure 3: APP setting option. Click setting in the upper right corner and modify your options.
|
||||
4. 点击 Run 按钮，自动编译 APP 并安装到手机。(该过程会自动下载预编译的 FastDeploy Android 库，需要联网)
|
||||
成功后效果如下，图一：APP 安装到手机；图二： APP 打开后的效果，会自动识别图片中的物体并标记；图三：APP设置选项，点击右上角的设置图片，可以设置不同选项进行体验。
|
||||
|
||||
| APP Icon | APP Effect | APP Settings
|
||||
| APP 图标 | APP 效果 | APP设置项
|
||||
| --- | --- | --- |
|
||||
|  |  |  |
|
||||
|
||||
## PaddleClasModel Java API Description
|
||||
- Model initialized API: The initialized API contains two ways: Firstly, initialize directly through the constructor. Secondly, initialize at the appropriate program node by calling the init function. PaddleClasModel initialization parameters are as follows.
|
||||
- modelFile: String. Model file path in paddle format, such as model.pdmodel
|
||||
- paramFile: String. Parameter file path in paddle format, such as model.pdiparams
|
||||
- configFile: String. Preprocessing file for model inference, such as infer_cfg.yml
|
||||
- labelFile: String. This optional parameter indicates the path of the label file and is used for visualization, such as imagenet1k_label_list.txt, each line containing one label
|
||||
- option: RuntimeOption. Optional parameter for model initialization. Default runtime options if not passing the parameter.
|
||||
## PaddleClasModel Java API 说明
|
||||
- 模型初始化 API: 模型初始化API包含两种方式，方式一是通过构造函数直接初始化；方式二是，通过调用init函数，在合适的程序节点进行初始化。PaddleClasModel初始化参数说明如下：
|
||||
- modelFile: String, paddle格式的模型文件路径，如 model.pdmodel
|
||||
- paramFile: String, paddle格式的参数文件路径，如 model.pdiparams
|
||||
- configFile: String, 模型推理的预处理配置文件，如 infer_cfg.yml
|
||||
- labelFile: String, 可选参数，表示label标签文件所在路径，用于可视化，如 imagenet1k_label_list.txt，每一行包含一个label
|
||||
- option: RuntimeOption，可选参数，模型初始化option。如果不传入该参数则会使用默认的运行时选项。
|
||||
|
||||
```java
|
||||
// Constructor: constructor w/o label file
|
||||
public PaddleClasModel(); // An empty constructor, which can be initialized by calling init
|
||||
// 构造函数: constructor w/o label file
|
||||
public PaddleClasModel(); // 空构造函数，之后可以调用init初始化
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, String labelFile);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, RuntimeOption option);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, String labelFile, RuntimeOption option);
|
||||
// Call init manually for initialization: call init manually w/o label file
|
||||
// 手动调用init初始化: call init manually w/o label file
|
||||
public boolean init(String modelFile, String paramsFile, String configFile, RuntimeOption option);
|
||||
public boolean init(String modelFile, String paramsFile, String configFile, String labelFile, RuntimeOption option);
|
||||
```
|
||||
- Model Prediction API: The Model Prediction API contains an API for direct prediction and an API for visualization. In direct prediction, we do not save the image and render the result on Bitmap. Instead, we merely predict the inference result. For prediction and visualization, the results are both predicted and visualized, the visualized images are saved to the specified path, and the visualized results are rendered in Bitmap (Now Bitmap in ARGB8888 format is supported). Afterward, the Bitmap can be displayed on the camera.
|
||||
- 模型预测 API：模型预测API包含直接预测的API以及带可视化功能的API。直接预测是指，不保存图片以及不渲染结果到Bitmap上，仅预测推理结果。预测并且可视化是指，预测结果以及可视化，并将可视化后的图片保存到指定的途径，以及将可视化结果渲染在Bitmap(目前支持ARGB8888格式的Bitmap), 后续可将该Bitmap在camera中进行显示。
|
||||
```java
|
||||
// Direct prediction: No image saving and no result rendering to Bitmap
|
||||
// 直接预测：不保存图片以及不渲染结果到Bitmap上
|
||||
public ClassifyResult predict(Bitmap ARGB8888Bitmap)；
|
||||
// Prediction and visualization: Predict and visualize the results, save the visualized image to the specified path, and render the visualized results on Bitmap
|
||||
// 预测并且可视化：预测结果以及可视化，并将可视化后的图片保存到指定的途径，以及将可视化结果渲染在Bitmap上
|
||||
public ClassifyResult predict(Bitmap ARGB8888Bitmap, String savedImagePath, float scoreThreshold)
|
||||
```
|
||||
- Model resource release API: Call release() API to release model resources. Return true for successful release and false for failure; call initialized() to determine whether the model was initialized successfully, with true indicating successful initialization and false indicating failure.
|
||||
- 模型资源释放 API：调用 release() API 可以释放模型资源，返回true表示释放成功，false表示失败；调用 initialized() 可以判断模型是否初始化成功，true表示初始化成功，false表示失败。
|
||||
```java
|
||||
public boolean release(); // Realise native resources
|
||||
public boolean initialized(); // Check if initialization is successful
|
||||
public boolean release(); // 释放native资源
|
||||
public boolean initialized(); // 检查是否初始化成功
|
||||
```
|
||||
- RuntimeOption settings
|
||||
- RuntimeOption设置说明
|
||||
```java
|
||||
public void enableLiteFp16(); // Enable fp16 accuracy inference
|
||||
public void disableLiteFP16(); // Disable fp16 accuracy inference
|
||||
public void setCpuThreadNum(int threadNum); // Set thread numbers
|
||||
public void setLitePowerMode(LitePowerMode mode); // Set power mode
|
||||
public void setLitePowerMode(String modeStr); // Set power mode through character string
|
||||
public void enableRecordTimeOfRuntime(); // Whether the print model is time-consuming
|
||||
public void enableLiteFp16(); // 开启fp16精度推理
|
||||
public void disableLiteFP16(); // 关闭fp16精度推理
|
||||
public void setCpuThreadNum(int threadNum); // 设置线程数
|
||||
public void setLitePowerMode(LitePowerMode mode); // 设置能耗模式
|
||||
public void setLitePowerMode(String modeStr); // 通过字符串形式设置能耗模式
|
||||
public void enableRecordTimeOfRuntime(); // 是否打印模型运行耗时
|
||||
```
|
||||
|
||||
- Model ClassifyResult
|
||||
- 模型结果ClassifyResult说明
|
||||
```java
|
||||
public float[] mScores; // [n] Score
|
||||
public int[] mLabelIds; // [n] Classification ID
|
||||
public boolean initialized(); // Whether the result is valid or not
|
||||
public float[] mScores; // [n] 得分
|
||||
public int[] mLabelIds; // [n] 分类ID
|
||||
public boolean initialized(); // 检测结果是否有效
|
||||
```
|
||||
|
||||
- Model Calling Example 1: Using Constructor or Default RuntimeOption
|
||||
- 模型调用示例1：使用构造函数以及默认的RuntimeOption
|
||||
```java
|
||||
import java.nio.ByteBuffer;
|
||||
import android.graphics.Bitmap;
|
||||
@@ -85,67 +83,67 @@ import android.opengl.GLES20;
|
||||
import com.baidu.paddle.fastdeploy.vision.ClassifyResult;
|
||||
import com.baidu.paddle.fastdeploy.vision.classification.PaddleClasModel;
|
||||
|
||||
// Initialize the model
|
||||
// 初始化模型
|
||||
PaddleClasModel model = new PaddleClasModel("MobileNetV1_x0_25_infer/inference.pdmodel",
|
||||
"MobileNetV1_x0_25_infer/inference.pdiparams",
|
||||
"MobileNetV1_x0_25_infer/inference_cls.yml");
|
||||
|
||||
// Read the image: The following is merely the pseudo code to read the Bitmap
|
||||
// 读取图片: 以下仅为读取Bitmap的伪代码
|
||||
ByteBuffer pixelBuffer = ByteBuffer.allocate(width * height * 4);
|
||||
GLES20.glReadPixels(0, 0, width, height, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, pixelBuffer);
|
||||
Bitmap ARGB8888ImageBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
|
||||
ARGB8888ImageBitmap.copyPixelsFromBuffer(pixelBuffer);
|
||||
|
||||
// Model inference
|
||||
// 模型推理
|
||||
ClassifyResult result = model.predict(ARGB8888ImageBitmap);
|
||||
|
||||
// Release model resources
|
||||
// 释放模型资源
|
||||
model.release();
|
||||
```
|
||||
|
||||
- Model calling example 2: Manually call init at the appropriate program node and self-define RuntimeOption
|
||||
- 模型调用示例2: 在合适的程序节点，手动调用init，并自定义RuntimeOption
|
||||
```java
|
||||
// import is as the above...
|
||||
// import 同上 ...
|
||||
import com.baidu.paddle.fastdeploy.RuntimeOption;
|
||||
import com.baidu.paddle.fastdeploy.LitePowerMode;
|
||||
import com.baidu.paddle.fastdeploy.vision.ClassifyResult;
|
||||
import com.baidu.paddle.fastdeploy.vision.classification.PaddleClasModel;
|
||||
// Create an empty model
|
||||
// 新建空模型
|
||||
PaddleClasModel model = new PaddleClasModel();
|
||||
// Model path
|
||||
// 模型路径
|
||||
String modelFile = "MobileNetV1_x0_25_infer/inference.pdmodel";
|
||||
String paramFile = "MobileNetV1_x0_25_infer/inference.pdiparams";
|
||||
String configFile = "MobileNetV1_x0_25_infer/inference_cls.yml";
|
||||
// Specify RuntimeOption
|
||||
// 指定RuntimeOption
|
||||
RuntimeOption option = new RuntimeOption();
|
||||
option.setCpuThreadNum(2);
|
||||
option.setLitePowerMode(LitePowerMode.LITE_POWER_HIGH);
|
||||
option.enableRecordTimeOfRuntime();
|
||||
option.enableLiteFp16();
|
||||
// Use init function for initialization
|
||||
// 使用init函数初始化
|
||||
model.init(modelFile, paramFile, configFile, option);
|
||||
// Bitmap reading, model prediction, and resource release are as above ...
|
||||
// Bitmap读取、模型预测、资源释放 同上 ...
|
||||
```
|
||||
Refer to [MainActivity](./app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification/ClassificationMainActivity.java) for more information
|
||||
更详细的用法请参考 [MainActivity](./app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification/ClassificationMainActivity.java) 中的用法
|
||||
|
||||
## Replace FastDeploy Prediction Library and Models
|
||||
It’s simple to replace the FastDeploy prediction library and models. The prediction library is located at `app/libs/fastdeploy-android-xxx-shared`, where `xxx` represents the version of your prediction library. The models are located at `app/src/main/assets/models/MobileNetV1_x0_25_infer`.
|
||||
- Steps to replace FastDeploy prediction library:
|
||||
- Download or compile the latest FastDeploy Android SDK, unzip and place it in the `app/libs`;
|
||||
- Modify the default value of the model path in `app/src/main/cpp/CMakeLists.txt` and to the prediction library path you download or compile. For example,
|
||||
## 替换 FastDeploy 预测库和模型
|
||||
替换FastDeploy预测库和模型的步骤非常简单。预测库所在的位置为 `app/libs/fastdeploy-android-xxx-shared`，其中 `xxx` 表示当前您使用的预测库版本号。模型所在的位置为，`app/src/main/assets/models/MobileNetV1_x0_25_infer`。
|
||||
- 替换FastDeploy预测库的步骤:
|
||||
- 下载或编译最新的FastDeploy Android预测库，解压缩后放在 `app/libs` 目录下;
|
||||
- 修改 `app/src/main/cpp/CMakeLists.txt` 中的预测库路径，指向您下载或编译的预测库路径。如：
|
||||
```cmake
|
||||
set(FastDeploy_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../../libs/fastdeploy-android-xxx-shared")
|
||||
```
|
||||
- Steps to replace PaddleClas models:
|
||||
- Put your PaddleClas model in `app/src/main/assets/models`;
|
||||
- Modify the default value of the model path in `app/src/main/res/values/strings.xml`. For example,
|
||||
- 替换PaddleClas模型的步骤：
|
||||
- 将您的PaddleClas分类模型放在 `app/src/main/assets/models` 目录下；
|
||||
- 修改 `app/src/main/res/values/strings.xml` 中模型路径的默认值，如：
|
||||
```xml
|
||||
<!-- Change this path to your model, such as models/MobileNetV2_x0_25_infer -->
|
||||
<!-- 将这个路径指修改成您的模型，如 models/MobileNetV2_x0_25_infer -->
|
||||
<string name="CLASSIFICATION_MODEL_DIR_DEFAULT">models/MobileNetV1_x0_25_infer</string>
|
||||
<string name="CLASSIFICATION_LABEL_PATH_DEFAULT">labels/imagenet1k_label_list.txt</string>
|
||||
```
|
||||
|
||||
## More Reference Documents
|
||||
For more FastDeploy Java API documentes and how to access FastDeploy C++ API via JNI, refer to:
|
||||
- [Use FastDeploy Java SDK in Android](../../../../../java/android/)
|
||||
- [Use FastDeploy C++ SDK in Android](../../../../../docs/en/faq/use_cpp_sdk_on_android.md)
|
||||
## 更多参考文档
|
||||
如果您想知道更多的FastDeploy Java API文档以及如何通过JNI来接入FastDeploy C++ API感兴趣，可以参考以下内容:
|
||||
- [在 Android 中使用 FastDeploy Java SDK](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android)
|
||||
- [在 Android 中使用 FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_cpp_sdk_on_android.md)
|
||||
|
||||
@@ -1,151 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
## 图像分类 PaddleClas Android Demo 使用文档
|
||||
|
||||
在 Android 上实现实时的图像分类功能，此 Demo 有很好的的易用性和开放性，如在 Demo 中跑自己训练好的模型等。
|
||||
|
||||
## 环境准备
|
||||
|
||||
1. 在本地环境安装好 Android Studio 工具，详细安装方法请见[Android Stuido 官网](https://developer.android.com/studio)。
|
||||
2. 准备一部 Android 手机，并开启 USB 调试模式。开启方法: `手机设置 -> 查找开发者选项 -> 打开开发者选项和 USB 调试模式`
|
||||
|
||||
## 部署步骤
|
||||
|
||||
1. 目标检测 PaddleClas Demo 位于 `fastdeploy/examples/vision/classification/paddleclas/android` 目录
|
||||
2. 用 Android Studio 打开 paddleclas/android 工程
|
||||
3. 手机连接电脑，打开 USB 调试和文件传输模式，并在 Android Studio 上连接自己的手机设备（手机需要开启允许从 USB 安装软件权限）
|
||||
|
||||
<p align="center">
|
||||
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/31974251/197338597-2c9e1cf0-569b-49b9-a7fb-cdec71921af8.png">
|
||||
</p>
|
||||
|
||||
> **注意：**
|
||||
>> 如果您在导入项目、编译或者运行过程中遇到 NDK 配置错误的提示，请打开 ` File > Project Structure > SDK Location`，修改 `Andriod SDK location` 为您本机配置的 SDK 所在路径。
|
||||
|
||||
4. 点击 Run 按钮，自动编译 APP 并安装到手机。(该过程会自动下载预编译的 FastDeploy Android 库，需要联网)
|
||||
成功后效果如下，图一：APP 安装到手机；图二： APP 打开后的效果，会自动识别图片中的物体并标记；图三：APP设置选项，点击右上角的设置图片，可以设置不同选项进行体验。
|
||||
|
||||
| APP 图标 | APP 效果 | APP设置项
|
||||
| --- | --- | --- |
|
||||
|  |  |  |
|
||||
|
||||
## PaddleClasModel Java API 说明
|
||||
- 模型初始化 API: 模型初始化API包含两种方式，方式一是通过构造函数直接初始化；方式二是，通过调用init函数，在合适的程序节点进行初始化。PaddleClasModel初始化参数说明如下：
|
||||
- modelFile: String, paddle格式的模型文件路径，如 model.pdmodel
|
||||
- paramFile: String, paddle格式的参数文件路径，如 model.pdiparams
|
||||
- configFile: String, 模型推理的预处理配置文件，如 infer_cfg.yml
|
||||
- labelFile: String, 可选参数，表示label标签文件所在路径，用于可视化，如 imagenet1k_label_list.txt，每一行包含一个label
|
||||
- option: RuntimeOption，可选参数，模型初始化option。如果不传入该参数则会使用默认的运行时选项。
|
||||
|
||||
```java
|
||||
// 构造函数: constructor w/o label file
|
||||
public PaddleClasModel(); // 空构造函数，之后可以调用init初始化
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, String labelFile);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, RuntimeOption option);
|
||||
public PaddleClasModel(String modelFile, String paramsFile, String configFile, String labelFile, RuntimeOption option);
|
||||
// 手动调用init初始化: call init manually w/o label file
|
||||
public boolean init(String modelFile, String paramsFile, String configFile, RuntimeOption option);
|
||||
public boolean init(String modelFile, String paramsFile, String configFile, String labelFile, RuntimeOption option);
|
||||
```
|
||||
- 模型预测 API：模型预测API包含直接预测的API以及带可视化功能的API。直接预测是指，不保存图片以及不渲染结果到Bitmap上，仅预测推理结果。预测并且可视化是指，预测结果以及可视化，并将可视化后的图片保存到指定的途径，以及将可视化结果渲染在Bitmap(目前支持ARGB8888格式的Bitmap), 后续可将该Bitmap在camera中进行显示。
|
||||
```java
|
||||
// 直接预测：不保存图片以及不渲染结果到Bitmap上
|
||||
public ClassifyResult predict(Bitmap ARGB8888Bitmap)；
|
||||
// 预测并且可视化：预测结果以及可视化，并将可视化后的图片保存到指定的途径，以及将可视化结果渲染在Bitmap上
|
||||
public ClassifyResult predict(Bitmap ARGB8888Bitmap, String savedImagePath, float scoreThreshold)
|
||||
```
|
||||
- 模型资源释放 API：调用 release() API 可以释放模型资源，返回true表示释放成功，false表示失败；调用 initialized() 可以判断模型是否初始化成功，true表示初始化成功，false表示失败。
|
||||
```java
|
||||
public boolean release(); // 释放native资源
|
||||
public boolean initialized(); // 检查是否初始化成功
|
||||
```
|
||||
- RuntimeOption设置说明
|
||||
```java
|
||||
public void enableLiteFp16(); // 开启fp16精度推理
|
||||
public void disableLiteFP16(); // 关闭fp16精度推理
|
||||
public void setCpuThreadNum(int threadNum); // 设置线程数
|
||||
public void setLitePowerMode(LitePowerMode mode); // 设置能耗模式
|
||||
public void setLitePowerMode(String modeStr); // 通过字符串形式设置能耗模式
|
||||
public void enableRecordTimeOfRuntime(); // 是否打印模型运行耗时
|
||||
```
|
||||
|
||||
- 模型结果ClassifyResult说明
|
||||
```java
|
||||
public float[] mScores; // [n] 得分
|
||||
public int[] mLabelIds; // [n] 分类ID
|
||||
public boolean initialized(); // 检测结果是否有效
|
||||
```
|
||||
|
||||
- 模型调用示例1：使用构造函数以及默认的RuntimeOption
|
||||
```java
|
||||
import java.nio.ByteBuffer;
|
||||
import android.graphics.Bitmap;
|
||||
import android.opengl.GLES20;
|
||||
|
||||
import com.baidu.paddle.fastdeploy.vision.ClassifyResult;
|
||||
import com.baidu.paddle.fastdeploy.vision.classification.PaddleClasModel;
|
||||
|
||||
// 初始化模型
|
||||
PaddleClasModel model = new PaddleClasModel("MobileNetV1_x0_25_infer/inference.pdmodel",
|
||||
"MobileNetV1_x0_25_infer/inference.pdiparams",
|
||||
"MobileNetV1_x0_25_infer/inference_cls.yml");
|
||||
|
||||
// 读取图片: 以下仅为读取Bitmap的伪代码
|
||||
ByteBuffer pixelBuffer = ByteBuffer.allocate(width * height * 4);
|
||||
GLES20.glReadPixels(0, 0, width, height, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, pixelBuffer);
|
||||
Bitmap ARGB8888ImageBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
|
||||
ARGB8888ImageBitmap.copyPixelsFromBuffer(pixelBuffer);
|
||||
|
||||
// 模型推理
|
||||
ClassifyResult result = model.predict(ARGB8888ImageBitmap);
|
||||
|
||||
// 释放模型资源
|
||||
model.release();
|
||||
```
|
||||
|
||||
- 模型调用示例2: 在合适的程序节点，手动调用init，并自定义RuntimeOption
|
||||
```java
|
||||
// import 同上 ...
|
||||
import com.baidu.paddle.fastdeploy.RuntimeOption;
|
||||
import com.baidu.paddle.fastdeploy.LitePowerMode;
|
||||
import com.baidu.paddle.fastdeploy.vision.ClassifyResult;
|
||||
import com.baidu.paddle.fastdeploy.vision.classification.PaddleClasModel;
|
||||
// 新建空模型
|
||||
PaddleClasModel model = new PaddleClasModel();
|
||||
// 模型路径
|
||||
String modelFile = "MobileNetV1_x0_25_infer/inference.pdmodel";
|
||||
String paramFile = "MobileNetV1_x0_25_infer/inference.pdiparams";
|
||||
String configFile = "MobileNetV1_x0_25_infer/inference_cls.yml";
|
||||
// 指定RuntimeOption
|
||||
RuntimeOption option = new RuntimeOption();
|
||||
option.setCpuThreadNum(2);
|
||||
option.setLitePowerMode(LitePowerMode.LITE_POWER_HIGH);
|
||||
option.enableRecordTimeOfRuntime();
|
||||
option.enableLiteFp16();
|
||||
// 使用init函数初始化
|
||||
model.init(modelFile, paramFile, configFile, option);
|
||||
// Bitmap读取、模型预测、资源释放 同上 ...
|
||||
```
|
||||
更详细的用法请参考 [MainActivity](./app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification/ClassificationMainActivity.java) 中的用法
|
||||
|
||||
## 替换 FastDeploy 预测库和模型
|
||||
替换FastDeploy预测库和模型的步骤非常简单。预测库所在的位置为 `app/libs/fastdeploy-android-xxx-shared`，其中 `xxx` 表示当前您使用的预测库版本号。模型所在的位置为，`app/src/main/assets/models/MobileNetV1_x0_25_infer`。
|
||||
- 替换FastDeploy预测库的步骤:
|
||||
- 下载或编译最新的FastDeploy Android预测库，解压缩后放在 `app/libs` 目录下;
|
||||
- 修改 `app/src/main/cpp/CMakeLists.txt` 中的预测库路径，指向您下载或编译的预测库路径。如：
|
||||
```cmake
|
||||
set(FastDeploy_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../../libs/fastdeploy-android-xxx-shared")
|
||||
```
|
||||
- 替换PaddleClas模型的步骤：
|
||||
- 将您的PaddleClas分类模型放在 `app/src/main/assets/models` 目录下；
|
||||
- 修改 `app/src/main/res/values/strings.xml` 中模型路径的默认值，如：
|
||||
```xml
|
||||
<!-- 将这个路径指修改成您的模型，如 models/MobileNetV2_x0_25_infer -->
|
||||
<string name="CLASSIFICATION_MODEL_DIR_DEFAULT">models/MobileNetV1_x0_25_infer</string>
|
||||
<string name="CLASSIFICATION_LABEL_PATH_DEFAULT">labels/imagenet1k_label_list.txt</string>
|
||||
```
|
||||
|
||||
## 更多参考文档
|
||||
如果您想知道更多的FastDeploy Java API文档以及如何通过JNI来接入FastDeploy C++ API感兴趣，可以参考以下内容:
|
||||
- [在 Android 中使用 FastDeploy Java SDK](../../../../../java/android/)
|
||||
- [在 Android 中使用 FastDeploy C++ SDK](../../../../../docs/cn/faq/use_cpp_sdk_on_android.md)
|
||||
16
examples/vision/classification/paddleclas/README_CN.md → examples/vision/classification/paddleclas/ascend/README.md
Executable file → Normal file
16
examples/vision/classification/paddleclas/README_CN.md → examples/vision/classification/paddleclas/ascend/README.md
Executable file → Normal file
@@ -1,7 +1,11 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 模型部署
|
||||
|
||||
## 模型版本说明
|
||||
# PaddleClas 模型在昇腾上的部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在昇腾上部署图像分类模型.
|
||||
|
||||
## 2. 模型版本说明
|
||||
|
||||
- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4)
|
||||
|
||||
@@ -18,14 +22,14 @@
|
||||
- [PP-HGNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md)
|
||||
- [ResNet系列模型（包含vd系列）](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
## 准备PaddleClas部署模型
|
||||
### 2.1 准备PaddleClas部署模型
|
||||
|
||||
PaddleClas模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
注意：PaddleClas导出的模型仅包含`inference.pdmodel`和`inference.pdiparams`两个文件，但为了满足部署的需求，同时也需准备其提供的通用[inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml)文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息，开发者可直接下载此文件使用。但需根据自己的需求修改yaml文件中的配置参数，具体可比照PaddleClas模型训练[config](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet)中的infer部分的配置信息进行修改。
|
||||
|
||||
|
||||
## 下载预训练模型
|
||||
### 2.2 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleClas导出的部分模型（含inference_cls.yaml文件），开发者可直接下载使用。
|
||||
|
||||
@@ -62,8 +66,6 @@ PaddleClas模型导出，请参考其文档说明[模型导出](https://github.c
|
||||
| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%|
|
||||
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
## 3. 详细部署的部署示例
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [服务化部署](serving)
|
||||
@@ -0,0 +1,53 @@
|
||||
# PaddleClas 昇腾 C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`, 供用户完成PaddleClas模型在昇腾AI处理器上的部署.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需确认以下两个步骤
|
||||
- 1. 在部署前，需自行编译基于昇腾AI处理器的预测库，参考文档[昇腾AI处理器部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
- 2. 部署时需要环境初始化, 请参考[如何使用C++在昇腾AI处理器部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_sdk_on_ascend.md)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/ascend/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/ascend/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 使用编译完成的FastDeploy库编译infer_demo
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-ascend
|
||||
make -j
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 使用昇腾部署
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
```
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 C++ API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas Python部署](../python)
|
||||
@@ -0,0 +1,56 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void AscendInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseAscend();
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(&im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/model path/to/image " << std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
std::string model_dir = argv[1];
|
||||
std::string test_image = argv[2];
|
||||
AscendInfer(model_dir, test_image);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
# PaddleClas 昇腾 Python部署示例
|
||||
本目录下提供`infer.py`快速完成PaddleClas在昇腾AI处理器上部署的示例.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需自行编译基于昇腾AI处理器的FastDeploy python wheel包并安装，参考文档，参考文档[昇腾AI处理器部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
```bash
|
||||
# 安装FastDpeloy 昇腾预测库 python包（详细文档请参考`部署环境准备`）
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/ascend/python
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/ascend/python
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 在Ascend AI 处理器上推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --topk 1
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 Python API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/image_classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
25
examples/vision/classification/paddleclas/python/infer.py → examples/vision/classification/paddleclas/ascend/python/infer.py
Executable file → Normal file
25
examples/vision/classification/paddleclas/python/infer.py → examples/vision/classification/paddleclas/ascend/python/infer.py
Executable file → Normal file
@@ -13,37 +13,14 @@ def parse_arguments():
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--topk", type=int, default=1, help="Return topk results.")
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
default='cpu',
|
||||
help="Type of inference device, support 'cpu' or 'gpu' or 'ipu' or 'kunlunxin' or 'ascend' ."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--use_trt",
|
||||
type=ast.literal_eval,
|
||||
default=False,
|
||||
help="Wether to use tensorrt.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
|
||||
option = fd.RuntimeOption()
|
||||
|
||||
if args.device.lower() == "gpu":
|
||||
option.use_gpu()
|
||||
|
||||
if args.device.lower() == "ipu":
|
||||
option.use_ipu()
|
||||
|
||||
if args.device.lower() == "kunlunxin":
|
||||
option.use_kunlunxin()
|
||||
|
||||
if args.device.lower() == "ascend":
|
||||
option.use_ascend()
|
||||
|
||||
if args.use_trt:
|
||||
option.use_trt_backend()
|
||||
return option
|
||||
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas C Deployment Example
|
||||
|
||||
This directory provides examples that `infer.c` fast finishes the deployment of PaddleClas models on CPU/GPU.
|
||||
|
||||
Before deployment, two steps require confirmation.
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
- 2. Download the precompiled deployment library and samples code according to your development environment. Refer to [FastDeploy Precompiled Library](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
|
||||
Taking ResNet50_vd inference on Linux as an example, the compilation test can be completed by executing the following command in this directory. FastDeploy version 1.0.4 or above (x.x.x>=1.0.4) is required to support this model.
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download FastDeploy precompiled library. Users can choose your appropriate version in the`FastDeploy Precompiled Library` mentioned above
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# Download ResNet50_vd model file and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# CPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
```
|
||||
|
||||
The above command works for Linux or MacOS. Refer to
|
||||
- [How to use FastDeploy C++ SDK in Windows](../../../../../docs/cn/faq/use_sdk_on_windows.md) for SDK use-pattern in Windows
|
||||
|
||||
## PaddleClas C Interface
|
||||
|
||||
### RuntimeOption
|
||||
|
||||
```c
|
||||
FD_C_RuntimeOptionWrapper* FD_C_CreateRuntimeOptionWrapper()
|
||||
```
|
||||
|
||||
> Create a RuntimeOption object, and return a pointer to manipulate it.
|
||||
>
|
||||
> **Return**
|
||||
>
|
||||
> * **fd_c_runtime_option_wrapper**(FD_C_RuntimeOptionWrapper*): Pointer to manipulate RuntimeOption object.
|
||||
|
||||
|
||||
```c
|
||||
void FD_C_RuntimeOptionWrapperUseCpu(
|
||||
FD_C_RuntimeOptionWrapper* fd_c_runtime_option_wrapper)
|
||||
```
|
||||
|
||||
> Enable Cpu inference.
|
||||
>
|
||||
> **Params**
|
||||
>
|
||||
> * **fd_c_runtime_option_wrapper**(FD_C_RuntimeOptionWrapper*): Pointer to manipulate RuntimeOption object.
|
||||
|
||||
```c
|
||||
void FD_C_RuntimeOptionWrapperUseGpu(
|
||||
FD_C_RuntimeOptionWrapper* fd_c_runtime_option_wrapper,
|
||||
int gpu_id)
|
||||
```
|
||||
> 开启GPU推理
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> * **fd_c_runtime_option_wrapper**(FD_C_RuntimeOptionWrapper*): Pointer to manipulate RuntimeOption object.
|
||||
> * **gpu_id**(int): gpu id
|
||||
|
||||
|
||||
### Model
|
||||
|
||||
```c
|
||||
|
||||
FD_C_PaddleClasModelWrapper* FD_C_CreatePaddleClasModelWrapper(
|
||||
const char* model_file, const char* params_file, const char* config_file,
|
||||
FD_C_RuntimeOptionWrapper* runtime_option,
|
||||
const FD_C_ModelFormat model_format)
|
||||
|
||||
```
|
||||
|
||||
> Create a PaddleClas model object, and return a pointer to manipulate it.
|
||||
>
|
||||
> **Params**
|
||||
>
|
||||
> * **model_file**(const char*): Model file path
|
||||
> * **params_file**(const char*): Parameter file path
|
||||
> * **config_file**(const char*): Configuration file path, which is the deployment yaml file exported by PaddleClas.
|
||||
> * **runtime_option**(FD_C_RuntimeOptionWrapper*): Backend inference configuration. None by default, which is the default configuration
|
||||
> * **model_format**(FD_C_ModelFormat): Model format. FD_C_ModelFormat_PADDLE format by default
|
||||
>
|
||||
> **Return**
|
||||
> * **fd_c_ppclas_wrapper**(FD_C_PaddleClasModelWrapper*): Pointer to manipulate PaddleClas object.
|
||||
|
||||
|
||||
#### Read and write image
|
||||
|
||||
```c
|
||||
FD_C_Mat FD_C_Imread(const char* imgpath)
|
||||
```
|
||||
|
||||
> Read an image, and return a pointer to cv::Mat.
|
||||
>
|
||||
> **Params**
|
||||
>
|
||||
> * **imgpath**(const char*): image path
|
||||
>
|
||||
> **Return**
|
||||
>
|
||||
> * **imgmat**(FD_C_Mat): pointer to cv::Mat object which holds the image.
|
||||
|
||||
|
||||
```c
|
||||
FD_C_Bool FD_C_Imwrite(const char* savepath, FD_C_Mat img);
|
||||
```
|
||||
|
||||
> Write image to a file.
|
||||
>
|
||||
> **Params**
|
||||
>
|
||||
> * **savepath**(const char*): save path
|
||||
> * **img**(FD_C_Mat): pointer to cv::Mat object
|
||||
>
|
||||
> **Return**
|
||||
>
|
||||
> * **result**(FD_C_Bool): bool to indicate success or failure
|
||||
|
||||
|
||||
#### Prediction
|
||||
|
||||
```c
|
||||
FD_C_Bool FD_C_PaddleClasModelWrapperPredict(
|
||||
__fd_take FD_C_PaddleClasModelWrapper* fd_c_ppclas_wrapper, FD_C_Mat img,
|
||||
FD_C_ClassifyResult* fd_c_ppclas_result)
|
||||
```
|
||||
>
|
||||
> Predict an image, and generate classification result.
|
||||
>
|
||||
> **Params**
|
||||
> * **fd_c_ppclas_wrapper**(FD_C_PaddleClasModelWrapper*): pointer to manipulate PaddleClas object
|
||||
> * **img**（FD_C_Mat）: pointer to cv::Mat object, which can be obained by FD_C_Imread interface
|
||||
> * **fd_c_ppclas_result** (FD_C_ClassifyResult*): The classification result, including label_id, and the corresponding confidence. Refer to [Visual Model Prediction Results](../../../../../docs/api/vision_results/) for the description of ClassifyResult
|
||||
|
||||
|
||||
#### Result
|
||||
|
||||
```c
|
||||
void FD_C_ClassifyResultStr(
|
||||
FD_C_ClassifyResult* fd_c_classify_result,
|
||||
char* str_buffer);
|
||||
```
|
||||
>
|
||||
> print result
|
||||
>
|
||||
> **Params**
|
||||
> * **fd_c_classify_result**(FD_C_ClassifyResult*): pointer to FD_C_ClassifyResult structure
|
||||
> * **str_buffer**(char*): used to store result string
|
||||
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Python Deployment](../python)
|
||||
- [Visual Model prediction results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/en/faq/how_to_change_backend.md)
|
||||
@@ -1,86 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas C++ Deployment Example
|
||||
|
||||
This directory provides examples that `infer.cc` fast finishes the deployment of PaddleClas models on CPU/GPU and GPU accelerated by TensorRT.
|
||||
|
||||
Before deployment, two steps require confirmation.
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
- 2. Download the precompiled deployment library and samples code according to your development environment. Refer to [FastDeploy Precompiled Library](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
|
||||
Taking ResNet50_vd inference on Linux as an example, the compilation test can be completed by executing the following command in this directory. FastDeploy version 0.7.0 or above (x.x.x>=0.7.0) is required to support this model.
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download FastDeploy precompiled library. Users can choose your appropriate version in the`FastDeploy Precompiled Library` mentioned above
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# Download ResNet50_vd model file and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# CPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
# TensorRT inference on GPU
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 2
|
||||
# IPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 3
|
||||
# KunlunXin XPU inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 4
|
||||
# Ascend inference
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 5
|
||||
```
|
||||
|
||||
The above command works for Linux or MacOS. Refer to
|
||||
- [How to use FastDeploy C++ SDK in Windows](../../../../../docs/cn/faq/use_sdk_on_windows.md) for SDK use-pattern in Windows
|
||||
|
||||
## PaddleClas C++ Interface
|
||||
|
||||
### PaddleClas Class
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::classification::PaddleClasModel(
|
||||
const string& model_file,
|
||||
const string& params_file,
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleClas model loading and initialization, where model_file and params_file are the Paddle inference files exported from the training model. Refer to [Model Export](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA) for more information
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default. (use the default configuration)
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
#### Predict function
|
||||
|
||||
> ```c++
|
||||
> PaddleClasModel::Predict(cv::Mat* im, ClassifyResult* result, int topk = 1)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output results directly.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **im**: Input images in HWC or BGR format
|
||||
> > * **result**: The classification result, including label_id, and the corresponding confidence. Refer to [Visual Model Prediction Results](../../../../../docs/api/vision_results/) for the description of ClassifyResult
|
||||
> > * **topk**(int): Return the topk classification results with the highest prediction probability. Default 1
|
||||
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Python Deployment](../python)
|
||||
- [Visual Model prediction results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/en/faq/how_to_change_backend.md)
|
||||
@@ -1,89 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成PaddleClas系列模型在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
以Linux上ResNet50_vd推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# CPU推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
# GPU上TensorRT推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 2
|
||||
# IPU推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 3
|
||||
# KunlunXin XPU推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 4
|
||||
# Huawei Ascend NPU推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 5
|
||||
```
|
||||
|
||||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
如果用户使用华为昇腾NPU部署, 请参考以下方式在部署前初始化部署环境:
|
||||
- [如何使用华为昇腾NPU部署](../../../../../docs/cn/faq/use_sdk_on_ascend.md)
|
||||
|
||||
## PaddleClas C++接口
|
||||
|
||||
### PaddleClas类
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::classification::PaddleClasModel(
|
||||
const string& model_file,
|
||||
const string& params_file,
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleClas模型加载和初始化，其中model_file, params_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
#### Predict函数
|
||||
|
||||
> ```c++
|
||||
> PaddleClasModel::Predict(cv::Mat* im, ClassifyResult* result, int topk = 1)
|
||||
> ```
|
||||
>
|
||||
> 模型预测接口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **im**: 输入图像，注意需为HWC，BGR格式
|
||||
> > * **result**: 分类结果，包括label_id，以及相应的置信度, ClassifyResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
> > * **topk**(int):返回预测概率最高的topk个分类结果，默认为1
|
||||
|
||||
|
||||
- [模型介绍](../../)
|
||||
- [Python部署](../python)
|
||||
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
@@ -1,202 +0,0 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void CpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseCpu();
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
void GpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
void IpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseIpu();
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
void KunlunXinInfer(const std::string& model_dir,
|
||||
const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseKunlunXin();
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
void TrtInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend();
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
void AscendInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseAscend();
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(&im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
if (argc < 4) {
|
||||
std::cout << "Usage: infer_demo path/to/model path/to/image run_option, "
|
||||
"e.g ./infer_demo ./ResNet50_vd ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run "
|
||||
"with ipu; 4: run with kunlunxin."
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (std::atoi(argv[3]) == 0) {
|
||||
CpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 1) {
|
||||
GpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 2) {
|
||||
TrtInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 3) {
|
||||
IpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 4) {
|
||||
KunlunXinInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 5) {
|
||||
AscendInfer(argv[1], argv[2]);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
73
examples/vision/classification/paddleclas/cpu-gpu/README.md
Normal file
73
examples/vision/classification/paddleclas/cpu-gpu/README.md
Normal file
@@ -0,0 +1,73 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# PaddleClas 模型在CPU与GPU上的部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署PaddleClas系列模型
|
||||
|
||||
## 2. 模型版本说明
|
||||
|
||||
- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [PP-LCNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNet.md)
|
||||
- [PP-LCNetV2系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- [EfficientNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md)
|
||||
- [GhostNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [MobileNet系列模型(包含v1,v2,v3)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [ShuffleNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [SqueezeNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Others.md)
|
||||
- [Inception系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Inception.md)
|
||||
- [PP-HGNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md)
|
||||
- [ResNet系列模型（包含vd系列）](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
### 2.1 准备PaddleClas部署模型
|
||||
|
||||
PaddleClas模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
注意：PaddleClas导出的模型仅包含`inference.pdmodel`和`inference.pdiparams`两个文件，但为了满足部署的需求，同时也需准备其提供的通用[inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml)文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息，开发者可直接下载此文件使用。但需根据自己的需求修改yaml文件中的配置参数，具体可比照PaddleClas模型训练[config](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet)中的infer部分的配置信息进行修改。
|
||||
|
||||
|
||||
### 2.2 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleClas导出的部分模型（含inference_cls.yaml文件），开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | Top1 | Top5 |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- |
|
||||
| [PPLCNet_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNet_x1_0_infer.tgz) | 12MB | 224x224 |71.32% | 90.03% |
|
||||
| [PPLCNetV2_base](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNetV2_base_infer.tgz) | 26MB | 224x224 |77.04% | 93.27% |
|
||||
| [EfficientNetB7](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB7_infer.tgz) | 255MB | 600x600 | 84.3% | 96.9% |
|
||||
| [EfficientNetB0](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_infer.tgz)| 19MB | 224x224 | 77.38% | 93.31% |
|
||||
| [EfficientNetB0_small](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_small_infer.tgz)| 18MB | 224x224 | 75.8% | 92.58% |
|
||||
| [GhostNet_x1_3](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_infer.tgz) | 27MB | 224x224 | 75.79% | 92.54% |
|
||||
| [GhostNet_x1_3_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_ssld_infer.tgz) | 29MB | 224x224 | 79.3% | 94.49% |
|
||||
| [GhostNet_x0_5](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x0_5_infer.tgz) | 10MB | 224x224 | 66.8% | 86.9% |
|
||||
| [MobileNetV1_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_x0_25_infer.tgz) | 1.9MB | 224x224 | 51.4% | 75.5% |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_ssld_infer.tgz) | 17MB | 224x224 | 77.9% | 93.9% |
|
||||
| [MobileNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_x0_25_infer.tgz) | 5.9MB | 224x224 | 53.2% | 76.5% |
|
||||
| [MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_infer.tgz) | 13MB | 224x224 | 72.15% | 90.65% |
|
||||
| [MobileNetV2_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_ssld_infer.tgz) | 14MB | 224x224 | 76.74% | 93.39% |
|
||||
| [MobileNetV3_small_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x1_0_infer.tgz) | 11MB | 224x224 | 68.24% | 88.06% |
|
||||
| [MobileNetV3_small_x0_35_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x0_35_ssld_infer.tgz) | 6.4MB | 224x224 | 55.55% | 77.71% |
|
||||
| [MobileNetV3_large_x1_0_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_large_x1_0_ssld_infer.tgz) | 22MB | 224x224 | 78.96% | 94.48% |
|
||||
| [ShuffleNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x0_25_infer.tgz) | 2.4MB | 224x224 | 49.9% | 73.79% |
|
||||
| [ShuffleNetV2_x2_0](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x2_0_infer.tgz) | 29MB | 224x224 | 73.15% | 91.2% |
|
||||
| [SqueezeNet1_1](https://bj.bcebos.com/paddlehub/fastdeploy/SqueezeNet1_1_infer.tgz) | 4.8MB | 224x224 | 60.1% | 81.9% |
|
||||
| [InceptionV3](https://bj.bcebos.com/paddlehub/fastdeploy/InceptionV3_infer.tgz) | 92MB | 299x299 | 79.14% | 94.59% |
|
||||
| [PPHGNet_tiny_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_tiny_ssld_infer.tgz) | 57MB | 224x224 | 81.95% | 96.12% |
|
||||
| [PPHGNet_small](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_small_infer.tgz) | 87MB | 224x224 | 81.51% | 95.82% |
|
||||
| [PPHGNet_base_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_base_ssld_infer.tgz) | 274MB | 224x224 | 85.0% | 97.35% |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) | 98MB | 224x224 | 79.12% | 94.44% |
|
||||
| [ResNet50](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_infer.tgz) | 91MB | 224x224 | 76.5% | 93% |
|
||||
| [ResNeXt50_32x4d](https://bj.bcebos.com/paddlehub/fastdeploy/ResNeXt50_32x4d_infer.tgz) | 89MB | 224x224 | 77.75% | 93.82% |
|
||||
| [DenseNet121](https://bj.bcebos.com/paddlehub/fastdeploy/DenseNet121_infer.tgz) | 29MB | 224x224 | 75.66% | 92.58% |
|
||||
| [PULC_person_exists](https://bj.bcebos.com/paddlehub/fastdeploy/person_exists_infer.tgz) | 6MB | 224x224 | | |
|
||||
| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%|
|
||||
|
||||
|
||||
## 3. 详细部署的部署示例
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [C部署](c)
|
||||
- [C#部署](csharp)
|
||||
56
examples/vision/classification/paddleclas/c/README_CN.md → examples/vision/classification/paddleclas/cpu-gpu/c/README.md
Normal file → Executable file
56
examples/vision/classification/paddleclas/c/README_CN.md → examples/vision/classification/paddleclas/cpu-gpu/c/README.md
Normal file → Executable file
@@ -1,16 +1,30 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas C 部署示例
|
||||
# PaddleClas CPU-GPU C部署示例
|
||||
|
||||
本目录下提供`infer_xxx.c`来调用C API快速完成PaddleClas系列模型在CPU/GPU上部署的示例。
|
||||
本目录下提供`infer.c`来调用C API快速完成PaddleClas模型在CPU/GPU上部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
## 1. 说明
|
||||
PaddleClas支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署图像分类模型.
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
## 2. 部署环境准备
|
||||
在部署前，需确认软硬件环境，同时下载预编译部署库，参考[FastDeploy安装文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#FastDeploy预编译库安装)安装FastDeploy预编译库.
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.4以上(x.x.x>=1.0.4)
|
||||
|
||||
以Linux上ResNet50_vd推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.4以上(x.x.x>=1.0.4)
|
||||
## 3. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 4.运行部署示例
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/cpu-gpu/c
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/cpu-gpu/c
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
@@ -25,19 +39,19 @@ tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# CPU推理
|
||||
# 使用CPU在OpenVINO推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU推理
|
||||
# 使用GPU在TensorRT推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
```
|
||||
|
||||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
- 注意，以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考文档: [如何在Windows中使用FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
如果用户使用华为昇腾NPU部署, 请参考以下方式在部署前初始化部署环境:
|
||||
- [如何使用华为昇腾NPU部署](../../../../../docs/cn/faq/use_sdk_on_ascend.md)
|
||||
|
||||
## PaddleClas C API接口
|
||||
## 5. PaddleClas C API接口简介
|
||||
下面提供了PaddleClas的C API简介
|
||||
- 如果用户想要更换部署后端或进行其他定制化操作, 请查看[C Runtime API](https://baidu-paddle.github.io/fastdeploy-api/c/html/runtime__option_8h.html).
|
||||
- 更多 PaddleClas C API 请查看 [C PaddleClas API](https://github.com/PaddlePaddle/FastDeploy/blob/develop/c_api/fastdeploy_capi/vision/classification/ppcls/model.h)
|
||||
|
||||
### 配置
|
||||
|
||||
@@ -101,7 +115,7 @@ FD_C_PaddleClasModelWrapper* FD_C_CreatePaddleClasModelWrapper(
|
||||
> * **fd_c_ppclas_wrapper**(FD_C_PaddleClasModelWrapper*): 指向PaddleClas模型对象的指针
|
||||
|
||||
|
||||
#### 读写图像
|
||||
### 读写图像
|
||||
|
||||
```c
|
||||
FD_C_Mat FD_C_Imread(const char* imgpath)
|
||||
@@ -134,7 +148,7 @@ FD_C_Bool FD_C_Imwrite(const char* savepath, FD_C_Mat img);
|
||||
> * **result**(FD_C_Bool): 表示操作是否成功
|
||||
|
||||
|
||||
#### Predict函数
|
||||
### Predict函数
|
||||
|
||||
```c
|
||||
FD_C_Bool FD_C_PaddleClasModelWrapperPredict(
|
||||
@@ -150,7 +164,7 @@ FD_C_Bool FD_C_PaddleClasModelWrapperPredict(
|
||||
> * **fd_c_ppclas_result**（FD_C_ClassifyResult*): 分类结果，包括label_id，以及相应的置信度, ClassifyResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
|
||||
|
||||
#### Predict结果
|
||||
### Predict结果
|
||||
|
||||
```c
|
||||
void FD_C_ClassifyResultStr(
|
||||
@@ -164,9 +178,9 @@ void FD_C_ClassifyResultStr(
|
||||
> * **str_buffer**(char*): 保存结果数据信息的字符串
|
||||
|
||||
|
||||
## 6. 其它文档
|
||||
|
||||
|
||||
- [模型介绍](../../)
|
||||
- [Python部署](../python)
|
||||
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas Python部署](../python)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
- [PaddleClas C# 部署](../csharp)
|
||||
@@ -23,7 +23,7 @@ const char sep = '\\';
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void CpuInfer(const char* model_dir, const char* image_file) {
|
||||
void CpuInfer(const char *model_dir, const char *image_file) {
|
||||
char model_file[100];
|
||||
char params_file[100];
|
||||
char config_file[100];
|
||||
@@ -34,10 +34,10 @@ void CpuInfer(const char* model_dir, const char* image_file) {
|
||||
snprintf(config_file, max_size, "%s%c%s", model_dir, sep,
|
||||
"inference_cls.yaml");
|
||||
|
||||
FD_C_RuntimeOptionWrapper* option = FD_C_CreateRuntimeOptionWrapper();
|
||||
FD_C_RuntimeOptionWrapper *option = FD_C_CreateRuntimeOptionWrapper();
|
||||
FD_C_RuntimeOptionWrapperUseCpu(option);
|
||||
|
||||
FD_C_PaddleClasModelWrapper* model = FD_C_CreatePaddleClasModelWrapper(
|
||||
FD_C_PaddleClasModelWrapper *model = FD_C_CreatePaddleClasModelWrapper(
|
||||
model_file, params_file, config_file, option, FD_C_ModelFormat_PADDLE);
|
||||
|
||||
if (!FD_C_PaddleClasModelWrapperInitialized(model)) {
|
||||
@@ -49,7 +49,8 @@ void CpuInfer(const char* model_dir, const char* image_file) {
|
||||
|
||||
FD_C_Mat im = FD_C_Imread(image_file);
|
||||
|
||||
FD_C_ClassifyResult* result = FD_C_CreateClassifyResult();
|
||||
FD_C_ClassifyResult *result =
|
||||
(FD_C_ClassifyResult *)malloc(sizeof(FD_C_ClassifyResult));
|
||||
|
||||
if (!FD_C_PaddleClasModelWrapperPredict(model, im, result)) {
|
||||
printf("Failed to predict.\n");
|
||||
@@ -70,7 +71,7 @@ void CpuInfer(const char* model_dir, const char* image_file) {
|
||||
FD_C_DestroyMat(im);
|
||||
}
|
||||
|
||||
void GpuInfer(const char* model_dir, const char* image_file) {
|
||||
void GpuInfer(const char *model_dir, const char *image_file) {
|
||||
char model_file[100];
|
||||
char params_file[100];
|
||||
char config_file[100];
|
||||
@@ -81,10 +82,10 @@ void GpuInfer(const char* model_dir, const char* image_file) {
|
||||
snprintf(config_file, max_size, "%s%c%s", model_dir, sep,
|
||||
"inference_cls.yaml");
|
||||
|
||||
FD_C_RuntimeOptionWrapper* option = FD_C_CreateRuntimeOptionWrapper();
|
||||
FD_C_RuntimeOptionWrapper *option = FD_C_CreateRuntimeOptionWrapper();
|
||||
FD_C_RuntimeOptionWrapperUseGpu(option, 0);
|
||||
|
||||
FD_C_PaddleClasModelWrapper* model = FD_C_CreatePaddleClasModelWrapper(
|
||||
FD_C_PaddleClasModelWrapper *model = FD_C_CreatePaddleClasModelWrapper(
|
||||
model_file, params_file, config_file, option, FD_C_ModelFormat_PADDLE);
|
||||
|
||||
if (!FD_C_PaddleClasModelWrapperInitialized(model)) {
|
||||
@@ -96,7 +97,8 @@ void GpuInfer(const char* model_dir, const char* image_file) {
|
||||
|
||||
FD_C_Mat im = FD_C_Imread(image_file);
|
||||
|
||||
FD_C_ClassifyResult* result = FD_C_CreateClassifyResult();
|
||||
FD_C_ClassifyResult *result =
|
||||
(FD_C_ClassifyResult *)malloc(sizeof(FD_C_ClassifyResult));
|
||||
|
||||
if (!FD_C_PaddleClasModelWrapperPredict(model, im, result)) {
|
||||
printf("Failed to predict.\n");
|
||||
@@ -117,7 +119,7 @@ void GpuInfer(const char* model_dir, const char* image_file) {
|
||||
FD_C_DestroyMat(im);
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 4) {
|
||||
printf(
|
||||
"Usage: infer_demo path/to/model_dir path/to/image run_option, "
|
||||
@@ -1,5 +1,5 @@
|
||||
PROJECT(infer_demo C CXX)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.12)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||
|
||||
# 指定下载解压后的fastdeploy库路径
|
||||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||
106
examples/vision/classification/paddleclas/cpu-gpu/cpp/README.md
Executable file
106
examples/vision/classification/paddleclas/cpu-gpu/cpp/README.md
Executable file
@@ -0,0 +1,106 @@
|
||||
# PaddleClas CPU-GPU C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成PaddleClas系列模型在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署图像分类模型.
|
||||
|
||||
## 2. 部署环境准备
|
||||
在部署前，需确认软硬件环境，同时下载预编译部署库，参考[FastDeploy安装文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#FastDeploy预编译库安装)安装FastDeploy预编译库.
|
||||
|
||||
## 3. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 4. 运行部署示例
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/cpu-gpu/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/cpu-gpu/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# 在CPU上使用Paddle Inference推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# 在CPU上使用OenVINO推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
# 在CPU上使用ONNX Runtime推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 2
|
||||
# 在CPU上使用Paddle Lite推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 3
|
||||
# 在GPU上使用Paddle Inference推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 4
|
||||
# 在GPU上使用Paddle TensorRT推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 5
|
||||
# 在GPU上使用ONNX Runtime推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 6
|
||||
# 在GPU上使用Nvidia TensorRT推理
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 7
|
||||
```
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
|
||||
## 5. 部署示例选项说明
|
||||
在我们使用`infer_demo`时, 输入了3个参数, 分别为分类模型, 预测图片, 与最后一位的数字选项.
|
||||
现在下表将解释最后一位数字选项的含义.
|
||||
|数字选项|含义|
|
||||
|:---:|:---:|
|
||||
|0| 在CPU上使用Paddle Inference推理 |
|
||||
|1| 在CPU上使用OenVINO推理 |
|
||||
|2| 在CPU上使用ONNX Runtime推理 |
|
||||
|3| 在CPU上使用Paddle Lite推理 |
|
||||
|4| 在GPU上使用Paddle Inference推理 |
|
||||
|5| 在GPU上使用Paddle TensorRT推理 |
|
||||
|6| 在GPU上使用ONNX Runtime推理 |
|
||||
|7| 在GPU上使用Nvidia TensorRT推理 |
|
||||
|
||||
- 关于如何通过FastDeploy使用更多不同的推理后端，以及如何使用不同的硬件，请参考文档：[如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
|
||||
## 6. 更多指南
|
||||
- [PaddleClas系列 C++ API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1classification.html)
|
||||
- [PaddleClas Python部署](../python)
|
||||
- [PaddleClas C 部署](../c)
|
||||
- [PaddleClas C# 部署](../csharp)
|
||||
|
||||
## 7. 常见问题
|
||||
- PaddleClas能在FastDeploy支持的多种后端上推理,支持情况如下表所示, 如何切换后端, 详见文档[如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
|
||||
|硬件类型|支持的后端|
|
||||
|:---:|:---:|
|
||||
|X86 CPU| Paddle Inference, ONNX Runtime, OpenVINO |
|
||||
|ARM CPU| Paddle Lite |
|
||||
|飞腾 CPU| ONNX Runtime |
|
||||
|NVIDIA GPU| Paddle Inference, ONNX Runtime, TensorRT |
|
||||
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
- [编译Jetson部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/jetson.md)
|
||||
@@ -19,16 +19,18 @@ const char sep = '\\';
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void InitAndInfer(const std::string& model_dir, const std::string& image_file,
|
||||
const fastdeploy::RuntimeOption& option) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
void InitAndInfer(const std::string &model_dir, const std::string &image_file,
|
||||
const fastdeploy::RuntimeOption &option) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
@@ -38,20 +40,14 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file,
|
||||
return;
|
||||
}
|
||||
|
||||
// print res
|
||||
std::cout << res.Str() << std::endl;
|
||||
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 4) {
|
||||
std::cout << "Usage: infer_demo path/to/quant_model "
|
||||
"path/to/image "
|
||||
"run_option, "
|
||||
"e.g ./infer_demo ./ResNet50_vd_quant ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run on cpu with ORT "
|
||||
"backend; 1: run "
|
||||
"on gpu with TensorRT backend. "
|
||||
std::cout << "Usage: infer_demo path/to/model path/to/image run_option, "
|
||||
"e.g ./infer_demo ./ResNet50_vd ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
@@ -61,19 +57,32 @@ int main(int argc, char* argv[]) {
|
||||
|
||||
if (flag == 0) {
|
||||
option.UseCpu();
|
||||
option.UseOrtBackend();
|
||||
option.UsePaddleBackend(); // Paddle Inference
|
||||
} else if (flag == 1) {
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend();
|
||||
option.SetTrtInputShape("inputs",{1, 3, 224, 224});
|
||||
option.UseCpu();
|
||||
option.UseOpenVINOBackend(); // OpenVINO
|
||||
} else if (flag == 2) {
|
||||
option.UseCpu();
|
||||
option.UseOrtBackend(); // ONNX Runtime
|
||||
} else if (flag == 3) {
|
||||
option.UseCpu();
|
||||
option.UseLiteBackend(); // Paddle Lite
|
||||
} else if (flag == 4) {
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend();
|
||||
option.EnablePaddleToTrt();
|
||||
option.UsePaddleBackend(); // Paddle Inference
|
||||
} else if (flag == 5) {
|
||||
option.UseGpu();
|
||||
option.UsePaddleInferBackend();
|
||||
option.paddle_infer_option.enable_trt = true;
|
||||
} else if (flag == 6) {
|
||||
option.UseGpu();
|
||||
option.UseOrtBackend(); // ONNX Runtime
|
||||
} else if (flag == 7) {
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend(); // TensorRT
|
||||
}
|
||||
|
||||
std::string model_dir = argv[1];
|
||||
std::string test_image = argv[2];
|
||||
InitAndInfer(model_dir, test_image, option);
|
||||
return 0;
|
||||
std::string image_dir = argv[2];
|
||||
InitAndInfer(model_dir, image_dir, option);
|
||||
}
|
||||
121
examples/vision/classification/paddleclas/cpu-gpu/csharp/README.md
Executable file
121
examples/vision/classification/paddleclas/cpu-gpu/csharp/README.md
Executable file
@@ -0,0 +1,121 @@
|
||||
# PaddleClas CPU-GPU C#部署示例
|
||||
|
||||
本目录下提供`infer.cs`来调用C# API快速完成PaddleClas模型在CPU/GPU上部署的示例。
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署图像分类模型.
|
||||
|
||||
## 2. 部署环境准备
|
||||
在部署前，需确认软硬件环境，同时下载预编译部署库，参考[FastDeploy安装文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#FastDeploy预编译库安装)安装FastDeploy预编译库. 在本目录执行如下命令即可在Windows完成编译测试，支持此模型需保证FastDeploy版本1.0.4以上(x.x.x>=1.0.4)
|
||||
|
||||
## 3. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 4. 部署示例
|
||||
|
||||
### 4.1 下载C#包管理程序nuget客户端
|
||||
> https://dist.nuget.org/win-x86-commandline/v6.4.0/nuget.exe
|
||||
|
||||
下载完成后将该程序添加到环境变量**PATH**中
|
||||
|
||||
### 4.2 下载模型文件和测试图片
|
||||
> https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz # (下载后解压缩)
|
||||
> https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
### 4.3 编译示例代码
|
||||
|
||||
本文档编译的示例代码的编译工具依赖VS 2019，**Windows打开x64 Native Tools Command Prompt for VS 2019命令工具**，通过如下命令开始编译
|
||||
|
||||
```shell
|
||||
## 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy\examples\vision\classification\paddleclas\cpu-gpu\csharp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd D:\PaddleClas\deploy\fastdeploy\cpu-gpu\csharp
|
||||
|
||||
mkdir build && cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -DFASTDEPLOY_INSTALL_DIR=D:\fastdeploy-win-x64-gpu-x.x.x -DCUDA_DIRECTORY="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2"
|
||||
|
||||
nuget restore
|
||||
msbuild infer_demo.sln /m:4 /p:Configuration=Release /p:Platform=x64
|
||||
```
|
||||
|
||||
关于使用Visual Studio 2019创建sln工程，或者CMake工程等方式编译的更详细信息，可参考如下文档
|
||||
- [在 Windows 使用 FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq/use_sdk_on_windows.md)
|
||||
- [FastDeploy C++库在Windows上的多种使用方式](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq/use_sdk_on_windows_build.md)
|
||||
|
||||
## 4.4 运行可执行程序
|
||||
|
||||
注意Windows上运行时，需要将FastDeploy依赖的库拷贝至可执行程序所在目录, 或者配置环境变量。FastDeploy提供了工具帮助我们快速将所有依赖库拷贝至可执行程序所在目录,通过如下命令将所有依赖的dll文件拷贝至可执行程序所在的目录(可能生成的可执行文件在Release下还有一层目录，这里假设生成的可执行文件在Release处)
|
||||
```shell
|
||||
cd D:\Download\fastdeploy-win-x64-gpu-x.x.x
|
||||
|
||||
fastdeploy_init.bat install %cd% D:\PaddleClas\deploy\fastdeploy\cpu-gpu\csharp\build\Release
|
||||
```
|
||||
|
||||
将dll拷贝到当前路径后，准备好模型和图片，使用如下命令运行可执行程序即可
|
||||
```shell
|
||||
cd Release
|
||||
# CPU推理
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU推理
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
```
|
||||
|
||||
## 5. PaddleClas C#接口简介
|
||||
|
||||
下面提供了PaddleClas的C# API简介
|
||||
|
||||
- 如果用户想要更换部署后端或进行其他定制化操作, 请查看[C# Runtime API](https://github.com/PaddlePaddle/FastDeploy/blob/develop/csharp/fastdeploy/runtime_option.cs).
|
||||
- 更多 PaddleClas C# API 请查看 [C# PaddleClas API](https://github.com/PaddlePaddle/FastDeploy/blob/develop/csharp/fastdeploy/vision/classification/ppcls/model.cs)
|
||||
|
||||
### 模型
|
||||
|
||||
```c#
|
||||
fastdeploy.vision.classification.PaddleClasModel(
|
||||
string model_file,
|
||||
string params_file,
|
||||
string config_file,
|
||||
fastdeploy.RuntimeOption runtime_option = null,
|
||||
fastdeploy.ModelFormat model_format = ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
> PaddleClasModel模型加载和初始化。
|
||||
|
||||
> **参数**
|
||||
|
||||
>> * **model_file**(str): 模型文件路径
|
||||
>> * **params_file**(str): 参数文件路径
|
||||
>> * **config_file**(str): 配置文件路径，即PaddleClas导出的部署yaml文件
|
||||
>> * **runtime_option**(RuntimeOption): 后端推理配置，默认为null，即采用默认配置
|
||||
>> * **model_format**(ModelFormat): 模型格式，默认为PADDLE格式
|
||||
|
||||
### Predict函数
|
||||
|
||||
```c#
|
||||
fastdeploy.ClassifyResult Predict(OpenCvSharp.Mat im)
|
||||
```
|
||||
|
||||
> 模型预测接口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
>> * **im**(Mat): 输入图像，注意需为HWC，BGR格式
|
||||
>>
|
||||
> **返回值**
|
||||
>
|
||||
>> * **result**: 分类结果，包括label_id，以及相应的置信度, ClassifyResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
|
||||
|
||||
## 6. 其它文档
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas Python部署](../python)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
- [PaddleClas C 部署](../c)
|
||||
93
examples/vision/classification/paddleclas/cpu-gpu/python/README.md
Executable file
93
examples/vision/classification/paddleclas/cpu-gpu/python/README.md
Executable file
@@ -0,0 +1,93 @@
|
||||
# PaddleClas CPU-GPU Python部署示例
|
||||
本目录下提供`infer.py`快速完成PaddleClas在CPU/GPU上部署的示例.
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署图像分类模型
|
||||
|
||||
## 2. 部署环境准备
|
||||
在部署前，需确认软硬件环境，同时下载预编译部署库，参考[FastDeploy安装文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#FastDeploy预编译库安装)安装FastDeploy预编译库.
|
||||
|
||||
## 3. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 4. 运行部署示例
|
||||
```bash
|
||||
# 安装FastDpeloy python包（详细文档请参考`部署环境准备`）
|
||||
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
|
||||
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
|
||||
|
||||
# 下载部署示例代码
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/cpu-gpu/python
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/cpu-gpu/python
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 在CPU上使用Paddle Inference推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --backend paddle --topk 1
|
||||
# 在CPU上使用OenVINO推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --backend openvino --topk 1
|
||||
# 在CPU上使用ONNX Runtime推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --backend ort --topk 1
|
||||
# 在CPU上使用Paddle Lite推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --backend pplite --topk 1
|
||||
# 在GPU上使用Paddle Inference推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --backend paddle --topk 1
|
||||
# 在GPU上使用Paddle TensorRT推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --backend pptrt --topk 1
|
||||
# 在GPU上使用ONNX Runtime推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --backend ort --topk 1
|
||||
# 在GPU上使用Nvidia TensorRT推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --backend trt --topk 1
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 5. 部署示例选项说明
|
||||
|
||||
|参数|含义|默认值
|
||||
|---|---|---|
|
||||
|--model|指定模型文件夹所在的路径|None|
|
||||
|--image|指定测试图片所在的路径|None|
|
||||
|--device|指定即将运行的硬件类型，支持的值为`[cpu, gpu]`，当设置为cpu时，可运行在x86 cpu/arm cpu等cpu上|cpu|
|
||||
|--device_id|使用gpu时, 指定设备号|0|
|
||||
|--backend|部署模型时使用的后端, 支持的值为`[paddle,pptrt,pplite,ort,openvino,trt]` |openvino|
|
||||
|--topk|返回的前topk准确率, 支持的为`1,5` |1|
|
||||
|
||||
关于如何通过FastDeploy使用更多不同的推理后端，以及如何使用不同的硬件，请参考文档：[如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
|
||||
## 6. 更多指南
|
||||
- [PaddleClas系列 Python API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/image_classification.html)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
- [PaddleClas C 部署](../c)
|
||||
- [PaddleClas C# 部署](../csharp)
|
||||
|
||||
## 7. 常见问题
|
||||
- PaddleClas能在FastDeploy支持的多种后端上推理,支持情况如下表所示, 如何切换后端, 详见文档[如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
|
||||
|硬件类型|支持的后端|
|
||||
|:---:|:---:|
|
||||
|X86 CPU| Paddle Inference, ONNX Runtime, OpenVINO |
|
||||
|ARM CPU| Paddle Lite |
|
||||
|飞腾 CPU| ONNX Runtime |
|
||||
|NVIDIA GPU| Paddle Inference, ONNX Runtime, TensorRT |
|
||||
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
- [编译Jetson部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/jetson.md)
|
||||
@@ -1,7 +1,6 @@
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import os
|
||||
import time
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
@@ -9,19 +8,16 @@ def parse_arguments():
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--model", required=True, help="Path of paddleclas model.")
|
||||
"--model", required=True, help="Path of PaddleClas model.")
|
||||
parser.add_argument(
|
||||
"--image", required=True, help="Path of test image file.")
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--topk", type=int, default=1, help="Return topk results.")
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
default='cpu',
|
||||
help="Type of inference device, support 'cpu' or 'gpu'.")
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
type=str,
|
||||
default="default",
|
||||
help="Type of inference backend, support ort/trt/paddle/openvino, default 'openvino' for cpu, 'tensorrt' for gpu"
|
||||
help="Type of inference device, support 'cpu' or 'gpu' or 'ipu' or 'kunlunxin' or 'ascend' ."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--device_id",
|
||||
@@ -29,38 +25,48 @@ def parse_arguments():
|
||||
default=0,
|
||||
help="Define which GPU card used to run model.")
|
||||
parser.add_argument(
|
||||
"--cpu_thread_num",
|
||||
type=int,
|
||||
default=9,
|
||||
help="Number of threads while inference on CPU.")
|
||||
"--backend",
|
||||
type=str,
|
||||
default="default",
|
||||
help="Type of inference backend, support ort/trt/paddle/openvino, default 'openvino' for cpu, 'tensorrt' for gpu"
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
option = fd.RuntimeOption()
|
||||
if args.device.lower() == "gpu":
|
||||
option.use_gpu(0)
|
||||
|
||||
option.set_cpu_thread_num(args.cpu_thread_num)
|
||||
option = fd.RuntimeOption()
|
||||
|
||||
if args.device.lower() == "gpu":
|
||||
option.use_gpu(args.device_id)
|
||||
|
||||
if args.backend.lower() == "trt":
|
||||
assert args.device.lower(
|
||||
) == "gpu", "TensorRT backend require inferences on device GPU."
|
||||
option.use_trt_backend()
|
||||
option.set_trt_input_shape("inputs", min_shape=[1, 3, 224, 224])
|
||||
elif args.backend.lower() == "pptrt":
|
||||
assert args.device.lower(
|
||||
) == "gpu", "TensorRT backend require inference on device GPU."
|
||||
option.use_trt_backend()
|
||||
option.enable_paddle_to_trt()
|
||||
|
||||
elif args.backend.lower() == "pptrt":
|
||||
assert args.device.lower(
|
||||
) == "gpu", "Paddle-TensorRT backend require inference on device GPU."
|
||||
option.use_paddle_infer_backend()
|
||||
option.paddle_infer_option.enable_trt = True
|
||||
|
||||
elif args.backend.lower() == "ort":
|
||||
option.use_ort_backend()
|
||||
|
||||
elif args.backend.lower() == "paddle":
|
||||
option.use_paddle_infer_backend()
|
||||
|
||||
elif args.backend.lower() == "openvino":
|
||||
assert args.device.lower(
|
||||
) == "cpu", "OpenVINO backend require inference on device CPU."
|
||||
option.use_openvino_backend()
|
||||
|
||||
elif args.backend.lower() == "pplite":
|
||||
assert args.device.lower(
|
||||
) == "cpu", "Paddle Lite backend require inference on device CPU."
|
||||
option.use_lite_backend()
|
||||
|
||||
return option
|
||||
|
||||
|
||||
@@ -72,11 +78,10 @@ runtime_option = build_option(args)
|
||||
model_file = os.path.join(args.model, "inference.pdmodel")
|
||||
params_file = os.path.join(args.model, "inference.pdiparams")
|
||||
config_file = os.path.join(args.model, "inference_cls.yaml")
|
||||
|
||||
model = fd.vision.classification.PaddleClasModel(
|
||||
model_file, params_file, config_file, runtime_option=runtime_option)
|
||||
|
||||
# 预测图片检测结果
|
||||
# 预测图片分类结果
|
||||
im = cv2.imread(args.image)
|
||||
result = model.predict(im)
|
||||
result = model.predict(im, args.topk)
|
||||
print(result)
|
||||
@@ -1,99 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas C# Deployment Example
|
||||
|
||||
This directory provides example `infer.cs` to fastly finish the deployment of PaddleClas models on CPU/GPU.
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Download the precompiled deployment library and samples code according to your development environment. Refer to [FastDeploy Precompiled Library](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
Please follow below instructions to compile and test in Windows. FastDeploy version 1.0.4 or above (x.x.x>=1.0.4) is required to support this model.
|
||||
|
||||
## 1. Download C# package management tool nuget client
|
||||
> https://dist.nuget.org/win-x86-commandline/v6.4.0/nuget.exe
|
||||
|
||||
Add nuget program into system variable **PATH**
|
||||
|
||||
## 2. Download model and image for test
|
||||
> https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz # (下载后解压缩)
|
||||
> https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
## 3. Compile example code
|
||||
|
||||
Open `x64 Native Tools Command Prompt for VS 2019` command tool on Windows, cd to the demo path of ppyoloe and execute commands
|
||||
|
||||
```shell
|
||||
cd D:\Download\fastdeploy-win-x64-gpu-x.x.x\examples\vision\classification\paddleclas\csharp
|
||||
|
||||
mkdir build && cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -DFASTDEPLOY_INSTALL_DIR=D:\Download\fastdeploy-win-x64-gpu-x.x.x -DCUDA_DIRECTORY="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2"
|
||||
|
||||
nuget restore
|
||||
msbuild infer_demo.sln /m:4 /p:Configuration=Release /p:Platform=x64
|
||||
```
|
||||
|
||||
For more information about how to use FastDeploy SDK to compile a project with Visual Studio 2019. Please refer to
|
||||
- [Using the FastDeploy C++ SDK on Windows Platform](../../../../../docs/en/faq/use_sdk_on_windows.md)
|
||||
|
||||
## 4. Execute compiled program
|
||||
fastdeploy.dll and related dynamic libraries are required by the program. FastDeploy provide a script to copy all required dll to your program path.
|
||||
|
||||
```shell
|
||||
cd D:\Download\fastdeploy-win-x64-gpu-x.x.x
|
||||
|
||||
fastdeploy_init.bat install %cd% D:\Download\fastdeploy-win-x64-gpu-x.x.x\examples\vision\classification\paddleclas\csharp\build\Release
|
||||
```
|
||||
Then you can run your program and test the model with image
|
||||
|
||||
```shell
|
||||
cd Release
|
||||
# CPU inference
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU inference
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
```
|
||||
|
||||
## PaddleClas C# Interface
|
||||
|
||||
### Model Class
|
||||
|
||||
```c#
|
||||
fastdeploy.vision.classification.PaddleClasModel(
|
||||
string model_file,
|
||||
string params_file,
|
||||
string config_file
|
||||
fastdeploy.RuntimeOption runtime_option = null,
|
||||
fastdeploy.ModelFormat model_format = ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
> PaddleClasModel initilization.
|
||||
|
||||
> **Params**
|
||||
|
||||
>> * **model_file**(str): Model file path
|
||||
>> * **params_file**(str): Parameter file path
|
||||
>> * **config_file**(str): Configuration file path, which is the deployment yaml file exported by PaddleClas
|
||||
>> * **runtime_option**(RuntimeOption): Backend inference configuration. null by default, which is the default configuration
|
||||
>> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
#### Predict Function
|
||||
|
||||
```c#
|
||||
fastdeploy.ClassifyResult Predict(OpenCvSharp.Mat im)
|
||||
```
|
||||
|
||||
> Model prediction interface. Input images and output results directly.
|
||||
>
|
||||
> **Params**
|
||||
>
|
||||
>> * **im**(Mat): Input images in HWC or BGR format
|
||||
>
|
||||
> **Return**
|
||||
>
|
||||
>> * **result**(ClassifyResult): The classification result, including label_id, and the corresponding confidence. Refer to [Visual Model Prediction Results](../../../../../docs/api/vision_results/) for the description of ClassifyResult
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Python Deployment](../python)
|
||||
- [Vision Model prediction results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/en/faq/how_to_change_backend.md)
|
||||
@@ -1,101 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas C#部署示例
|
||||
|
||||
本目录下提供`infer.cs`来调用C# API快速完成PaddleClas系列模型在CPU/GPU上部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
在本目录执行如下命令即可在Windows完成编译测试，支持此模型需保证FastDeploy版本1.0.4以上(x.x.x>=1.0.4)
|
||||
|
||||
## 1. 下载C#包管理程序nuget客户端
|
||||
> https://dist.nuget.org/win-x86-commandline/v6.4.0/nuget.exe
|
||||
|
||||
下载完成后将该程序添加到环境变量**PATH**中
|
||||
|
||||
## 2. 下载模型文件和测试图片
|
||||
> https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz # (下载后解压缩)
|
||||
> https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
## 3. 编译示例代码
|
||||
|
||||
本文档编译的示例代码可在解压的库中找到，编译工具依赖VS 2019的安装，**Windows打开x64 Native Tools Command Prompt for VS 2019命令工具**，通过如下命令开始编译
|
||||
|
||||
```shell
|
||||
cd D:\Download\fastdeploy-win-x64-gpu-x.x.x\examples\vision\classification\paddleclas\csharp
|
||||
|
||||
mkdir build && cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -DFASTDEPLOY_INSTALL_DIR=D:\Download\fastdeploy-win-x64-gpu-x.x.x -DCUDA_DIRECTORY="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2"
|
||||
|
||||
nuget restore
|
||||
msbuild infer_demo.sln /m:4 /p:Configuration=Release /p:Platform=x64
|
||||
```
|
||||
|
||||
关于使用Visual Studio 2019创建sln工程，或者CMake工程等方式编译的更详细信息，可参考如下文档
|
||||
- [在 Windows 使用 FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
- [FastDeploy C++库在Windows上的多种使用方式](../../../../../docs/cn/faq/use_sdk_on_windows_build.md)
|
||||
|
||||
## 4. 运行可执行程序
|
||||
|
||||
注意Windows上运行时，需要将FastDeploy依赖的库拷贝至可执行程序所在目录, 或者配置环境变量。FastDeploy提供了工具帮助我们快速将所有依赖库拷贝至可执行程序所在目录,通过如下命令将所有依赖的dll文件拷贝至可执行程序所在的目录(可能生成的可执行文件在Release下还有一层目录，这里假设生成的可执行文件在Release处)
|
||||
```shell
|
||||
cd D:\Download\fastdeploy-win-x64-gpu-x.x.x
|
||||
|
||||
fastdeploy_init.bat install %cd% D:\Download\fastdeploy-win-x64-gpu-x.x.x\examples\vision\classification\paddleclas\csharp\build\Release
|
||||
```
|
||||
|
||||
将dll拷贝到当前路径后，准备好模型和图片，使用如下命令运行可执行程序即可
|
||||
```shell
|
||||
cd Release
|
||||
# CPU推理
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 0
|
||||
# GPU推理
|
||||
infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg 1
|
||||
```
|
||||
|
||||
## PaddleClas C#接口
|
||||
|
||||
### 模型
|
||||
|
||||
```c#
|
||||
fastdeploy.vision.classification.PaddleClasModel(
|
||||
string model_file,
|
||||
string params_file,
|
||||
string config_file,
|
||||
fastdeploy.RuntimeOption runtime_option = null,
|
||||
fastdeploy.ModelFormat model_format = ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
> PaddleClasModel模型加载和初始化。
|
||||
|
||||
> **参数**
|
||||
|
||||
>> * **model_file**(str): 模型文件路径
|
||||
>> * **params_file**(str): 参数文件路径
|
||||
>> * **config_file**(str): 配置文件路径，即PaddleClas导出的部署yaml文件
|
||||
>> * **runtime_option**(RuntimeOption): 后端推理配置，默认为null，即采用默认配置
|
||||
>> * **model_format**(ModelFormat): 模型格式，默认为PADDLE格式
|
||||
|
||||
#### Predict函数
|
||||
|
||||
```c#
|
||||
fastdeploy.ClassifyResult Predict(OpenCvSharp.Mat im)
|
||||
```
|
||||
|
||||
> 模型预测接口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
>> * **im**(Mat): 输入图像，注意需为HWC，BGR格式
|
||||
>>
|
||||
> **返回值**
|
||||
>
|
||||
>> * **result**: 分类结果，包括label_id，以及相应的置信度, ClassifyResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
|
||||
|
||||
- [模型介绍](../../)
|
||||
- [Python部署](../python)
|
||||
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
@@ -0,0 +1,71 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# PaddleClas 模型在Graphcore IPU的部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在Graphcore IPU上部署图像分类模型.
|
||||
|
||||
## 2. 模型版本说明
|
||||
|
||||
- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [PP-LCNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNet.md)
|
||||
- [PP-LCNetV2系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- [EfficientNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md)
|
||||
- [GhostNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [MobileNet系列模型(包含v1,v2,v3)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [ShuffleNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [SqueezeNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Others.md)
|
||||
- [Inception系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Inception.md)
|
||||
- [PP-HGNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md)
|
||||
- [ResNet系列模型（包含vd系列）](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
### 2.1 准备PaddleClas部署模型
|
||||
|
||||
PaddleClas模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
注意：PaddleClas导出的模型仅包含`inference.pdmodel`和`inference.pdiparams`两个文件，但为了满足部署的需求，同时也需准备其提供的通用[inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml)文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息，开发者可直接下载此文件使用。但需根据自己的需求修改yaml文件中的配置参数，具体可比照PaddleClas模型训练[config](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet)中的infer部分的配置信息进行修改。
|
||||
|
||||
|
||||
### 2.2 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleClas导出的部分模型（含inference_cls.yaml文件），开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | Top1 | Top5 |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- |
|
||||
| [PPLCNet_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNet_x1_0_infer.tgz) | 12MB | 224x224 |71.32% | 90.03% |
|
||||
| [PPLCNetV2_base](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNetV2_base_infer.tgz) | 26MB | 224x224 |77.04% | 93.27% |
|
||||
| [EfficientNetB7](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB7_infer.tgz) | 255MB | 600x600 | 84.3% | 96.9% |
|
||||
| [EfficientNetB0](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_infer.tgz)| 19MB | 224x224 | 77.38% | 93.31% |
|
||||
| [EfficientNetB0_small](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_small_infer.tgz)| 18MB | 224x224 | 75.8% | 92.58% |
|
||||
| [GhostNet_x1_3](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_infer.tgz) | 27MB | 224x224 | 75.79% | 92.54% |
|
||||
| [GhostNet_x1_3_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_ssld_infer.tgz) | 29MB | 224x224 | 79.3% | 94.49% |
|
||||
| [GhostNet_x0_5](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x0_5_infer.tgz) | 10MB | 224x224 | 66.8% | 86.9% |
|
||||
| [MobileNetV1_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_x0_25_infer.tgz) | 1.9MB | 224x224 | 51.4% | 75.5% |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_ssld_infer.tgz) | 17MB | 224x224 | 77.9% | 93.9% |
|
||||
| [MobileNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_x0_25_infer.tgz) | 5.9MB | 224x224 | 53.2% | 76.5% |
|
||||
| [MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_infer.tgz) | 13MB | 224x224 | 72.15% | 90.65% |
|
||||
| [MobileNetV2_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_ssld_infer.tgz) | 14MB | 224x224 | 76.74% | 93.39% |
|
||||
| [MobileNetV3_small_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x1_0_infer.tgz) | 11MB | 224x224 | 68.24% | 88.06% |
|
||||
| [MobileNetV3_small_x0_35_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x0_35_ssld_infer.tgz) | 6.4MB | 224x224 | 55.55% | 77.71% |
|
||||
| [MobileNetV3_large_x1_0_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_large_x1_0_ssld_infer.tgz) | 22MB | 224x224 | 78.96% | 94.48% |
|
||||
| [ShuffleNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x0_25_infer.tgz) | 2.4MB | 224x224 | 49.9% | 73.79% |
|
||||
| [ShuffleNetV2_x2_0](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x2_0_infer.tgz) | 29MB | 224x224 | 73.15% | 91.2% |
|
||||
| [SqueezeNet1_1](https://bj.bcebos.com/paddlehub/fastdeploy/SqueezeNet1_1_infer.tgz) | 4.8MB | 224x224 | 60.1% | 81.9% |
|
||||
| [InceptionV3](https://bj.bcebos.com/paddlehub/fastdeploy/InceptionV3_infer.tgz) | 92MB | 299x299 | 79.14% | 94.59% |
|
||||
| [PPHGNet_tiny_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_tiny_ssld_infer.tgz) | 57MB | 224x224 | 81.95% | 96.12% |
|
||||
| [PPHGNet_small](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_small_infer.tgz) | 87MB | 224x224 | 81.51% | 95.82% |
|
||||
| [PPHGNet_base_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_base_ssld_infer.tgz) | 274MB | 224x224 | 85.0% | 97.35% |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) | 98MB | 224x224 | 79.12% | 94.44% |
|
||||
| [ResNet50](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_infer.tgz) | 91MB | 224x224 | 76.5% | 93% |
|
||||
| [ResNeXt50_32x4d](https://bj.bcebos.com/paddlehub/fastdeploy/ResNeXt50_32x4d_infer.tgz) | 89MB | 224x224 | 77.75% | 93.82% |
|
||||
| [DenseNet121](https://bj.bcebos.com/paddlehub/fastdeploy/DenseNet121_infer.tgz) | 29MB | 224x224 | 75.66% | 92.58% |
|
||||
| [PULC_person_exists](https://bj.bcebos.com/paddlehub/fastdeploy/person_exists_infer.tgz) | 6MB | 224x224 | | |
|
||||
| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%|
|
||||
|
||||
|
||||
## 3. 详细部署的部署示例
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -0,0 +1,14 @@
|
||||
PROJECT(infer_demo C CXX)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||
|
||||
# 指定下载解压后的fastdeploy库路径
|
||||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||
|
||||
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
|
||||
|
||||
# 添加FastDeploy依赖头文件
|
||||
include_directories(${FASTDEPLOY_INCS})
|
||||
|
||||
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
|
||||
# 添加FastDeploy库依赖
|
||||
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
|
||||
@@ -0,0 +1,51 @@
|
||||
# PaddleClas Graphcore IPU C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`, 供用户完成PaddleClas模型在Graphcore IPU上的部署.
|
||||
|
||||
## 1. 部署环境准备
|
||||
- 在部署前，需自行编译基于Graphcore IPU的预测库，参考文档[Graphcore IPU部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/graphcore/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/graphcore/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 使用编译完成的FastDeploy库编译infer_demo
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-graphcore
|
||||
make -j
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 使用IPU部署
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
```
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 C++ API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas Python部署](../python)
|
||||
@@ -0,0 +1,56 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void IPUInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseIPU();
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(&im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/model path/to/image " << std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
std::string model_dir = argv[1];
|
||||
std::string test_image = argv[2];
|
||||
IPUInfer(model_dir, test_image);
|
||||
return 0;
|
||||
}
|
||||
44
examples/vision/classification/paddleclas/graphcore/python/README.md
Executable file
44
examples/vision/classification/paddleclas/graphcore/python/README.md
Executable file
@@ -0,0 +1,44 @@
|
||||
# PaddleClas Graphcore IPU Python部署示例
|
||||
本目录下提供`infer.py`快速完成PaddleClas在Graphcore IPU上部署的示例.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需自行编译基于Graphcore IPU的FastDeploy python wheel包并安装，参考文档，参考文档[Graphcore IPU部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
```bash
|
||||
# 安装FastDpeloy Graphcore IPU预测库 python包（详细文档请参考`部署环境准备`）
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/graphcore/python
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/graphcore/python
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 在Graphcore AI 处理器上推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --topk 1
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 Python API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/image_classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
@@ -0,0 +1,41 @@
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import os
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
import argparse
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--model", required=True, help="Path of PaddleClas model.")
|
||||
parser.add_argument(
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--topk", type=int, default=1, help="Return topk results.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
|
||||
option = fd.RuntimeOption()
|
||||
option.use_ipu()
|
||||
|
||||
return option
|
||||
|
||||
|
||||
args = parse_arguments()
|
||||
|
||||
# 配置runtime，加载模型
|
||||
runtime_option = build_option(args)
|
||||
|
||||
model_file = os.path.join(args.model, "inference.pdmodel")
|
||||
params_file = os.path.join(args.model, "inference.pdiparams")
|
||||
config_file = os.path.join(args.model, "inference_cls.yaml")
|
||||
model = fd.vision.classification.PaddleClasModel(
|
||||
model_file, params_file, config_file, runtime_option=runtime_option)
|
||||
|
||||
# 预测图片分类结果
|
||||
im = cv2.imread(args.image)
|
||||
result = model.predict(im, args.topk)
|
||||
print(result)
|
||||
@@ -0,0 +1,69 @@
|
||||
# PaddleClas 模型在昆仑芯上的部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在昆仑芯上部署图像分类模型.
|
||||
|
||||
## 2. 模型版本说明
|
||||
|
||||
- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [PP-LCNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNet.md)
|
||||
- [PP-LCNetV2系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- [EfficientNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md)
|
||||
- [GhostNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [MobileNet系列模型(包含v1,v2,v3)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [ShuffleNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md)
|
||||
- [SqueezeNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Others.md)
|
||||
- [Inception系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Inception.md)
|
||||
- [PP-HGNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md)
|
||||
- [ResNet系列模型（包含vd系列）](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
### 2.1 准备PaddleClas部署模型
|
||||
|
||||
PaddleClas模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
注意：PaddleClas导出的模型仅包含`inference.pdmodel`和`inference.pdiparams`两个文件，但为了满足部署的需求，同时也需准备其提供的通用[inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml)文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息，开发者可直接下载此文件使用。但需根据自己的需求修改yaml文件中的配置参数，具体可比照PaddleClas模型训练[config](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet)中的infer部分的配置信息进行修改。
|
||||
|
||||
|
||||
### 2.2 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleClas导出的部分模型（含inference_cls.yaml文件），开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | Top1 | Top5 |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- |
|
||||
| [PPLCNet_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNet_x1_0_infer.tgz) | 12MB | 224x224 |71.32% | 90.03% |
|
||||
| [PPLCNetV2_base](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNetV2_base_infer.tgz) | 26MB | 224x224 |77.04% | 93.27% |
|
||||
| [EfficientNetB7](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB7_infer.tgz) | 255MB | 600x600 | 84.3% | 96.9% |
|
||||
| [EfficientNetB0](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_infer.tgz)| 19MB | 224x224 | 77.38% | 93.31% |
|
||||
| [EfficientNetB0_small](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_small_infer.tgz)| 18MB | 224x224 | 75.8% | 92.58% |
|
||||
| [GhostNet_x1_3](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_infer.tgz) | 27MB | 224x224 | 75.79% | 92.54% |
|
||||
| [GhostNet_x1_3_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_ssld_infer.tgz) | 29MB | 224x224 | 79.3% | 94.49% |
|
||||
| [GhostNet_x0_5](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x0_5_infer.tgz) | 10MB | 224x224 | 66.8% | 86.9% |
|
||||
| [MobileNetV1_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_x0_25_infer.tgz) | 1.9MB | 224x224 | 51.4% | 75.5% |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_ssld_infer.tgz) | 17MB | 224x224 | 77.9% | 93.9% |
|
||||
| [MobileNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_x0_25_infer.tgz) | 5.9MB | 224x224 | 53.2% | 76.5% |
|
||||
| [MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_infer.tgz) | 13MB | 224x224 | 72.15% | 90.65% |
|
||||
| [MobileNetV2_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_ssld_infer.tgz) | 14MB | 224x224 | 76.74% | 93.39% |
|
||||
| [MobileNetV3_small_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x1_0_infer.tgz) | 11MB | 224x224 | 68.24% | 88.06% |
|
||||
| [MobileNetV3_small_x0_35_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x0_35_ssld_infer.tgz) | 6.4MB | 224x224 | 55.55% | 77.71% |
|
||||
| [MobileNetV3_large_x1_0_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_large_x1_0_ssld_infer.tgz) | 22MB | 224x224 | 78.96% | 94.48% |
|
||||
| [ShuffleNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x0_25_infer.tgz) | 2.4MB | 224x224 | 49.9% | 73.79% |
|
||||
| [ShuffleNetV2_x2_0](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x2_0_infer.tgz) | 29MB | 224x224 | 73.15% | 91.2% |
|
||||
| [SqueezeNet1_1](https://bj.bcebos.com/paddlehub/fastdeploy/SqueezeNet1_1_infer.tgz) | 4.8MB | 224x224 | 60.1% | 81.9% |
|
||||
| [InceptionV3](https://bj.bcebos.com/paddlehub/fastdeploy/InceptionV3_infer.tgz) | 92MB | 299x299 | 79.14% | 94.59% |
|
||||
| [PPHGNet_tiny_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_tiny_ssld_infer.tgz) | 57MB | 224x224 | 81.95% | 96.12% |
|
||||
| [PPHGNet_small](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_small_infer.tgz) | 87MB | 224x224 | 81.51% | 95.82% |
|
||||
| [PPHGNet_base_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_base_ssld_infer.tgz) | 274MB | 224x224 | 85.0% | 97.35% |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) | 98MB | 224x224 | 79.12% | 94.44% |
|
||||
| [ResNet50](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_infer.tgz) | 91MB | 224x224 | 76.5% | 93% |
|
||||
| [ResNeXt50_32x4d](https://bj.bcebos.com/paddlehub/fastdeploy/ResNeXt50_32x4d_infer.tgz) | 89MB | 224x224 | 77.75% | 93.82% |
|
||||
| [DenseNet121](https://bj.bcebos.com/paddlehub/fastdeploy/DenseNet121_infer.tgz) | 29MB | 224x224 | 75.66% | 92.58% |
|
||||
| [PULC_person_exists](https://bj.bcebos.com/paddlehub/fastdeploy/person_exists_infer.tgz) | 6MB | 224x224 | | |
|
||||
| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%|
|
||||
|
||||
|
||||
## 3. 详细部署的部署示例
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -0,0 +1,14 @@
|
||||
PROJECT(infer_demo C CXX)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||
|
||||
# 指定下载解压后的fastdeploy库路径
|
||||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||
|
||||
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
|
||||
|
||||
# 添加FastDeploy依赖头文件
|
||||
include_directories(${FASTDEPLOY_INCS})
|
||||
|
||||
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
|
||||
# 添加FastDeploy库依赖
|
||||
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
|
||||
@@ -0,0 +1,51 @@
|
||||
# PaddleClas 昆仑芯XPU C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`, 供用户完成PaddleClas模型在昆仑芯XPU上的部署.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需自行编译基于昆仑芯XPUXPU的预测库，参考文档[昆仑芯XPU部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/kunlunxin/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/kunlunxin/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 使用编译完成的FastDeploy库编译infer_demo
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-kunlunxin
|
||||
make -j
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 使用昆仑芯XPU部署
|
||||
./infer_demo ResNet50_vd_infer ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
```
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 C++ API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas Python部署](../python)
|
||||
@@ -0,0 +1,56 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void KunlunInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseKunlunXin();
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::ClassifyResult res;
|
||||
if (!model.Predict(&im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/model path/to/image " << std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
std::string model_dir = argv[1];
|
||||
std::string test_image = argv[2];
|
||||
KunlunInfer(model_dir, test_image);
|
||||
return 0;
|
||||
}
|
||||
44
examples/vision/classification/paddleclas/kunlunxin/python/README.md
Executable file
44
examples/vision/classification/paddleclas/kunlunxin/python/README.md
Executable file
@@ -0,0 +1,44 @@
|
||||
# PaddleClas 昆仑芯XPU Python部署示例
|
||||
本目录下提供`infer.py`快速完成PaddleClas在昆仑芯XPU上部署的示例.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需自行编译基于昆仑芯XPU的FastDeploy python wheel包并安装，参考文档，参考文档[昆仑芯XPU部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以在[FastDeploy支持的PaddleClas模型列表](../README.md)中下载所需模型.
|
||||
|
||||
## 3. 运行部署示例
|
||||
```bash
|
||||
# 安装FastDpeloy 预测库 python包（详细文档请参考`部署环境准备`）
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/kunlunxin/python
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/kunlunxin/python
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 在昆仑芯XPU AI 处理器上推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --topk 1
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## 4. 更多指南
|
||||
- [PaddleClas系列 Python API查阅](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/image_classification.html)
|
||||
- [FastDeploy部署PaddleClas模型概览](../../)
|
||||
- [PaddleClas C++ 部署](../cpp)
|
||||
@@ -0,0 +1,41 @@
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import os
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
import argparse
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--model", required=True, help="Path of PaddleClas model.")
|
||||
parser.add_argument(
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--topk", type=int, default=1, help="Return topk results.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
|
||||
option = fd.RuntimeOption()
|
||||
option.use_kunlunxin()
|
||||
|
||||
return option
|
||||
|
||||
|
||||
args = parse_arguments()
|
||||
|
||||
# 配置runtime，加载模型
|
||||
runtime_option = build_option(args)
|
||||
|
||||
model_file = os.path.join(args.model, "inference.pdmodel")
|
||||
params_file = os.path.join(args.model, "inference.pdiparams")
|
||||
config_file = os.path.join(args.model, "inference_cls.yaml")
|
||||
model = fd.vision.classification.PaddleClasModel(
|
||||
model_file, params_file, config_file, runtime_option=runtime_option)
|
||||
|
||||
# 预测图片分类结果
|
||||
im = cv2.imread(args.image)
|
||||
result = model.predict(im, args.topk)
|
||||
print(result)
|
||||
@@ -1,80 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# Example of PaddleClas models Python Deployment
|
||||
|
||||
Before deployment, two steps require confirmation.
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
- 2. Install the FastDeploy Python whl package. Please refer to [FastDeploy Python Installation](../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
|
||||
This directory provides examples that `infer.py` fast finishes the deployment of ResNet50_vd on CPU/GPU and GPU accelerated by TensorRT. The script is as follows
|
||||
|
||||
```bash
|
||||
# Download deployment example code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/python
|
||||
|
||||
# Download the ResNet50_vd model file and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# CPU inference
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --topk 1
|
||||
# GPU inference
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --topk 1
|
||||
# Use TensorRT inference on GPU （Attention: It is somewhat time-consuming for the operation of model serialization when running TensorRT inference for the first time. Please be patient.）
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --use_trt True --topk 1
|
||||
# IPU inference（Attention: It is somewhat time-consuming for the operation of model serialization when running IPU inference for the first time. Please be patient.）
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device ipu --topk 1
|
||||
# XPU inference
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device xpu --topk 1
|
||||
```
|
||||
|
||||
The result returned after running is as follows
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## PaddleClasModel Python Interface
|
||||
|
||||
```python
|
||||
fd.vision.classification.PaddleClasModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleClas model loading and initialization, where model_file and params_file are the Paddle inference files exported from the training model. Refer to [Model Export](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA) for more information
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend Inference configuration. None by default. (use the default configuration)
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
### predict function
|
||||
|
||||
> ```python
|
||||
> PaddleClasModel.predict(input_image, topk=1)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output classification topk results directly.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): Input data in HWC or BGR format
|
||||
> > * **topk**(int): Return the topk classification results with the highest prediction probability. Default 1
|
||||
|
||||
> **Return**
|
||||
>
|
||||
> > Return `fastdeploy.vision.ClassifyResult` structure. Refer to [Visual Model Prediction Results](../../../../../docs/api/vision_results/) for the description of the structure.
|
||||
|
||||
|
||||
## Other documents
|
||||
|
||||
- [PaddleClas Model Description](..)
|
||||
- [PaddleClas C++ Deployment](../cpp)
|
||||
- [Model prediction results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/en/faq/how_to_change_backend.md)
|
||||
@@ -1,82 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas模型 Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
本目录下提供`infer.py`快速完成ResNet50_vd在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/python
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# CPU推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device cpu --topk 1
|
||||
# GPU推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --topk 1
|
||||
# GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待）
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device gpu --use_trt True --topk 1
|
||||
# IPU推理（注意：IPU推理首次运行会有序列化模型的操作，有一定耗时，需要耐心等待）
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device ipu --topk 1
|
||||
# 昆仑芯XPU推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device kunlunxin --topk 1
|
||||
# 华为昇腾NPU推理
|
||||
python infer.py --model ResNet50_vd_infer --image ILSVRC2012_val_00000010.jpeg --device ascend --topk 1
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示
|
||||
```bash
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.686229,
|
||||
)
|
||||
```
|
||||
|
||||
## PaddleClasModel Python接口
|
||||
|
||||
```python
|
||||
fd.vision.classification.PaddleClasModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleClas模型加载和初始化，其中model_file, params_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA)
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
### predict函数
|
||||
|
||||
> ```python
|
||||
> PaddleClasModel.predict(input_image, topk=1)
|
||||
> ```
|
||||
>
|
||||
> 模型预测结口，输入图像直接输出分类topk结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): 输入数据，注意需为HWC，BGR格式
|
||||
> > * **topk**(int):返回预测概率最高的topk个分类结果，默认为1
|
||||
|
||||
> **返回**
|
||||
>
|
||||
> > 返回`fastdeploy.vision.ClassifyResult`结构体，结构体说明参考文档[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
|
||||
|
||||
## 其它文档
|
||||
|
||||
- [PaddleClas 模型介绍](..)
|
||||
- [PaddleClas C++部署](../cpp)
|
||||
- [模型预测结果说明](../../../../../docs/api/vision_results/)
|
||||
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
@@ -1,51 +1,43 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Quantification Model Deployment
|
||||
FastDeploy supports the deployment of quantification models and provides a convenient tool for automatic model compression.
|
||||
Users can use it to deploy models after quantification or directly deploy quantized models provided by FastDeploy.
|
||||
|
||||
## FastDeploy one-click auto-compression tool
|
||||
FastDeploy provides a one-click auto-compression tool that allows users to quantize models by simply entering a configuration file.
|
||||
Refer to [one-click auto-compression tool](../../../../../tools/common_tools/auto_compression/) for details.
|
||||
Attention:The quantized classification model still requires the inference_cls.yaml file in the FP32 model folder. The model folder after personal quantification does not contain this yaml file. But users can copy this yaml file from the FP32 model folder to your quantized model folder.
|
||||
# PaddleClas 量化模型部署-FastDeploy
|
||||
|
||||
## Download the quantized PaddleClas model
|
||||
Users can also directly download the quantized models in the table below.
|
||||
FastDeploy已支持部署量化模型,并提供一键模型自动化压缩的工具.
|
||||
用户可以使用一键模型自动化压缩工具,自行对模型量化后部署, 也可以直接下载FastDeploy提供的量化模型进行部署.
|
||||
|
||||
Benchmark table description:
|
||||
- Runtime latency: model’s inference latency on multiple Runtimes, including CPU->GPU data copy, GPU inference, and GPU->CPU data copy time. It does not include the pre and post processing time of the model.
|
||||
- End2End latency: model’s latency in the actual inference scenario, including the pre and post processing time of the model.
|
||||
- Measured latency: The average latency after 1000 times of inference in milliseconds.
|
||||
- INT8 + FP16: Enable FP16 inference for Runtime while inferring the INT8 quantification model
|
||||
- INT8 + FP16 + PM: Use Pinned Memory to speed up the GPU->CPU data copy while inferring the INT8 quantization model with FP16 turned on.
|
||||
- Maximum speedup ratio: Obtained by dividing the FP32 latency by the highest INT8 inference latency.
|
||||
- The strategy is to use a few unlabeled data sets to train the model for quantification and to verify the accuracy on the full validation set. The INT8 accuracy does not represent the highest value.
|
||||
- The CPU is Intel(R) Xeon(R) Gold 6271C, and the number of CPU threads is fixed to 1. The GPU is Tesla T4 with TensorRT version 8.4.15.
|
||||
## 1. FastDeploy一键模型自动化压缩工具
|
||||
|
||||
### Runtime Benchmark
|
||||
| Model |Inference Backend |Deployment Hardware | FP32 Runtime Latency | INT8 Runtime Latency | INT8 + FP16 Runtime Latency | INT8+FP16+PM Runtime Latency | Maximum Speedup Ratio | FP32 Top1 | INT8 Top1 | Quantification Method |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | TensorRT | GPU | 3.55 | 0.99|0.98|1.06 | 3.62 | 79.12 | 79.06 | Offline |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle-TensorRT | GPU | 3.46 |None |0.87|1.03 | 3.98 | 79.12 | 79.06 | Offline |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | ONNX Runtime | CPU | 76.14 | 35.43 |None|None | 2.15 | 79.12 | 78.87| Offline|
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle Inference | CPU | 76.21 | 24.01 |None|None | 3.17 | 79.12 | 78.55 | Offline|
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | TensorRT | GPU | 0.91 | 0.43 |0.49 | 0.54 | 2.12 |77.89 | 76.86 | Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle-TensorRT | GPU | 0.88| None| 0.49|0.51 | 1.80 |77.89 | 76.86 | Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | ONNX Runtime | CPU | 30.53 | 9.59|None|None | 3.18 |77.89 | 75.09 |Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle Inference | CPU | 12.29 | 4.68 | None|None|2.62 |77.89 | 71.36 |Offline |
|
||||
FastDeploy 提供了一键模型自动化压缩工具, 能够简单地通过输入一个配置文件, 对模型进行量化.
|
||||
详细教程请见: [一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression)。**注意**: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
|
||||
### End2End Benchmark
|
||||
| Model |Inference Backend |Deployment Hardware | FP32 End2End Latency | INT8 End2End Latency | INT8 + FP16 End2End Latency | INT8+FP16+PM End2End Latency | Maximum Speedup Ratio | FP32 Top1 | INT8 Top1 | Quantification Method |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | TensorRT | GPU | 4.92| 2.28|2.24|2.23 | 2.21 | 79.12 | 79.06 | Offline |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle-TensorRT | GPU | 4.48|None |2.09|2.10 | 2.14 | 79.12 | 79.06 | Offline |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | ONNX Runtime | CPU | 77.43 | 41.90 |None|None | 1.85 | 79.12 | 78.87| Offline|
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle Inference | CPU | 80.60 | 27.75 |None|None | 2.90 | 79.12 | 78.55 | Offline|
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | TensorRT | GPU | 2.19 | 1.48|1.57| 1.57 | 1.48 |77.89 | 76.86 | Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle-TensorRT | GPU | 2.04| None| 1.47|1.45 | 1.41 |77.89 | 76.86 | Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | ONNX Runtime | CPU | 34.02 | 12.97|None|None | 2.62 |77.89 | 75.09 |Offline |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle Inference | CPU | 16.31 | 7.42 | None|None| 2.20 |77.89 | 71.36 |Offline |
|
||||
## 2. 下载量化完成的PaddleClas模型
|
||||
|
||||
## Detailed Deployment Tutorials
|
||||
用户也可以直接下载下表中的量化模型进行部署.(点击模型名字即可下载)
|
||||
|
||||
- [Python Deployment](python)
|
||||
- [C++ Deployment](cpp)
|
||||
| 模型 | 量化方式 |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | 离线量化 |
|
||||
|
||||
量化后模型的Benchmark比较，请参考[量化模型 Benchmark](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/quantize.md)
|
||||
|
||||
## 3. 部署量化模型
|
||||
|
||||
### 3.1 部署代码
|
||||
FastDeploy 部署量化模型与部署FP32模型完全一致, 用户只需要将输入的模型换为量化后的模型即可.
|
||||
如果硬件在量化模型部署过程有特殊处理，也会在文档中特别标明.
|
||||
因此本目录下，不提供代码文件, 量化模型部署参考对应的硬件部署即可, 具体请点击下一小节里的链接.
|
||||
|
||||
### 3.2 支持部署量化模型的硬件
|
||||
|
||||
|硬件类型|该硬件是否支持|使用指南|Python|C++|
|
||||
|:---:|:---:|:---:|:---:|:---:|
|
||||
|X86 CPU|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|NVIDIA GPU|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|飞腾CPU|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|ARM CPU|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|Intel GPU(集成显卡)|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|Intel GPU(独立显卡)|✅|[链接](cpu-gpu)|✅|✅|
|
||||
|昆仑|✅|[链接](kunlunxin)|✅|✅|
|
||||
|昇腾|✅|[链接](ascend)|✅|✅|
|
||||
|瑞芯微|✅|[链接](rockchip)|✅|✅|
|
||||
|晶晨|✅|[链接](amlogic)|--|✅|
|
||||
|算能|✅|[链接](sophgo)|✅|✅|
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 量化模型部署
|
||||
FastDeploy已支持部署量化模型,并提供一键模型自动化压缩的工具.
|
||||
用户可以使用一键模型自动化压缩工具,自行对模型量化后部署, 也可以直接下载FastDeploy提供的量化模型进行部署.
|
||||
|
||||
## FastDeploy一键模型自动化压缩工具
|
||||
FastDeploy 提供了一键模型自动化压缩工具, 能够简单地通过输入一个配置文件, 对模型进行量化.
|
||||
详细教程请见: [一键模型自动化压缩工具](../../../../../tools/common_tools/auto_compression/)
|
||||
注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
|
||||
## 下载量化完成的PaddleClas模型
|
||||
用户也可以直接下载下表中的量化模型进行部署.
|
||||
|
||||
Benchmark表格说明:
|
||||
- Runtime时延为模型在各种Runtime上的推理时延,包含CPU->GPU数据拷贝,GPU推理,GPU->CPU数据拷贝时间. 不包含模型各自的前后处理时间.
|
||||
- 端到端时延为模型在实际推理场景中的时延, 包含模型的前后处理.
|
||||
- 所测时延均为推理1000次后求得的平均值, 单位是毫秒.
|
||||
- INT8 + FP16 为在推理INT8量化模型的同时, 给Runtime 开启FP16推理选项
|
||||
- INT8 + FP16 + PM, 为在推理INT8量化模型和开启FP16的同时, 开启使用Pinned Memory的选项,可加速GPU->CPU数据拷贝的速度
|
||||
- 最大加速比, 为FP32时延除以INT8推理的最快时延,得到最大加速比.
|
||||
- 策略为量化蒸馏训练时, 采用少量无标签数据集训练得到量化模型, 并在全量验证集上验证精度, INT8精度并不代表最高的INT8精度.
|
||||
- CPU为Intel(R) Xeon(R) Gold 6271C, 所有测试中固定CPU线程数为1. GPU为Tesla T4, TensorRT版本8.4.15.
|
||||
|
||||
### Runtime Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 Runtime时延 | INT8 Runtime时延 | INT8 + FP16 Runtime时延 | INT8+FP16+PM Runtime时延 | 最大加速比 | FP32 Top1 | INT8 Top1 | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | TensorRT | GPU | 3.55 | 0.99|0.98|1.06 | 3.62 | 79.12 | 79.06 | 离线量化 |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle-TensorRT | GPU | 3.46 |None |0.87|1.03 | 3.98 | 79.12 | 79.06 | 离线量化 |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | ONNX Runtime | CPU | 76.14 | 35.43 |None|None | 2.15 | 79.12 | 78.87| 离线量化|
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle Inference | CPU | 76.21 | 24.01 |None|None | 3.17 | 79.12 | 78.55 | 离线量化|
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | TensorRT | GPU | 0.91 | 0.43 |0.49 | 0.54 | 2.12 |77.89 | 76.86 | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle-TensorRT | GPU | 0.88| None| 0.49|0.51 | 1.80 |77.89 | 76.86 | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | ONNX Runtime | CPU | 30.53 | 9.59|None|None | 3.18 |77.89 | 75.09 |离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle Inference | CPU | 12.29 | 4.68 | None|None|2.62 |77.89 | 71.36 |离线量化 |
|
||||
|
||||
### 端到端 Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 End2End时延 | INT8 End2End时延 | INT8 + FP16 End2End时延 | INT8+FP16+PM End2End时延 | 最大加速比 | FP32 Top1 | INT8 Top1 | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | TensorRT | GPU | 4.92| 2.28|2.24|2.23 | 2.21 | 79.12 | 79.06 | 离线量化 |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle-TensorRT | GPU | 4.48|None |2.09|2.10 | 2.14 | 79.12 | 79.06 | 离线量化 |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | ONNX Runtime | CPU | 77.43 | 41.90 |None|None | 1.85 | 79.12 | 78.87| 离线量化|
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | Paddle Inference | CPU | 80.60 | 27.75 |None|None | 2.90 | 79.12 | 78.55 | 离线量化|
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | TensorRT | GPU | 2.19 | 1.48|1.57| 1.57 | 1.48 |77.89 | 76.86 | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle-TensorRT | GPU | 2.04| None| 1.47|1.45 | 1.41 |77.89 | 76.86 | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | ONNX Runtime | CPU | 34.02 | 12.97|None|None | 2.62 |77.89 | 75.09 |离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | Paddle Inference | CPU | 16.31 | 7.42 | None|None| 2.20 |77.89 | 71.36 |离线量化 |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -1,37 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Quantitative Model C++ Deployment Example
|
||||
`infer.cc` in this directory can help you quickly complete the inference acceleration of PaddleClas quantization model deployment on CPU/GPU.
|
||||
|
||||
## Deployment Preparations
|
||||
### FastDeploy Environment Preparations
|
||||
- 1. For the software and hardware requirements, please refer to [FastDeploy Environment Requirements](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
- 2. For the installation of FastDeploy Python whl package, please refer to [FastDeploy Python Installation](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
|
||||
### Quantized Model Preparations
|
||||
- 1. You can directly use the quantized model provided by FastDeploy for deployment.
|
||||
- 2. You can use [one-click automatical compression tool](../../../../../../tools/common_tools/auto_compression/) provided by FastDeploy to quantize model by yourself, and use the generated quantized model for deployment.(Note: The quantized classification model still needs the inference_cls.yaml file in the FP32 model folder. Self-quantized model folder does not contain this yaml file, you can copy it from the FP32 model folder to the quantized model folder.)
|
||||
|
||||
## Take the Quantized PP-YOLOE-l Model as an example for Deployment, FastDeploy version 0.7.0 or higher is required (x.x.x>=0.7.0)
|
||||
Run the following commands in this directory to compile and deploy the quantized model.
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download pre-compiled FastDeploy libraries. You can choose the appropriate version from `pre-compiled FastDeploy libraries` mentioned above.
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# Download the ResNet50_Vd quantized model and test images provided by FastDeloy.
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# Use ONNX Runtime inference quantization model on CPU.
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 0
|
||||
# Use TensorRT inference quantization model on GPU.
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 1
|
||||
# Use Paddle-TensorRT inference quantization model on GPU.
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 2
|
||||
```
|
||||
@@ -1,37 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 量化模型 C++部署示例
|
||||
本目录下提供的`infer.cc`,可以帮助用户快速完成PaddleClas量化模型在CPU/GPU上的部署推理加速.
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy环境准备
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
### 量化模型准备
|
||||
- 1. 用户可以直接使用由FastDeploy提供的量化模型进行部署.
|
||||
- 2. 用户可以使用FastDeploy提供的[一键模型自动化压缩工具](../../../../../../tools/common_tools/auto_compression/),自行进行模型量化, 并使用产出的量化模型进行部署.(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可.)
|
||||
|
||||
## 以量化后的ResNet50_Vd模型为例, 进行部署，支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)
|
||||
在本目录执行如下命令即可完成编译,以及量化模型部署.
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
#下载FastDeloy提供的ResNet50_Vd量化模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
|
||||
# 在CPU上使用ONNX Runtime推理量化模型
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 0
|
||||
# 在GPU上使用TensorRT推理量化模型
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 1
|
||||
# 在GPU上使用Paddle-TensorRT推理量化模型
|
||||
./infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg 2
|
||||
```
|
||||
@@ -1,32 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Quantitative Model Python Deployment Example
|
||||
`infer.py` in this directory can help you quickly complete the inference acceleration of PaddleClas quantization model deployment on CPU/GPU.
|
||||
|
||||
## Deployment Preparations
|
||||
### FastDeploy Environment Preparations
|
||||
- 1. For the software and hardware requirements, please refer to [FastDeploy Environment Requirements](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
- 2. For the installation of FastDeploy Python whl package, please refer to [FastDeploy Python Installation](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md).
|
||||
|
||||
### Quantized Model Preparations
|
||||
- 1. You can directly use the quantized model provided by FastDeploy for deployment.
|
||||
- 2. You can use [one-click automatical compression tool](../../../../../../tools/common_tools/auto_compression/) provided by FastDeploy to quantize model by yourself, and use the generated quantized model for deployment.(Note: The quantized classification model still needs the inference_cls.yaml file in the FP32 model folder. Self-quantized model folder does not contain this yaml file, you can copy it from the FP32 model folder to the quantized model folder.)
|
||||
|
||||
|
||||
## Take the Quantized ResNet50_Vd Model as an example for Deployment
|
||||
```bash
|
||||
# Download sample deployment code.
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd examples/vision/classification/paddleclas/quantize/python
|
||||
|
||||
# Download the ResNet50_Vd quantized model and test images provided by FastDeloy.
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Use ONNX Runtime inference quantization model on CPU.
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device cpu --backend ort
|
||||
# Use TensorRT inference quantization model on GPU.
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device gpu --backend trt
|
||||
# Use Paddle-TensorRT inference quantization model on GPU.
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device gpu --backend pptrt
|
||||
```
|
||||
@@ -1,32 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 量化模型 Python部署示例
|
||||
本目录下提供的`infer.py`,可以帮助用户快速完成PaddleClas量化模型在CPU/GPU上的部署推理加速.
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy环境准备
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
### 量化模型准备
|
||||
- 1. 用户可以直接使用由FastDeploy提供的量化模型进行部署.
|
||||
- 2. 用户可以使用FastDeploy提供的[一键模型自动化压缩工具](../../../../../../tools/common_tools/auto_compression/),自行进行模型量化, 并使用产出的量化模型进行部署.(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可.)
|
||||
|
||||
|
||||
## 以量化后的ResNet50_Vd模型为例, 进行部署
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd examples/vision/classification/paddleclas/quantize/python
|
||||
|
||||
#下载FastDeloy提供的ResNet50_Vd量化模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 在CPU上使用ONNX Runtime推理量化模型
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device cpu --backend ort
|
||||
# 在GPU上使用TensorRT推理量化模型
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device gpu --backend trt
|
||||
# 在GPU上使用Paddle-TensorRT推理量化模型
|
||||
python infer.py --model resnet50_vd_ptq --image ILSVRC2012_val_00000010.jpeg --device gpu --backend pptrt
|
||||
```
|
||||
@@ -1,84 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Model RKNPU2 Deployment
|
||||
|
||||
## Convert the model
|
||||
Taking ResNet50_vd as an example, this document demonstrates how to convert classification model to RKNN model.
|
||||
|
||||
### Export the ONNX model
|
||||
```bash
|
||||
# Install paddle2onnx
|
||||
pip install paddle2onnx
|
||||
|
||||
# Download ResNet50_vd model files and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
|
||||
# From static map to ONNX model. Attention: Align the save_file with the zip file name
|
||||
paddle2onnx --model_dir ResNet50_vd_infer \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--save_file ResNet50_vd_infer/ResNet50_vd_infer.onnx \
|
||||
--enable_dev_version True \
|
||||
--opset_version 10 \
|
||||
--enable_onnx_checker True
|
||||
|
||||
# Fix shape. Attention: the inputs here should correspond to the name of the inputs shown in netron.app, which may be image or x
|
||||
python -m paddle2onnx.optimize --input_model ResNet50_vd_infer/ResNet50_vd_infer.onnx \
|
||||
--output_model ResNet50_vd_infer/ResNet50_vd_infer.onnx \
|
||||
--input_shape_dict "{'inputs':[1,3,224,224]}"
|
||||
```
|
||||
|
||||
### Write the model export configuration file
|
||||
Taking the example of RKNN model from RK3588, we need to edit tools/rknpu2/config/ResNet50_vd_infer_rknn.yaml to convert ONNX model to RKNN model.
|
||||
|
||||
If you need to perform the normalize operation on NPU, configure the normalize parameters based on your model. For example:
|
||||
```yaml
|
||||
model_path: ./ResNet50_vd_infer/ResNet50_vd_infer.onnx
|
||||
output_folder: ./ResNet50_vd_infer
|
||||
mean:
|
||||
-
|
||||
- 123.675
|
||||
- 116.28
|
||||
- 103.53
|
||||
std:
|
||||
-
|
||||
- 58.395
|
||||
- 57.12
|
||||
- 57.375
|
||||
outputs_nodes:
|
||||
do_quantization: False
|
||||
dataset: "./ResNet50_vd_infer/dataset.txt"
|
||||
```
|
||||
|
||||
To **normalize on CPU**, refer to the following yaml：
|
||||
```yaml
|
||||
model_path: ./ResNet50_vd_infer/ResNet50_vd_infer.onnx
|
||||
output_folder: ./ResNet50_vd_infer
|
||||
mean:
|
||||
-
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
std:
|
||||
-
|
||||
- 1
|
||||
- 1
|
||||
- 1
|
||||
outputs_nodes:
|
||||
do_quantization: False
|
||||
dataset: "./ResNet50_vd_infer/dataset.txt"
|
||||
```
|
||||
Here we perform the normalize operation on NPU.
|
||||
|
||||
|
||||
### From ONNX model to RKNN model
|
||||
```shell
|
||||
python tools/rknpu2/export.py \
|
||||
--config_path tools/rknpu2/config/ResNet50_vd_infer_rknn.yaml \
|
||||
--target_platform rk3588
|
||||
```
|
||||
|
||||
## Other Links
|
||||
- [Cpp Deployment](./cpp)
|
||||
- [Python Deployment](./python)
|
||||
- [Vision Model Prediction Results](../../../../../docs/api/vision_results/)
|
||||
@@ -1,77 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas C++ Deployment Example
|
||||
|
||||
This directory demonstrates the deployment of ResNet50_vd model on RKNPU2. The following deployment process takes ResNet50_vd as an example.
|
||||
|
||||
Before deployment, the following two steps need to be confirmed:
|
||||
|
||||
1. Hardware and software environment meets the requirements.
|
||||
2. Download the pre-compiled deployment repository or compile the FastDeploy repository from scratch according to the development environment.
|
||||
|
||||
For the above steps, please refer to [How to Build RKNPU2 Deployment Environment](../../../../../../docs/en/build_and_install/rknpu2.md).
|
||||
|
||||
## Generate Basic Directory Files
|
||||
|
||||
The routine consists of the following parts:
|
||||
```text
|
||||
.
|
||||
├── CMakeLists.txt
|
||||
├── build # Compile Folder
|
||||
├── images # Folder for images
|
||||
├── infer.cc
|
||||
├── ppclas_model_dir # Folder for models
|
||||
└── thirdpartys # Folder for sdk
|
||||
```
|
||||
|
||||
First, please build a directory structure
|
||||
```bash
|
||||
mkdir build
|
||||
mkdir images
|
||||
mkdir ppclas_model_dir
|
||||
mkdir thirdpartys
|
||||
```
|
||||
|
||||
## Compile
|
||||
|
||||
### Compile and Copy SDK to folder thirdpartys
|
||||
|
||||
Please refer to [How to Build RKNPU2 Deployment Environment](../../../../../../docs/en/build_and_install/rknpu2.md) to compile SDK.After compiling, the fastdeploy-0.0.3 directory will be created in the build directory, please move it to the thirdpartys directory.
|
||||
|
||||
### Copy model and configuration files to folder Model
|
||||
In the process of Paddle dynamic map model -> Paddle static map model -> ONNX mdoel, ONNX file and the corresponding yaml configuration file will be generated. Please move the configuration file to the folder model.
|
||||
After converting to RKNN, the model file also needs to be copied to folder model. Please refer to ([ResNet50_vd RKNN model](../README.md)).
|
||||
|
||||
### Prepare Test Images to folder image
|
||||
```bash
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
### Compile example
|
||||
|
||||
```bash
|
||||
cd build
|
||||
cmake ..
|
||||
make -j8
|
||||
make install
|
||||
```
|
||||
|
||||
## Running Routines
|
||||
|
||||
```bash
|
||||
cd ./build/install
|
||||
./rknpu_test ./ppclas_model_dir ./images/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
## Results
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
|
||||
## Notes
|
||||
The input requirement for the model on RKNPU is to use NHWC format, and image normalization will be embedded into the model when converting the RKNN model, so we need to call DisablePermute(C++) or disable_permute(Python) first when deploying with FastDeploy to disable data format conversion in the preprocessing stage.
|
||||
|
||||
## Other Documents
|
||||
- [ResNet50_vd Python Deployment](../python)
|
||||
- [Prediction results](../../../../../../docs/api/vision_results/)
|
||||
- [Converting ResNet50_vd RKNN model](../README.md)
|
||||
@@ -1,77 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas C++部署示例
|
||||
|
||||
本目录下用于展示 ResNet50_vd 模型在RKNPU2上的部署，以下的部署过程以 ResNet50_vd 为例子。
|
||||
|
||||
在部署前，需确认以下两个步骤:
|
||||
|
||||
1. 软硬件环境满足要求
|
||||
2. 根据开发环境，下载预编译部署库或者从头编译FastDeploy仓库
|
||||
|
||||
以上步骤请参考[RK2代NPU部署库编译](../../../../../../docs/cn/build_and_install/rknpu2.md)实现
|
||||
|
||||
## 生成基本目录文件
|
||||
|
||||
该例程由以下几个部分组成
|
||||
```text
|
||||
.
|
||||
├── CMakeLists.txt
|
||||
├── build # 编译文件夹
|
||||
├── images # 存放图片的文件夹
|
||||
├── infer.cc
|
||||
├── ppclas_model_dir # 存放模型文件的文件夹
|
||||
└── thirdpartys # 存放sdk的文件夹
|
||||
```
|
||||
|
||||
首先需要先生成目录结构
|
||||
```bash
|
||||
mkdir build
|
||||
mkdir images
|
||||
mkdir ppclas_model_dir
|
||||
mkdir thirdpartys
|
||||
```
|
||||
|
||||
## 编译
|
||||
|
||||
### 编译并拷贝SDK到thirdpartys文件夹
|
||||
|
||||
请参考[RK2代NPU部署库编译](../../../../../../docs/cn/build_and_install/rknpu2.md)仓库编译SDK，编译完成后，将在build目录下生成fastdeploy-0.0.3目录，请移动它至thirdpartys目录下.
|
||||
|
||||
### 拷贝模型文件，以及配置文件至model文件夹
|
||||
在Paddle动态图模型 -> Paddle静态图模型 -> ONNX模型的过程中，将生成ONNX文件以及对应的yaml配置文件，请将配置文件存放到model文件夹内。
|
||||
转换为RKNN后的模型文件也需要拷贝至model，转换方案: ([ResNet50_vd RKNN模型](../README.md))。
|
||||
|
||||
### 准备测试图片至image文件夹
|
||||
```bash
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
### 编译example
|
||||
|
||||
```bash
|
||||
cd build
|
||||
cmake ..
|
||||
make -j8
|
||||
make install
|
||||
```
|
||||
|
||||
## 运行例程
|
||||
|
||||
```bash
|
||||
cd ./build/install
|
||||
./rknpu_test ./ppclas_model_dir ./images/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
## 运行结果展示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
|
||||
## 注意事项
|
||||
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisablePermute(C++)或`disable_permute(Python)，在预处理阶段禁用数据格式的转换。
|
||||
|
||||
## 其它文档
|
||||
- [ResNet50_vd Python 部署](../python)
|
||||
- [模型预测结果说明](../../../../../../docs/api/vision_results/)
|
||||
- [转换ResNet50_vd RKNN模型文档](../README.md)
|
||||
@@ -1,35 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Python Deployment Example
|
||||
|
||||
Before deployment, the following step need to be confirmed:
|
||||
|
||||
- 1. Hardware and software environment meets the requirements, please refer to [Environment Requirements for FastDeploy](../../../../../../docs/en/build_and_install/rknpu2.md).
|
||||
|
||||
This directory provides `infer.py` for a quick example of ResNet50_vd deployment on RKNPU. This can be done by running the following script.
|
||||
|
||||
```bash
|
||||
# Download the deploying demo code.
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rknpu2/python
|
||||
|
||||
# Download images.
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Inference.
|
||||
python3 infer.py --model_file ./ResNet50_vd_infer/ResNet50_vd_infer_rk3588.rknn --config_file ResNet50_vd_infer/inference_cls.yaml --image ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Results
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
## Notes
|
||||
The input requirement for the model on RKNPU is to use NHWC format, and image normalization will be embedded into the model when converting the RKNN model, so we need to call DisablePermute(C++) or disable_permute(Python) first when deploying with FastDeploy to disable data format conversion in the preprocessing stage.
|
||||
|
||||
## Other Documents
|
||||
- [ResNet50_vd C++ Deployment](../cpp)
|
||||
- [Prediction Results](../../../../../../docs/api/vision_results/)
|
||||
- [Converting ResNet50_vd RKNN model](../README.md)
|
||||
@@ -1,35 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../../docs/cn/build_and_install/rknpu2.md)
|
||||
|
||||
本目录下提供`infer.py`快速完成 ResNet50_vd 在RKNPU上部署的示例。执行如下脚本即可完成
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rknpu2/python
|
||||
|
||||
# 下载图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 推理
|
||||
python3 infer.py --model_file ./ResNet50_vd_infer/ResNet50_vd_infer_rk3588.rknn --config_file ResNet50_vd_infer/inference_cls.yaml --image ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 运行完成后返回结果如下所示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
## 注意事项
|
||||
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisablePermute(C++)或`disable_permute(Python)，在预处理阶段禁用数据格式的转换。
|
||||
|
||||
## 其它文档
|
||||
- [ResNet50_vd C++部署](../cpp)
|
||||
- [模型预测结果说明](../../../../../../docs/api/vision_results/)
|
||||
- [转换ResNet50_vd RKNN模型文档](../README.md)
|
||||
@@ -1,10 +1,14 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 模型RKNPU2部署
|
||||
|
||||
## 转换模型
|
||||
下面以 ResNet50_vd为例子，教大家如何转换分类模型到RKNN模型。
|
||||
# PaddleClas 模型在RKNPU2上部署方案-FastDeploy
|
||||
|
||||
### 导出ONNX模型
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在RKNPU2上部署相关模型.
|
||||
|
||||
## 2. 转换模型
|
||||
下面以 ResNet50_vd为例子，教大家如何转换分类模型到RKNN模型.
|
||||
目前FastDeploy只在PaddleClas测试过ResNet50_vd模型, 欢迎用户尝试其他的PaddleClas模型进行部署.
|
||||
|
||||
### 2.1 导出ONNX模型
|
||||
```bash
|
||||
# 安装 paddle2onnx
|
||||
pip install paddle2onnx
|
||||
@@ -28,8 +32,8 @@ python -m paddle2onnx.optimize --input_model ResNet50_vd_infer/ResNet50_vd_infer
|
||||
--input_shape_dict "{'inputs':[1,3,224,224]}"
|
||||
```
|
||||
|
||||
### 编写模型导出配置文件
|
||||
以转化RK3588的RKNN模型为例子，我们需要编辑tools/rknpu2/config/ResNet50_vd_infer_rknn.yaml，来转换ONNX模型到RKNN模型。
|
||||
### 2.2 编写模型导出配置文件
|
||||
以转化RK3588的RKNN模型为例子，我们需要编辑./rknpu2_tools/config/ResNet50_vd_infer_rknn.yaml，来转换ONNX模型到RKNN模型。
|
||||
|
||||
如果你需要在NPU上执行normalize操作，请根据你的模型配置normalize参数，例如:
|
||||
```yaml
|
||||
@@ -71,14 +75,14 @@ dataset: "./ResNet50_vd_infer/dataset.txt"
|
||||
这里我们选择在NPU上执行normalize操作.
|
||||
|
||||
|
||||
### ONNX模型转RKNN模型
|
||||
### 2.3 ONNX模型转RKNN模型
|
||||
```shell
|
||||
python tools/rknpu2/export.py \
|
||||
--config_path tools/rknpu2/config/ResNet50_vd_infer_rknn.yaml \
|
||||
python ./rknpu2_tools/export.py \
|
||||
--config_path ./rknpu2_tools/config/ResNet50_vd_infer_rknn.yaml \
|
||||
--target_platform rk3588
|
||||
```
|
||||
|
||||
## 其他链接
|
||||
## 3. 其他链接
|
||||
- [Cpp部署](./cpp)
|
||||
- [Python部署](./python)
|
||||
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||
@@ -0,0 +1,89 @@
|
||||
# PaddleClas RKNPU2 C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`, 供用户完成PaddleClas模型在RKNPU2的部署.
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需确认以下两个步骤
|
||||
- 1. 在部署前，需自行编译基于RKNPU2的预测库，参考文档[RKNPU2部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
- 2. 同时请用户参考[FastDeploy RKNPU2资源导航](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/rknpu2.md)
|
||||
|
||||
## 2.部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以参考[RKNPU2模型转换](../README.md), 来准备模型.
|
||||
|
||||
## 3.部署示例
|
||||
|
||||
### 3.1 生成基本目录文件
|
||||
|
||||
该例程由以下几个部分组成
|
||||
```text
|
||||
.
|
||||
├── CMakeLists.txt
|
||||
├── build # 编译文件夹
|
||||
├── images # 存放图片的文件夹
|
||||
├── infer.cc
|
||||
├── ppclas_model_dir # 存放模型文件的文件夹
|
||||
└── thirdpartys # 存放sdk的文件夹
|
||||
```
|
||||
|
||||
首先需要先生成目录结构
|
||||
```bash
|
||||
mkdir build
|
||||
mkdir images
|
||||
mkdir ppclas_model_dir
|
||||
mkdir thirdpartys
|
||||
```
|
||||
|
||||
### 3.2 编译
|
||||
|
||||
#### 3.2.1 编译并拷贝SDK到thirdpartys文件夹
|
||||
|
||||
请参考[RK2代NPU部署库编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/rknpu2.md)仓库编译SDK，编译完成后，将在build目录下生成fastdeploy-x.x.x目录，请移动它至thirdpartys目录下.
|
||||
|
||||
#### 3.2.2 拷贝模型文件，以及配置文件至model文件夹
|
||||
在Paddle动态图模型 -> Paddle静态图模型 -> ONNX模型的过程中，将生成ONNX文件以及对应的yaml配置文件，请将配置文件存放到model文件夹内。
|
||||
转换为RKNN后的模型文件也需要拷贝至model，转换方案: ([ResNet50_vd RKNN模型](../README.md))。
|
||||
|
||||
#### 3.2.3 准备测试图片至image文件夹
|
||||
```bash
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
#### 3.2.4 编译example
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rockchip/rknpu2/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/rockchip/rknpu2/cpp
|
||||
|
||||
|
||||
cd build
|
||||
cmake ..
|
||||
make -j8
|
||||
make install
|
||||
```
|
||||
|
||||
#### 3.2.5 运行例程
|
||||
|
||||
```bash
|
||||
cd ./build/install
|
||||
./rknpu_test ./ppclas_model_dir ./images/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
#### 3.2.6 运行结果展示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
|
||||
#### 3.2.7 注意事项
|
||||
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisablePermute(C++)或`disable_permute(Python)，在预处理阶段禁用数据格式的转换。
|
||||
|
||||
## 4. 其它文档
|
||||
- [ResNet50_vd Python 部署](../python)
|
||||
- [转换ResNet50_vd RKNN模型文档](../README.md)
|
||||
@@ -14,7 +14,7 @@
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
|
||||
void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
void RKNPU2Infer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + "/ResNet50_vd_infer_rk3588.rknn";
|
||||
auto params_file = "";
|
||||
auto config_file = model_dir + "/inference_cls.yaml";
|
||||
@@ -25,7 +25,7 @@ void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto format = fastdeploy::ModelFormat::RKNN;
|
||||
|
||||
auto model = fastdeploy::vision::classification::PaddleClasModel(
|
||||
model_file, params_file, config_file,option,format);
|
||||
model_file, params_file, config_file, option, format);
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
@@ -45,11 +45,12 @@ void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
tc.PrintInfo("PPClas in RKNPU2");
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout
|
||||
<< "Usage: rknpu_test path/to/model_dir path/to/image run_option, "
|
||||
"e.g ./rknpu_test ./ppclas_model_dir ./images/ILSVRC2012_val_00000010.jpeg"
|
||||
"e.g ./rknpu_test ./ppclas_model_dir "
|
||||
"./images/ILSVRC2012_val_00000010.jpeg"
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
# PaddleClas Python部署示例
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需确认以下两个步骤
|
||||
- 1. 在部署前，需自行编译基于RKNPU2的Python预测库，参考文档[RKNPU2部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
- 2. 同时请用户参考[FastDeploy RKNPU2资源导航](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/rknpu2.md)
|
||||
|
||||
## 2. 部署模型准备
|
||||
在部署前, 请准备好您所需要运行的推理模型, 您可以参考[RKNPU2模型转换](../README.md), 来准备模型.
|
||||
|
||||
## 3. 部署示例
|
||||
|
||||
本目录下提供`infer.py`快速完成 ResNet50_vd 在RKNPU上部署的示例
|
||||
|
||||
```bash
|
||||
# 安装FastDpeloy RKNPU2 python包（详细文档请参考`部署环境准备`）
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rockchip/rknpu2/python
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/rockchip/rknpu2/python
|
||||
|
||||
# 下载图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 推理
|
||||
python3 infer.py --model_file ./ResNet50_vd_infer/ResNet50_vd_infer_rk3588.rknn --config_file ResNet50_vd_infer/inference_cls.yaml --image ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 运行完成后返回结果如下所示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
```
|
||||
|
||||
## 4.注意事项
|
||||
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisablePermute(C++)或`disable_permute(Python)，在预处理阶段禁用数据格式的转换。
|
||||
|
||||
## 5. 其它文档
|
||||
- [ResNet50_vd C++部署](../cpp)
|
||||
- [转换ResNet50_vd RKNN模型文档](../README.md)
|
||||
@@ -0,0 +1,15 @@
|
||||
model_path: ./ResNet50_vd_infer/ResNet50_vd_infer.onnx
|
||||
output_folder: ./ResNet50_vd_infer
|
||||
mean:
|
||||
-
|
||||
- 123.675
|
||||
- 116.28
|
||||
- 103.53
|
||||
std:
|
||||
-
|
||||
- 58.395
|
||||
- 57.12
|
||||
- 57.375
|
||||
outputs_nodes:
|
||||
do_quantization: False
|
||||
dataset: "./ResNet50_vd_infer/dataset.txt"
|
||||
@@ -0,0 +1,80 @@
|
||||
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import os
|
||||
import yaml
|
||||
import argparse
|
||||
from rknn.api import RKNN
|
||||
|
||||
|
||||
def get_config():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--verbose", default=True, help="rknntoolkit verbose")
|
||||
parser.add_argument("--config_path")
|
||||
parser.add_argument("--target_platform")
|
||||
args = parser.parse_args()
|
||||
return args
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config = get_config()
|
||||
with open(config.config_path) as file:
|
||||
file_data = file.read()
|
||||
yaml_config = yaml.safe_load(file_data)
|
||||
print(yaml_config)
|
||||
model = RKNN(config.verbose)
|
||||
|
||||
# Config
|
||||
mean_values = yaml_config["mean"]
|
||||
std_values = yaml_config["std"]
|
||||
model.config(
|
||||
mean_values=mean_values,
|
||||
std_values=std_values,
|
||||
target_platform=config.target_platform)
|
||||
|
||||
# Load ONNX model
|
||||
if yaml_config["outputs_nodes"] is None:
|
||||
ret = model.load_onnx(model=yaml_config["model_path"])
|
||||
else:
|
||||
ret = model.load_onnx(
|
||||
model=yaml_config["model_path"],
|
||||
outputs=yaml_config["outputs_nodes"])
|
||||
assert ret == 0, "Load model failed!"
|
||||
|
||||
# Build model
|
||||
ret = model.build(
|
||||
do_quantization=yaml_config["do_quantization"],
|
||||
dataset=yaml_config["dataset"])
|
||||
assert ret == 0, "Build model failed!"
|
||||
|
||||
# Init Runtime
|
||||
ret = model.init_runtime()
|
||||
assert ret == 0, "Init runtime environment failed!"
|
||||
|
||||
# Export
|
||||
if not os.path.exists(yaml_config["output_folder"]):
|
||||
os.mkdir(yaml_config["output_folder"])
|
||||
|
||||
name_list = os.path.basename(yaml_config["model_path"]).split(".")
|
||||
model_base_name = ""
|
||||
for name in name_list[0:-1]:
|
||||
model_base_name += name
|
||||
model_device_name = config.target_platform.lower()
|
||||
if yaml_config["do_quantization"]:
|
||||
model_save_name = model_base_name + "_" + model_device_name + "_quantized" + ".rknn"
|
||||
else:
|
||||
model_save_name = model_base_name + "_" + model_device_name + "_unquantized" + ".rknn"
|
||||
ret = model.export_rknn(
|
||||
os.path.join(yaml_config["output_folder"], model_save_name))
|
||||
assert ret == 0, "Export rknn model failed!"
|
||||
print("Export OK!")
|
||||
22
examples/vision/classification/paddleclas/rockchip/rv1126/README.md
Executable file
22
examples/vision/classification/paddleclas/rockchip/rv1126/README.md
Executable file
@@ -0,0 +1,22 @@
|
||||
|
||||
# PaddleClas 图像分类模瑞芯微NPU部署方案-FastDeploy
|
||||
|
||||
## 1. 说明
|
||||
本示例基于RV1126来介绍如何使用FastDeploy部署PaddleClas量化模型，支持如下芯片的部署：
|
||||
- Rockchip RV1109
|
||||
- Rockchip RV1126
|
||||
- Rockchip RK1808
|
||||
|
||||
## 2. 使用预导出的模型列表
|
||||
|
||||
FastDeploy提供预先量化好的模型进行部署. 更多模型, 欢迎用户参考[FastDeploy 一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression) 来实现模型量化, 并完成部署.
|
||||
|
||||
|
||||
| 模型 | 量化方式 |
|
||||
|:---------------| :----- |
|
||||
| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar) | 离线量化 |
|
||||
| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar) | 离线量化 |
|
||||
|
||||
## 3. 详细部署示例
|
||||
在 RV1126 上只支持 C++ 的部署。
|
||||
- [C++部署](cpp)
|
||||
61
examples/vision/classification/paddleclas/rockchip/rv1126/cpp/README.md
Executable file
61
examples/vision/classification/paddleclas/rockchip/rv1126/cpp/README.md
Executable file
@@ -0,0 +1,61 @@
|
||||
# PaddleClas rv1126 开发板 C++ 部署示例
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PaddleClas 量化模型在 rv1126 上的部署推理加速。
|
||||
|
||||
## 1. 部署环境准备
|
||||
在部署前，需确认以下两个步骤
|
||||
- 1. 在部署前，需自行编译基于rv的预测库，参考文档[rv1126部署环境编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#自行编译安装)
|
||||
|
||||
## 2. 量化模型准备
|
||||
1. 需要特别注意的是，在 rv1126 上部署的模型需要是量化后的模型. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户也可以使用 FastDeploy 提供的[一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression/)，自行进行模型量化, 并使用产出的量化模型进行部署。(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从 FP32 模型文件夹下复制此 yaml 文件到量化后的模型文件夹内即可.)
|
||||
|
||||
## 3. 在 RV1126 上部署量化后的 ResNet50_Vd 分类模型
|
||||
请按照以下步骤完成在 RV1126 上部署 ResNet50_Vd 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/rv1126.md#基于-paddlelite-的-fastdeploy-交叉编译库编译)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ PaddleClas/deploy/fastdeploy/rockchip/rv1126/cpp/
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rockchip/rv1126/cpp/
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/rockchip/rv1126/cpp/
|
||||
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
cd PaddleClas/deploy/fastdeploy/rockchip/rv1126/cpp/
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=armhf ..
|
||||
make -j8
|
||||
make install
|
||||
# 成功编译之后，会生成 install 文件夹，里面有一个运行 demo 和部署所需的库
|
||||
```
|
||||
|
||||
5. 基于 adb 工具部署 ResNet50 分类模型到 Rockchip RV1126，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd PaddleClas/deploy/fastdeploy/rockchip/rv1126/cpp/build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
部署成功后运行结果如下：
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
@@ -12,6 +12,7 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
#include <string>
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
@@ -19,7 +20,7 @@ const char sep = '\\';
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
void InitAndInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "inference.pdmodel";
|
||||
auto params_file = model_dir + sep + "inference.pdiparams";
|
||||
auto config_file = model_dir + sep + "inference_cls.yaml";
|
||||
@@ -41,10 +42,9 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/quant_model "
|
||||
"path/to/image "
|
||||
@@ -1,12 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Quantification Model Deployment on RV1126
|
||||
FastDeploy currently supports the deployment of PaddleClas quantification models to RV1126 based on Paddle Lite.
|
||||
|
||||
For model quantization and download of quantized models, refer to [Model Quantization](../quantize/README.md).
|
||||
|
||||
|
||||
## Detailed Deployment Tutorials
|
||||
|
||||
Only C++ deployment is supported on RV1126.
|
||||
|
||||
- [C++ Deployment](cpp)
|
||||
@@ -1,12 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 量化模型在 RV1126 上的部署
|
||||
目前 FastDeploy 已经支持基于 Paddle Lite 部署 PaddleClas 量化模型到 RV1126 上。
|
||||
|
||||
模型的量化和量化模型的下载请参考：[模型量化](../quantize/README.md)
|
||||
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
在 RV1126 上只支持 C++ 的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -1,57 +0,0 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas RV1126 Development Board C++ Deployment Example
|
||||
`infer.cc` in this directory can help you quickly complete the inference acceleration of PaddleClas quantization model deployment on RV1126.
|
||||
|
||||
## Deployment Preparations
|
||||
### FastDeploy Cross-compile Environment Preparations
|
||||
1. For the software and hardware environment, and the cross-compile environment, please refer to [Preparations for FastDeploy Cross-compile environment](../../../../../../docs/en/build_and_install/rv1126.md#Cross-compilation-environment-construction).
|
||||
|
||||
### Model Preparations
|
||||
1. You can directly use the quantized model provided by FastDeploy for deployment.
|
||||
2. You can use [one-click automatical compression tool](../../../../../../tools/common_tools/auto_compression/) provided by FastDeploy to quantize model by yourself, and use the generated quantized model for deployment.(Note: The quantized classification model still needs the inference_cls.yaml file in the FP32 model folder. Self-quantized model folder does not contain this yaml file, you can copy it from the FP32 model folder to the quantized model folder.)
|
||||
|
||||
For more information, please refer to [Model Quantization](../../quantize/README.md).
|
||||
|
||||
## Deploying the Quantized ResNet50_Vd Segmentation model on RV1126
|
||||
Please follow these steps to complete the deployment of the ResNet50_Vd quantization model on RV1126.
|
||||
1. Cross-compile the FastDeploy library as described in [Cross-compile FastDeploy](../../../../../../docs/en/build_and_install/rv1126.md#FastDeploy-cross-compilation-library-compilation-based-on-Paddle-Lite).
|
||||
|
||||
2. Copy the compiled library to the current directory. You can run this line:
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
```
|
||||
|
||||
3. Download the model and example images required for deployment in current path.
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. Compile the deployment example. You can run the following lines:
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=armhf ..
|
||||
make -j8
|
||||
make install
|
||||
# After success, an install folder will be created with a running demo and libraries required for deployment.
|
||||
```
|
||||
|
||||
5. Deploy the ResNet50 segmentation model to Rockchip RV1126 based on adb. You can run the following lines:
|
||||
```bash
|
||||
# Go to the install directory.
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/build/install/
|
||||
# The following line represents: bash run_with_adb.sh, demo needed to run, model path, image path, DEVICE ID.
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
The output is:
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
|
||||
Please note that the model deployed on RV1126 needs to be quantized. You can refer to [Model Quantization](../../../../../../docs/en/quantize.md).
|
||||
@@ -1,57 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas RV1126 开发板 C++ 部署示例
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PaddleClas 量化模型在 RV1126 上的部署推理加速。
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy 交叉编译环境准备
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](../../../../../../docs/cn/build_and_install/rv1126.md#交叉编译环境搭建)
|
||||
|
||||
### 量化模型准备
|
||||
1. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户可以使用 FastDeploy 提供的[一键模型自动化压缩工具](../../../../../../tools/common_tools/auto_compression/)，自行进行模型量化, 并使用产出的量化模型进行部署。(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从 FP32 模型文件夹下复制此 yaml 文件到量化后的模型文件夹内即可.)
|
||||
|
||||
更多量化相关相关信息可查阅[模型量化](../../quantize/README.md)
|
||||
|
||||
## 在 RV1126 上部署量化后的 ResNet50_Vd 分类模型
|
||||
请按照以下步骤完成在 RV1126 上部署 ResNet50_Vd 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](../../../../../../docs/cn/build_and_install/rv1126.md#基于-paddlelite-的-fastdeploy-交叉编译库编译)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar
|
||||
tar -xvf resnet50_vd_ptq.tar
|
||||
cp -r resnet50_vd_ptq models
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp -r ILSVRC2012_val_00000010.jpeg images
|
||||
```
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=armhf ..
|
||||
make -j8
|
||||
make install
|
||||
# 成功编译之后，会生成 install 文件夹，里面有一个运行 demo 和部署所需的库
|
||||
```
|
||||
|
||||
5. 基于 adb 工具部署 ResNet50 分类模型到 Rockchip RV1126，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/rv1126/cpp/build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo resnet50_vd_ptq ILSVRC2012_val_00000010.jpeg $DEVICE_ID
|
||||
```
|
||||
|
||||
部署成功后运行结果如下：
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/200767389-26519e50-9e4f-4fe1-8d52-260718f73476.png">
|
||||
|
||||
需要特别注意的是，在 RV1126 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../../../../docs/cn/quantize.md)
|
||||
@@ -1,51 +1,56 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Service Deployment Example
|
||||
# PaddleClas服务化部署示例
|
||||
|
||||
Before the service deployment, please confirm
|
||||
PaddleClas 服务化部署示例是利用FastDeploy Serving搭建的服务化部署示例。FastDeploy Serving是基于Triton Inference Server框架封装的适用于高并发、高吞吐量请求的服务化部署框架，是一套可用于实际生产的完备且性能卓越的服务化部署框架.
|
||||
|
||||
- 1. Refer to [FastDeploy Service Deployment](../../../../../serving/README.md) for software and hardware environment requirements and image pull commands.
|
||||
## 1. 部署环境准备
|
||||
在服务化部署前，需确认服务化镜像的软硬件环境要求和镜像拉取命令，请参考[FastDeploy服务化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/README_CN.md)
|
||||
|
||||
|
||||
## Start the Service
|
||||
## 2. 启动服务
|
||||
|
||||
```bash
|
||||
# Download the example code for deployment
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/serving
|
||||
|
||||
# Download ResNet50_vd model files and test images
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/serving
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Put the configuration file into the preprocessing directory
|
||||
# 将配置文件放入预处理目录
|
||||
mv ResNet50_vd_infer/inference_cls.yaml models/preprocess/1/inference_cls.yaml
|
||||
|
||||
# Place the model under models/runtime/1 and rename them to model.pdmodel和model.pdiparams
|
||||
# 将模型放入 models/runtime/1目录下, 并重命名为model.pdmodel和model.pdiparams
|
||||
mv ResNet50_vd_infer/inference.pdmodel models/runtime/1/model.pdmodel
|
||||
mv ResNet50_vd_infer/inference.pdiparams models/runtime/1/model.pdiparams
|
||||
|
||||
# Pull the fastdeploy image (x.y.z represent the image version. Refer to the serving document to replace them with numbers)
|
||||
# GPU image
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU image
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# Run the container named fd_serving and mount it in the /serving directory of the container
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# Start the service (The CUDA_VISIBLE_DEVICES environment variable is not set, which entitles the scheduling authority of all GPU cards)
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **Attention**:
|
||||
>> **注意**:
|
||||
|
||||
>> To pull images from other hardware, refer to [Service Deployment Master Document](../../../../../serving/README.md)
|
||||
>> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README_CN.md)
|
||||
|
||||
>> If "Address already in use" appears when running fastdeployserver to start the service, use `--grpc-port` to specify the port number and change the request port number in the client demo.
|
||||
>> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改客户端示例中的请求端口号.
|
||||
|
||||
>> Other startup parameters can be checked by fastdeployserver --help
|
||||
>> 其他启动参数可以使用 fastdeployserver --help 查看
|
||||
|
||||
Successful service start brings the following output:
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
@@ -54,40 +59,46 @@ I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0
|
||||
```
|
||||
|
||||
|
||||
## Client Request
|
||||
## 3. 客户端请求
|
||||
|
||||
Execute the following command in the physical machine to send the grpc request and output the result
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
# Download test images
|
||||
#下载测试图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Install client dependencies
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient\[all\]
|
||||
|
||||
# Send the request
|
||||
# 发送请求
|
||||
python3 paddlecls_grpc_client.py
|
||||
```
|
||||
|
||||
The result is returned in json format and printed after sending the request:
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
output_name: CLAS_RESULT
|
||||
{'label_ids': [153], 'scores': [0.6862289905548096]}
|
||||
```
|
||||
|
||||
## Configuration Change
|
||||
## 4. 配置修改
|
||||
|
||||
The current default configuration runs the TensorRT engine on GPU. If you want to run it on CPU or other inference engines, please modify the configuration in `models/runtime/config.pbtxt`. Refer to [Configuration Document](../../../../../serving/docs/EN/model_configuration-en.md) for more information.
|
||||
当前默认配置在GPU上运行TensorRT引擎， 如果要在CPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](../../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
## Use VisualDL for serving deployment visualization
|
||||
## 5. 使用VisualDL进行可视化部署
|
||||
|
||||
You can use VisualDL for [serving deployment visualization](../../../../../serving/docs/EN/vdl_management-en.md) , the above model preparation, deployment, configuration modification and client request operations can all be performed based on VisualDL.
|
||||
可以[使用 VisualDL 进行 Serving 可视化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/vdl_management.md)，上述启动服务、配置修改以及客户端请求的操作都可以基于VisualDL进行。
|
||||
|
||||
The serving deployment of PaddleClas by VisualDL only needs the following three steps:
|
||||
通过VisualDL的可视化界面对PaddleClas进行服务化部署只需要如下三步：
|
||||
```text
|
||||
1. Load the model repository: ./vision/classification/paddleclas/serving/models
|
||||
2. Download the model resource file: click the runtime model, click the version number 1 to add the pre-training model, and select the image classification model ResNet50_vd to download.
|
||||
3. Start the service: Click the "launch server" button and input the launch parameters.
|
||||
1. 载入模型库：PaddleClas/deploy/fastdeploy/serving/models
|
||||
2. 下载模型资源文件：点击runtime模型，点击版本号1添加预训练模型，选择图像分类模型ResNet50_vd进行下载。
|
||||
3. 启动服务：点击启动服务按钮，输入启动参数。
|
||||
```
|
||||
<p align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22424850/211708702-828d8ad8-4e85-457f-9c62-12f53fc81853.gif" width="100%"/>
|
||||
</p>
|
||||
|
||||
## 6. 常见问题
|
||||
- [如何编写客户端 HTTP/GRPC 请求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/client.md)
|
||||
- [如何编译服务化部署镜像](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/compile.md)
|
||||
- [服务化部署原理及动态Batch介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/demo.md)
|
||||
- [模型仓库介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/model_repository.md)
|
||||
|
||||
@@ -1,93 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas 服务化部署示例
|
||||
|
||||
在服务化部署前，需确认
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
|
||||
|
||||
## 启动服务
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/serving
|
||||
|
||||
# 下载ResNet50_vd模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar -xvf ResNet50_vd_infer.tgz
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 将配置文件放入预处理目录
|
||||
mv ResNet50_vd_infer/inference_cls.yaml models/preprocess/1/inference_cls.yaml
|
||||
|
||||
# 将模型放入 models/runtime/1目录下, 并重命名为model.pdmodel和model.pdiparams
|
||||
mv ResNet50_vd_infer/inference.pdmodel models/runtime/1/model.pdmodel
|
||||
mv ResNet50_vd_infer/inference.pdiparams models/runtime/1/model.pdiparams
|
||||
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **注意**:
|
||||
|
||||
>> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README_CN.md)
|
||||
|
||||
>> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改客户端示例中的请求端口号.
|
||||
|
||||
>> 其他启动参数可以使用 fastdeployserver --help 查看
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
#下载测试图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient\[all\]
|
||||
|
||||
# 发送请求
|
||||
python3 paddlecls_grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
output_name: CLAS_RESULT
|
||||
{'label_ids': [153], 'scores': [0.6862289905548096]}
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在GPU上运行TensorRT引擎， 如果要在CPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](../../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
## 使用VisualDL进行可视化部署
|
||||
|
||||
可以使用VisualDL进行[Serving可视化部署](../../../../../serving/docs/zh_CN/vdl_management.md)，上述启动服务、配置修改以及客户端请求的操作都可以基于VisualDL进行。
|
||||
|
||||
通过VisualDL的可视化界面对PaddleClas进行服务化部署只需要如下三步：
|
||||
```text
|
||||
1. 载入模型库：./vision/classification/paddleclas/serving/models
|
||||
2. 下载模型资源文件：点击runtime模型，点击版本号1添加预训练模型，选择图像分类模型ResNet50_vd进行下载。
|
||||
3. 启动服务：点击启动服务按钮，输入启动参数。
|
||||
```
|
||||
<p align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22424850/211708702-828d8ad8-4e85-457f-9c62-12f53fc81853.gif" width="100%"/>
|
||||
</p>
|
||||
@@ -1,29 +1,29 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleDetection SOPHGO Deployment Example
|
||||
# PaddleClas SOPHGO部署示例
|
||||
|
||||
## Supporting Model List
|
||||
## 1. 说明
|
||||
PaddleClas支持通过FastDeploy在SOPHGO上部署相关模型.
|
||||
|
||||
Currently FastDeploy supports the following model deployment: [ResNet series model](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/en/models/ResNet_and_vd_en.md).
|
||||
## 2. 支持模型列表
|
||||
|
||||
## Preparing ResNet Model Deployment and Conversion
|
||||
目前FastDeploy支持的如下模型的部署[ResNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
Before deploying SOPHGO-TPU model, you need to first convert Paddle model to bmodel. Specific steps are as follows:
|
||||
- Convert Paddle dynamic map model to ONNX model, please refer to [Paddle2ONNX model conversion](https://github.com/PaddlePaddle/Paddle2ONNX/tree/develop/model_zoo/classification).
|
||||
- For the process of converting ONNX model to bmodel, please refer to [TPU-MLIR](https://github.com/sophgo/tpu-mlir).
|
||||
## 3.准备ResNet部署模型以及转换模型
|
||||
|
||||
## Model Converting Example
|
||||
SOPHGO-TPU部署模型前需要将Paddle模型转换成bmodel模型，具体步骤如下:
|
||||
- Paddle动态图模型转换为ONNX模型，请参考[Paddle2ONNX模型转换](https://github.com/PaddlePaddle/Paddle2ONNX/tree/develop/model_zoo/classification)
|
||||
- ONNX模型转换bmodel模型的过程，请参考[TPU-MLIR](https://github.com/sophgo/tpu-mlir)。
|
||||
|
||||
Here we take [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) as an example to show you how to convert Paddle model to SOPHGO-TPU model.
|
||||
下面以[ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz)为例子,教大家如何转换Paddle模型到SOPHGO-TPU模型。
|
||||
|
||||
## Export ONNX Model
|
||||
### 3.1 导出ONNX模型
|
||||
|
||||
### Download and Unzip Paddle ResNet50_vd Static Map Model
|
||||
#### 3.1.1 下载Paddle ResNet50_vd静态图模型并解压
|
||||
```shell
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar xvf ResNet50_vd_infer.tgz
|
||||
```
|
||||
|
||||
### Convert Static Map Model to ONNX Model, note that the save_file here aligns with the zip name
|
||||
#### 3.1.2 静态图转ONNX模型，注意，这里的save_file请和压缩包名对齐
|
||||
```shell
|
||||
paddle2onnx --model_dir ResNet50_vd_infer \
|
||||
--model_filename inference.pdmodel \
|
||||
@@ -31,32 +31,34 @@ paddle2onnx --model_dir ResNet50_vd_infer \
|
||||
--save_file ResNet50_vd_infer.onnx \
|
||||
--enable_dev_version True
|
||||
```
|
||||
### Export bmodel
|
||||
|
||||
Take converting BM1684x model to bmodel as an example. You need to download [TPU-MLIR](https://github.com/sophgo/tpu-mlir) project. For the process of installation, please refer to [TPU-MLIR Document](https://github.com/sophgo/tpu-mlir/blob/master/README.md).
|
||||
### 1. Installation
|
||||
### 3.2 导出bmodel模型
|
||||
|
||||
以转化BM1684x的bmodel模型为例子，我们需要下载[TPU-MLIR](https://github.com/sophgo/tpu-mlir)工程，安装过程具体参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。
|
||||
|
||||
#### 3.2.1 安装
|
||||
``` shell
|
||||
docker pull sophgo/tpuc_dev:latest
|
||||
|
||||
# myname1234 is just an example, you can customize your own name.
|
||||
# myname1234是一个示例，也可以设置其他名字
|
||||
docker run --privileged --name myname1234 -v $PWD:/workspace -it sophgo/tpuc_dev:latest
|
||||
|
||||
source ./envsetup.sh
|
||||
./build.sh
|
||||
```
|
||||
|
||||
### 2. Convert ONNX model to bmodel
|
||||
#### 3.2.2 ONNX模型转换为bmodel模型
|
||||
``` shell
|
||||
mkdir ResNet50_vd_infer && cd ResNet50_vd_infer
|
||||
|
||||
# Put the test image in this file, and put the ResNet50_vd_infer.onnx into this folder.
|
||||
# 在该文件中放入测试图片，同时将上一步转换好的ResNet50_vd_infer.onnx放入该文件夹中
|
||||
cp -rf ${REGRESSION_PATH}/dataset/COCO2017 .
|
||||
cp -rf ${REGRESSION_PATH}/image .
|
||||
# Put in the onnx model file ResNet50_vd_infer.onnx.
|
||||
# 放入onnx模型文件ResNet50_vd_infer.onnx
|
||||
|
||||
mkdir workspace && cd workspace
|
||||
|
||||
# Convert ONNX model to mlir model, the parameter --output_names can be viewed via NETRON.
|
||||
# 将ONNX模型转换为mlir模型，其中参数--output_names可以通过NETRON查看
|
||||
model_transform.py \
|
||||
--model_name ResNet50_vd_infer \
|
||||
--model_def ../ResNet50_vd_infer.onnx \
|
||||
@@ -70,7 +72,7 @@ model_transform.py \
|
||||
--test_result ResNet50_vd_infer_top_outputs.npz \
|
||||
--mlir ResNet50_vd_infer.mlir
|
||||
|
||||
# Convert mlir model to BM1684x F32 bmodel.
|
||||
# 将mlir模型转换为BM1684x的F32 bmodel模型
|
||||
model_deploy.py \
|
||||
--mlir ResNet50_vd_infer.mlir \
|
||||
--quantize F32 \
|
||||
@@ -79,7 +81,8 @@ model_deploy.py \
|
||||
--test_reference ResNet50_vd_infer_top_outputs.npz \
|
||||
--model ResNet50_vd_infer_1684x_f32.bmodel
|
||||
```
|
||||
The final bmodel, ResNet50_vd_infer_1684x_f32.bmodel, can run on BM1684x. If you want to further accelerate the model, you can convert ONNX model to INT8 bmodel. For details, please refer to [TPU-MLIR Document](https://github.com/sophgo/tpu-mlir/blob/master/README.md).
|
||||
最终获得可以在BM1684x上能够运行的bmodel模型ResNet50_vd_infer_1684x_f32.bmodel。如果需要进一步对模型进行加速，可以将ONNX模型转换为INT8 bmodel，具体步骤参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。
|
||||
|
||||
## Other Documents
|
||||
- [Cpp Deployment](./cpp)
|
||||
## 4. 其他链接
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
|
||||
@@ -1,85 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleDetection SOPHGO部署示例
|
||||
|
||||
## 支持模型列表
|
||||
|
||||
目前FastDeploy支持的如下模型的部署[ResNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md)
|
||||
|
||||
## 准备ResNet部署模型以及转换模型
|
||||
|
||||
SOPHGO-TPU部署模型前需要将Paddle模型转换成bmodel模型，具体步骤如下:
|
||||
- Paddle动态图模型转换为ONNX模型，请参考[Paddle2ONNX模型转换](https://github.com/PaddlePaddle/Paddle2ONNX/tree/develop/model_zoo/classification)
|
||||
- ONNX模型转换bmodel模型的过程，请参考[TPU-MLIR](https://github.com/sophgo/tpu-mlir)。
|
||||
|
||||
## 模型转换example
|
||||
|
||||
下面以[ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz)为例子,教大家如何转换Paddle模型到SOPHGO-TPU模型。
|
||||
|
||||
## 导出ONNX模型
|
||||
|
||||
### 下载Paddle ResNet50_vd静态图模型并解压
|
||||
```shell
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz
|
||||
tar xvf ResNet50_vd_infer.tgz
|
||||
```
|
||||
|
||||
### 静态图转ONNX模型，注意，这里的save_file请和压缩包名对齐
|
||||
```shell
|
||||
paddle2onnx --model_dir ResNet50_vd_infer \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--save_file ResNet50_vd_infer.onnx \
|
||||
--enable_dev_version True
|
||||
```
|
||||
### 导出bmodel模型
|
||||
|
||||
以转化BM1684x的bmodel模型为例子，我们需要下载[TPU-MLIR](https://github.com/sophgo/tpu-mlir)工程，安装过程具体参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。
|
||||
### 1. 安装
|
||||
``` shell
|
||||
docker pull sophgo/tpuc_dev:latest
|
||||
|
||||
# myname1234是一个示例，也可以设置其他名字
|
||||
docker run --privileged --name myname1234 -v $PWD:/workspace -it sophgo/tpuc_dev:latest
|
||||
|
||||
source ./envsetup.sh
|
||||
./build.sh
|
||||
```
|
||||
|
||||
### 2. ONNX模型转换为bmodel模型
|
||||
``` shell
|
||||
mkdir ResNet50_vd_infer && cd ResNet50_vd_infer
|
||||
|
||||
# 在该文件中放入测试图片，同时将上一步转换好的ResNet50_vd_infer.onnx放入该文件夹中
|
||||
cp -rf ${REGRESSION_PATH}/dataset/COCO2017 .
|
||||
cp -rf ${REGRESSION_PATH}/image .
|
||||
# 放入onnx模型文件ResNet50_vd_infer.onnx
|
||||
|
||||
mkdir workspace && cd workspace
|
||||
|
||||
# 将ONNX模型转换为mlir模型，其中参数--output_names可以通过NETRON查看
|
||||
model_transform.py \
|
||||
--model_name ResNet50_vd_infer \
|
||||
--model_def ../ResNet50_vd_infer.onnx \
|
||||
--input_shapes [[1,3,224,224]] \
|
||||
--mean 0.0,0.0,0.0 \
|
||||
--scale 0.0039216,0.0039216,0.0039216 \
|
||||
--keep_aspect_ratio \
|
||||
--pixel_format rgb \
|
||||
--output_names save_infer_model/scale_0.tmp_1 \
|
||||
--test_input ../image/dog.jpg \
|
||||
--test_result ResNet50_vd_infer_top_outputs.npz \
|
||||
--mlir ResNet50_vd_infer.mlir
|
||||
|
||||
# 将mlir模型转换为BM1684x的F32 bmodel模型
|
||||
model_deploy.py \
|
||||
--mlir ResNet50_vd_infer.mlir \
|
||||
--quantize F32 \
|
||||
--chip bm1684x \
|
||||
--test_input ResNet50_vd_infer_in_f32.npz \
|
||||
--test_reference ResNet50_vd_infer_top_outputs.npz \
|
||||
--model ResNet50_vd_infer_1684x_f32.bmodel
|
||||
```
|
||||
最终获得可以在BM1684x上能够运行的bmodel模型ResNet50_vd_infer_1684x_f32.bmodel。如果需要进一步对模型进行加速，可以将ONNX模型转换为INT8 bmodel，具体步骤参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。
|
||||
|
||||
## 其他链接
|
||||
- [Cpp部署](./cpp)
|
||||
@@ -1,62 +1,67 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas C++ Deployment Example
|
||||
# PaddleClas C++部署示例
|
||||
|
||||
`infer.cc` in this directory provides a quick example of accelerated deployment of the ResNet50_vd model on SOPHGO BM1684x.
|
||||
本目录下提供`infer.cc`快速完成ResNet50_vd模型在SOPHGO BM1684x板子上加速部署的示例。
|
||||
|
||||
Before deployment, the following two steps need to be confirmed:
|
||||
## 1. 部署环境准备
|
||||
在部署前，需自行编译基于SOPHGO硬件的预测库，参考文档[SOPHGO硬件部署环境](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#算能硬件部署环境)
|
||||
|
||||
1. Hardware and software environment meets the requirements.
|
||||
2. Compile the FastDeploy repository from scratch according to the development environment.
|
||||
## 2. 生成基本目录文件
|
||||
|
||||
For the above steps, please refer to [How to Build SOPHGO Deployment Environment](../../../../../../docs/en/build_and_install/sophgo.md).
|
||||
|
||||
## Generate Basic Directory Files
|
||||
|
||||
The routine consists of the following parts:
|
||||
该例程由以下几个部分组成
|
||||
```text
|
||||
.
|
||||
├── CMakeLists.txt
|
||||
├── build # Compile Folder
|
||||
├── image # Folder for images
|
||||
├── build # 编译文件夹
|
||||
├── image # 存放图片的文件夹
|
||||
├── infer.cc
|
||||
├── preprocess_config.yaml # Preprocessing configuration sample file.
|
||||
└── model # Folder for models
|
||||
├── preprocess_config.yaml #示例前处理配置文件
|
||||
└── model # 存放模型文件的文件夹
|
||||
```
|
||||
## 3. 部署实例
|
||||
|
||||
## Compile
|
||||
|
||||
### Compile and Copy SDK to folder thirdpartys
|
||||
### 3.1 编译并拷贝SDK到thirdpartys文件夹
|
||||
|
||||
Please refer to [How to Build SOPHGO Deployment Environment](../../../../../../docs/en/build_and_install/sophgo.md) to compile SDK.After compiling, the fastdeploy-0.0.3 directory will be created in the build directory.
|
||||
请参考[SOPHGO部署库编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/sophgo.md)仓库编译SDK，编译完成后，将在build目录下生成fastdeploy-x.x.x目录.
|
||||
|
||||
### Copy model and configuration files to folder Model
|
||||
Convert Paddle model to SOPHGO bmodel model. For the conversion steps, please refer to [Document](../README.md).
|
||||
Please copy the converted SOPHGO bmodel to folder model.
|
||||
Copy the preprocessing configuration file to folder model as well.
|
||||
### 3.2 拷贝模型文件，以及配置文件至model文件夹
|
||||
将Paddle模型转换为SOPHGO bmodel模型，转换步骤参考[文档](../README.md)
|
||||
将转换后的SOPHGO bmodel模型文件拷贝至model中
|
||||
将前处理配置文件也拷贝到model中
|
||||
```bash
|
||||
cp preprocess_config.yaml ./model
|
||||
```
|
||||
|
||||
### Prepare Test Images to folder image
|
||||
### 3.3 准备测试图片至image文件夹
|
||||
```bash
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp ILSVRC2012_val_00000010.jpeg ./images
|
||||
```
|
||||
|
||||
### Compile example
|
||||
### 3.4 编译example
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/sophgo/cpp
|
||||
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/sophgo/cpp
|
||||
|
||||
cd build
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-0.0.3
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-x.x.x
|
||||
make
|
||||
```
|
||||
|
||||
## Running Routines
|
||||
### 3.5 运行例程
|
||||
|
||||
```bash
|
||||
./infer_demo model images/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Model Conversion](../)
|
||||
## 4. 其它文档
|
||||
- [ResNet50_vd python部署](../python)
|
||||
- [转换ResNet50_vd SOPHGO模型文档](../README.md)
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成ResNet50_vd模型在SOPHGO BM1684x板子上加速部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤:
|
||||
|
||||
1. 软硬件环境满足要求
|
||||
2. 根据开发环境，从头编译FastDeploy仓库
|
||||
|
||||
以上步骤请参考[SOPHGO部署库编译](../../../../../../docs/cn/build_and_install/sophgo.md)实现
|
||||
|
||||
## 生成基本目录文件
|
||||
|
||||
该例程由以下几个部分组成
|
||||
```text
|
||||
.
|
||||
├── CMakeLists.txt
|
||||
├── build # 编译文件夹
|
||||
├── image # 存放图片的文件夹
|
||||
├── infer.cc
|
||||
├── preprocess_config.yaml #示例前处理配置文件
|
||||
└── model # 存放模型文件的文件夹
|
||||
```
|
||||
|
||||
## 编译
|
||||
|
||||
### 编译并拷贝SDK到thirdpartys文件夹
|
||||
|
||||
请参考[SOPHGO部署库编译](../../../../../../docs/cn/build_and_install/sophgo.md)仓库编译SDK，编译完成后，将在build目录下生成fastdeploy-0.0.3目录.
|
||||
|
||||
### 拷贝模型文件，以及配置文件至model文件夹
|
||||
将Paddle模型转换为SOPHGO bmodel模型，转换步骤参考[文档](../README.md)
|
||||
将转换后的SOPHGO bmodel模型文件拷贝至model中
|
||||
将前处理配置文件也拷贝到model中
|
||||
```bash
|
||||
cp preprocess_config.yaml ./model
|
||||
```
|
||||
|
||||
### 准备测试图片至image文件夹
|
||||
```bash
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
cp ILSVRC2012_val_00000010.jpeg ./images
|
||||
```
|
||||
|
||||
### 编译example
|
||||
|
||||
```bash
|
||||
cd build
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-0.0.3
|
||||
make
|
||||
```
|
||||
|
||||
## 运行例程
|
||||
|
||||
```bash
|
||||
./infer_demo model images/ILSVRC2012_val_00000010.jpeg
|
||||
```
|
||||
|
||||
|
||||
- [模型介绍](../../)
|
||||
- [模型转换](../)
|
||||
@@ -12,6 +12,7 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
#include <string>
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
@@ -19,7 +20,7 @@ const char sep = '\\';
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
void InitAndInfer(const std::string &model_dir, const std::string &image_file) {
|
||||
auto model_file = model_dir + sep + "resnet50_1684x_f32.bmodel";
|
||||
auto params_file = model_dir + sep + "";
|
||||
auto config_file = model_dir + sep + "preprocess_config.yaml";
|
||||
@@ -41,10 +42,9 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 3) {
|
||||
std::cout << "Usage: infer_demo path/to/model "
|
||||
"path/to/image "
|
||||
|
||||
@@ -1,33 +1,39 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleClas Python Deployment Example
|
||||
# PaddleClas Python部署示例
|
||||
本目录下提供`infer.py`快速完成 ResNet50_vd 在SOPHGO TPU上部署的示例.
|
||||
|
||||
Before deployment, the following step need to be confirmed:
|
||||
|
||||
- 1. Hardware and software environment meets the requirements. Please refer to [FastDeploy Environment Requirement](../../../../../../docs/en/build_and_install/sophgo.md).
|
||||
## 1. 部署环境准备
|
||||
|
||||
`infer.py` in this directory provides a quick example of deployment of the ResNet50_vd model on SOPHGO TPU. Please run the following script:
|
||||
在部署前，需自行编译基于算能硬件的FastDeploy python wheel包并安装，参考文档[算能硬件部署环境](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install#算能硬件部署环境)
|
||||
|
||||
## 2.运行部署示例
|
||||
```bash
|
||||
# Download the sample deployment code.
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/sophgo/python
|
||||
|
||||
# Download images.
|
||||
# 如果您希望从PaddleClas下载示例代码，请运行
|
||||
git clone https://github.com/PaddlePaddle/PaddleClas.git
|
||||
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
|
||||
git checkout develop
|
||||
cd PaddleClas/deploy/fastdeploy/sophgo/python
|
||||
|
||||
# 下载图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Inference. Need to manually set the model, configuration file and image path used for inference.
|
||||
# 推理转换好的模型
|
||||
# 手动设置推理使用的模型、配置文件和图片路径
|
||||
python3 infer.py --auto False --model_file ./bmodel/resnet50_1684x_f32.bmodel --config_file ResNet50_vd_infer/inference_cls.yaml --image ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# Automatic completion of downloading data - model compilation - inference, no need to set up model, configuration file and image paths.
|
||||
# 自动完成下载数据-模型编译-推理，不需要设置模型、配置文件和图片路径
|
||||
python3 infer.py --auto True --model '' --config_file '' --image ''
|
||||
|
||||
# The returned result.
|
||||
# 运行完成后返回结果如下所示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
```
|
||||
|
||||
## Other Documents
|
||||
- [ResNet50_vd C++ Deployment](../cpp)
|
||||
- [Converting ResNet50_vd SOPHGO model](../README.md)
|
||||
## 4. 其它文档
|
||||
- [ResNet50_vd C++部署](../python)
|
||||
- [转换ResNet50_vd SOPHGO模型文档](../README.md)
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleClas Python部署示例
|
||||
|
||||
在部署前，需确认以下步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../../docs/cn/build_and_install/sophgo.md)
|
||||
|
||||
本目录下提供`infer.py`快速完成 ResNet50_vd 在SOPHGO TPU上部署的示例。执行如下脚本即可完成
|
||||
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/classification/paddleclas/sophgo/python
|
||||
|
||||
# 下载图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 手动设置推理使用的模型、配置文件和图片路径
|
||||
python3 infer.py --auto False --model_file ./bmodel/resnet50_1684x_f32.bmodel --config_file ResNet50_vd_infer/inference_cls.yaml --image ILSVRC2012_val_00000010.jpeg
|
||||
|
||||
# 自动完成下载数据-模型编译-推理，不需要设置模型、配置文件和图片路径
|
||||
python3 infer.py --auto True --model '' --config_file '' --image ''
|
||||
|
||||
|
||||
# 运行完成后返回结果如下所示
|
||||
ClassifyResult(
|
||||
label_ids: 153,
|
||||
scores: 0.684570,
|
||||
)
|
||||
```
|
||||
|
||||
## 其它文档
|
||||
- [ResNet50_vd C++部署](../cpp)
|
||||
- [转换ResNet50_vd SOPHGO模型文档](../README.md)
|
||||
31
examples/vision/classification/paddleclas/sophgo/python/infer.py
Executable file → Normal file
31
examples/vision/classification/paddleclas/sophgo/python/infer.py
Executable file → Normal file
@@ -3,13 +3,18 @@ import cv2
|
||||
import os
|
||||
from subprocess import run
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
import argparse
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--auto", required=True, help="Auto download, convert, compile and infer if True")
|
||||
parser.add_argument(
|
||||
"--auto",
|
||||
required=True,
|
||||
help="Auto download, convert, compile and infer if True")
|
||||
parser.add_argument("--model", required=True, help="Path of bmodel")
|
||||
parser.add_argument("--config_file", required=True, help="Path of config file")
|
||||
parser.add_argument(
|
||||
"--config_file", required=True, help="Path of config file")
|
||||
parser.add_argument(
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
@@ -17,6 +22,7 @@ def parse_arguments():
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def download():
|
||||
cmd_str = 'wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz'
|
||||
jpg_str = 'wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg'
|
||||
@@ -27,27 +33,33 @@ def download():
|
||||
run(jpg_str, shell=True)
|
||||
run(tar_str, shell=True)
|
||||
|
||||
|
||||
def paddle2onnx():
|
||||
cmd_str = 'paddle2onnx --model_dir ResNet50_vd_infer \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--save_file ResNet50_vd_infer.onnx \
|
||||
--enable_dev_version True'
|
||||
|
||||
print(cmd_str)
|
||||
run(cmd_str, shell=True)
|
||||
|
||||
|
||||
def mlir_prepare():
|
||||
mlir_path = os.getenv("MODEL_ZOO_PATH")
|
||||
mlir_path = mlir_path[:-13]
|
||||
cmd_list = ['mkdir ResNet50',
|
||||
'cp -rf ' + os.path.join(mlir_path, 'regression/dataset/COCO2017/') + ' ./ResNet50',
|
||||
'cp -rf ' + os.path.join(mlir_path, 'regression/image/') + ' ./ResNet50',
|
||||
'cp ResNet50_vd_infer.onnx ./ResNet50/',
|
||||
'mkdir ./ResNet50/workspace']
|
||||
cmd_list = [
|
||||
'mkdir ResNet50', 'cp -rf ' + os.path.join(
|
||||
mlir_path, 'regression/dataset/COCO2017/') + ' ./ResNet50',
|
||||
'cp -rf ' + os.path.join(mlir_path,
|
||||
'regression/image/') + ' ./ResNet50',
|
||||
'cp ResNet50_vd_infer.onnx ./ResNet50/', 'mkdir ./ResNet50/workspace'
|
||||
]
|
||||
for str in cmd_list:
|
||||
print(str)
|
||||
run(str, shell=True)
|
||||
|
||||
|
||||
def onnx2mlir():
|
||||
cmd_str = 'model_transform.py \
|
||||
--model_name ResNet50_vd_infer \
|
||||
@@ -61,11 +73,13 @@ def onnx2mlir():
|
||||
--test_input ../image/dog.jpg \
|
||||
--test_result ./ResNet50_vd_infer_top_outputs.npz \
|
||||
--mlir ./ResNet50_vd_infer.mlir'
|
||||
|
||||
print(cmd_str)
|
||||
os.chdir('./ResNet50/workspace/')
|
||||
run(cmd_str, shell=True)
|
||||
os.chdir('../../')
|
||||
|
||||
|
||||
def mlir2bmodel():
|
||||
cmd_str = 'model_deploy.py \
|
||||
--mlir ./ResNet50_vd_infer.mlir \
|
||||
@@ -74,6 +88,7 @@ def mlir2bmodel():
|
||||
--test_input ./ResNet50_vd_infer_in_f32.npz \
|
||||
--test_reference ./ResNet50_vd_infer_top_outputs.npz \
|
||||
--model ./ResNet50_vd_infer_1684x_f32.bmodel'
|
||||
|
||||
print(cmd_str)
|
||||
os.chdir('./ResNet50/workspace')
|
||||
run(cmd_str, shell=True)
|
||||
@@ -82,7 +97,7 @@ def mlir2bmodel():
|
||||
|
||||
args = parse_arguments()
|
||||
|
||||
if(args.auto):
|
||||
if (args.auto):
|
||||
download()
|
||||
paddle2onnx()
|
||||
mlir_prepare()
|
||||
|
||||
@@ -1,35 +1,26 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# MobileNet Front-end Deployment Example
|
||||
# MobileNet 前端部署示例
|
||||
|
||||
This document introduces the deployment of PaddleClas's mobilenet models for image classification to run in the browser, and the js interface in the @paddle-js-models/mobilenet npm package.
|
||||
本节介绍部署PaddleClas的图像分类mobilenet模型在浏览器中运行，以及@paddle-js-models/mobilenet npm包中的js接口。
|
||||
|
||||
## 1. 前端部署图像分类模型
|
||||
|
||||
## Front-end Deployment of Image Classification Model
|
||||
图像分类模型web demo使用[**参考文档**](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo)
|
||||
|
||||
To use the web demo of image classification models, refer to [**Reference Document**](../../../../application/js/web_demo/)
|
||||
|
||||
|
||||
## MobileNet js Interface
|
||||
## 2. MobileNet js接口
|
||||
|
||||
```
|
||||
import * as mobilenet from "@paddle-js-models/mobilenet";
|
||||
# mobilenet model loading and initialization
|
||||
# mobilenet模型加载和初始化
|
||||
await mobilenet.load()
|
||||
# mobilenet model performs the prediction and obtains the classification result
|
||||
# mobilenet模型执行预测，并获得分类的类别
|
||||
const res = await mobilenet.classify(img);
|
||||
console.log(res);
|
||||
```
|
||||
|
||||
**load() function parameter**
|
||||
**load()函数参数**
|
||||
|
||||
> * **Config**(dict): The configuration parameter for the image classification model. Default {Path: 'https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/mobileNetV2_fuse_activation/model.json', fill: '#fff', mean: [0.485, 0.456, 0.406],std: [0.229, 0.224, 0.225]}; Among them, modelPath is the path of the js model, fill is the padding value in the image pre-processing, and mean/std are the mean and standard deviation in the pre-processing
|
||||
|
||||
**classify() function parameter**
|
||||
> * **img**(HTMLImageElement): Enter an image parameter in HTMLImageElement.
|
||||
> * **Config**(dict): 图像分类模型配置参数，默认值为 {Path: 'https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/mobileNetV2_fuse_activation/model.json', fill: '#fff', mean: [0.485, 0.456, 0.406],std: [0.229, 0.224, 0.225]}; 其中，modelPath为js模型路径，fill 为图像预处理padding的值，mean和std分别为预处理的均值和标准差。
|
||||
|
||||
|
||||
|
||||
## Other Documents
|
||||
|
||||
- [PaddleClas model python deployment](../../paddleclas/python/)
|
||||
- [PaddleClas model C++ deployment](../cpp/)
|
||||
**classify()函数参数**
|
||||
> * **img**(HTMLImageElement): 输入图像参数，类型为HTMLImageElement。
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# MobileNet 前端部署示例
|
||||
|
||||
本节介绍部署PaddleClas的图像分类mobilenet模型在浏览器中运行，以及@paddle-js-models/mobilenet npm包中的js接口。
|
||||
|
||||
|
||||
## 前端部署图像分类模型
|
||||
|
||||
图像分类模型web demo使用[**参考文档**](../../../../application/js/web_demo/)
|
||||
|
||||
|
||||
## MobileNet js接口
|
||||
|
||||
```
|
||||
import * as mobilenet from "@paddle-js-models/mobilenet";
|
||||
# mobilenet模型加载和初始化
|
||||
await mobilenet.load()
|
||||
# mobilenet模型执行预测，并获得分类的类别
|
||||
const res = await mobilenet.classify(img);
|
||||
console.log(res);
|
||||
```
|
||||
|
||||
**load()函数参数**
|
||||
|
||||
> * **Config**(dict): 图像分类模型配置参数，默认值为 {Path: 'https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/mobileNetV2_fuse_activation/model.json', fill: '#fff', mean: [0.485, 0.456, 0.406],std: [0.229, 0.224, 0.225]}; 其中，modelPath为js模型路径，fill 为图像预处理padding的值，mean和std分别为预处理的均值和标准差。
|
||||
|
||||
|
||||
**classify()函数参数**
|
||||
> * **img**(HTMLImageElement): 输入图像参数，类型为HTMLImageElement。
|
||||
|
||||
|
||||
|
||||
## 其它文档
|
||||
|
||||
- [PaddleClas模型 python部署](../../paddleclas/python/)
|
||||
- [PaddleClas模型 C++部署](../cpp/)
|
||||
Reference in New Issue
Block a user