mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-30 03:22:05 +08:00
Update PaddleSeg example directory
This commit is contained in:
felixhjh
@@ -16,3 +16,6 @@ FastDeploy根据视觉模型的任务类型，定义了不同的结构体(`fastd
|
||||
| OCRResult | [C++/Python文档](./ocr_result.md) | 文本框检测，分类和文本识别返回结果 | OCR系列模型等 |
|
||||
| MOTResult | [C++/Python文档](./mot_result.md) | 多目标跟踪返回结果 | pptracking系列模型等 |
|
||||
| HeadPoseResult | [C++/Python文档](./headpose_result.md) | 头部姿态估计返回结果 | FSANet系列模型等 |
|
||||
|
||||
## 常见问题
|
||||
- [如何将视觉模型预测结果转换为numpy格式](./faq_CN.md)
|
||||
|
||||
25
docs/api/vision_results/faq_CN.md
Normal file
25
docs/api/vision_results/faq_CN.md
Normal file
@@ -0,0 +1,25 @@
|
||||
[English](faq.md)| 简体中文
|
||||
# 视觉模型预测结果常见问题
|
||||
|
||||
## 将视觉模型预测结果转换为numpy格式
|
||||
|
||||
这里以[SegmentationResult](./segmentation_result_CN.md)为例，展示如何抽取SegmentationResult中的label_map或者score_map来转为numpy格式，同时也可以利用已有数据new SegmentationResult结构体
|
||||
```
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
model = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file, params_file, config_file)
|
||||
im = cv2.imread(image)
|
||||
result = model.predict(im)
|
||||
# convert label_map and score_map to numpy format
|
||||
numpy_label_map = np.array(result.label_map)
|
||||
numpy_score_map = np.array(result.score_map)
|
||||
|
||||
# create SegmentationResult object
|
||||
result = fd.C.vision.SegmentationResult()
|
||||
result.label_map = numpy_label_map.tolist()
|
||||
result.score_map = numpy_score_map.tolist()
|
||||
```
|
||||
>> **注意**: 以上为示例代码，具体请参考[PaddleSeg example](../../../examples/vision/segmentation/paddleseg/)
|
||||
@@ -14,6 +14,7 @@ struct SegmentationResult {
|
||||
std::vector<int64_t> shape;
|
||||
bool contain_score_map = false;
|
||||
void Clear();
|
||||
void Free();
|
||||
std::string Str();
|
||||
};
|
||||
```
|
||||
@@ -22,6 +23,7 @@ struct SegmentationResult {
|
||||
- **score_map**: 成员变量，与label_map一一对应的所预测的分割类别概率值(当导出模型时指定`--output_op argmax`)或者经过softmax归一化化后的概率值(当导出模型时指定`--output_op softmax`或者导出模型时指定`--output_op none`同时模型初始化的时候设置模型[类成员属性](../../../examples/vision/segmentation/paddleseg/cpp/)`apply_softmax=True`)
|
||||
- **shape**: 成员变量，表示输出图片的shape，为H\*W
|
||||
- **Clear()**: 成员函数，用于清除结构体中存储的结果
|
||||
- **Free()**: 成员函数，用于清除结构体中存储的结果并释放内存
|
||||
- **Str()**: 成员函数，将结构体中的信息以字符串形式输出（用于Debug）
|
||||
|
||||
## Python 定义
|
||||
|
||||
@@ -2,6 +2,16 @@
|
||||
|
||||
# 晶晨 A311D 部署环境编译安装
|
||||
|
||||
## 导航目录
|
||||
|
||||
* [简介以及编译选项](#简介以及编译选项)
|
||||
* [交叉编译环境搭建](#交叉编译环境搭建)
|
||||
* [基于 Paddle Lite 的 FastDeploy 交叉编译库编译](#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
* [准备设备运行环境](#准备设备运行环境)
|
||||
* [基于 FastDeploy 在 A311D 上的部署示例](#基于-fastdeploy-在-a311d-上的部署示例)
|
||||
|
||||
## 简介以及编译选项
|
||||
|
||||
FastDeploy 基于 Paddle Lite 后端支持在晶晨 NPU 上进行部署推理。
|
||||
更多详细的信息请参考：[Paddle Lite部署示例](https://www.paddlepaddle.org.cn/lite/develop/demo_guides/verisilicon_timvx.html)。
|
||||
|
||||
|
||||
@@ -2,6 +2,16 @@
|
||||
|
||||
# 瑞芯微 RV1126 部署环境编译安装
|
||||
|
||||
## 导航目录
|
||||
|
||||
* [简介以及编译选项](#简介以及编译选项)
|
||||
* [交叉编译环境搭建](#交叉编译环境搭建)
|
||||
* [基于 Paddle Lite 的 FastDeploy 交叉编译库编译](#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
* [准备设备运行环境](#准备设备运行环境)
|
||||
* [基于 FastDeploy 在 RV1126 上的部署示例](#基于-fastdeploy-在-rv1126-上的部署示例)
|
||||
|
||||
## 简介以及编译选项
|
||||
|
||||
FastDeploy基于 Paddle Lite 后端支持在瑞芯微（Rockchip）Soc 上进行部署推理。
|
||||
更多详细的信息请参考：[Paddle Lite部署示例](https://www.paddlepaddle.org.cn/lite/develop/demo_guides/verisilicon_timvx.html)。
|
||||
|
||||
|
||||
@@ -1,47 +1,23 @@
|
||||
# PaddleSeg 模型部署
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
## FastDeploy介绍
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具，使用FastDeploy可以简单高效的在10+款硬件上对PaddleSeg模型进行快速部署
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
## 详细文档
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/deeplabv3/README.md)
|
||||
- [NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU](cpu-gpu)
|
||||
- [昆仑](kunlun)
|
||||
- [升腾](ascend)
|
||||
- [瑞芯微](rockchip)
|
||||
- [晶晨](amlogic)
|
||||
- [算能](sophgo)
|
||||
- [Android ARM CPU部署](android)
|
||||
- [服务化Serving部署](serving)
|
||||
- [模型自动化压缩工具](quantize)
|
||||
- [web部署](web)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
## 常见问题
|
||||
遇到问题可查看常见问题集合文档或搜索 FastDeploy issues，链接如下。若都无法解决，欢迎给 FastDeploy 提交新的issue
|
||||
[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq)
|
||||
[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues)
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PP-LiteSeg 量化模型在 A311D 上的部署
|
||||
目前 FastDeploy 已经支持基于 Paddle Lite 部署 PP-LiteSeg 量化模型到 A311D 上。
|
||||
|
||||
模型的量化和量化模型的下载请参考：[模型量化](../quantize/README.md)
|
||||
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
在 A311D 上只支持 C++ 的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -1,6 +1,6 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# Deployment of PP-LiteSeg Quantification Model on A311D
|
||||
Now FastDeploy allows deploying PP-LiteSeg quantization model to A311D based on Paddle Lite.
|
||||
# Deployment of PP-LiteSeg Quantification Model on A311D
|
||||
Now FastDeploy allows deploying PP-LiteSeg quantization model to A311D based on Paddle Lite.
|
||||
|
||||
For model quantization and download of quantized models, refer to [Model Quantization](../quantize/README.md)
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
[English](README.md) | 简体中文
|
||||
# 在晶晨A311D上使用FastDeploy部署PaddleSeg模型
|
||||
晶晨A311D是一款先进的AI应用处理器。目前，FastDeploy支持在A311D上基于Paddle-Lite部署PaddleSeg相关模型
|
||||
|
||||
## 晶晨A311D支持的PaddleSeg模型
|
||||
由于晶晨A311D的NPU仅支持INT8量化模型的部署，因此所支持的量化模型如下：
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型，开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes-without-argmax](https://bj.bcebos.com/fastdeploy/models/rk1/ppliteseg.tar.gz)| 31MB | 1024x512 | 77.04% | 77.73% | 77.46% |
|
||||
>> **注意**: FastDeploy模型量化的方法及一键自动化压缩工具可以参考[模型量化](../../../quantize/README.md)
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
目前，A311D上只支持C++的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -1,31 +1,31 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PP-LiteSeg 量化模型 C++ 部署示例
|
||||
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PP-LiteSeg 量化模型在 A311D 上的部署推理加速。
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PP-LiteSeg 量化模型在 晶晨A311D 上的部署推理加速。
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy 交叉编译环境准备
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](../../../../../../docs/cn/build_and_install/a311d.md#交叉编译环境搭建)
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/a311d.md#交叉编译环境搭建)
|
||||
|
||||
### 模型准备
|
||||
1. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户可以使用 FastDeploy 提供的一键模型自动化压缩工具,自行进行模型量化, 并使用产出的量化模型进行部署.(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的 deploy.yaml 文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可.)
|
||||
3. 模型需要异构计算，异构计算文件可以参考：[异构计算](./../../../../../../docs/cn/faq/heterogeneous_computing_on_timvx_npu.md)，由于 FastDeploy 已经提供了模型，可以先测试我们提供的异构文件，验证精度是否符合要求。
|
||||
3. 模型需要异构计算，异构计算文件可以参考：[异构计算](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/heterogeneous_computing_on_timvx_npu.md)，由于 FastDeploy 已经提供了模型，可以先测试我们提供的异构文件，验证精度是否符合要求。
|
||||
|
||||
更多量化相关相关信息可查阅[模型量化](../../quantize/README.md)
|
||||
更多量化相关相关信息可查阅[模型量化](../../../quantize/README.md)
|
||||
|
||||
## 在 A311D 上部署量化后的 PP-LiteSeg 分割模型
|
||||
请按照以下步骤完成在 A311D 上部署 PP-LiteSeg 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](../../../../../../docs/cn/build_and_install/a311d.md#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/a311d.md#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/fastdeploy/models/rk1/ppliteseg.tar.gz
|
||||
tar -xvf ppliteseg.tar.gz
|
||||
@@ -36,7 +36,7 @@ cp -r cityscapes_demo.png images
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=arm64 ..
|
||||
make -j8
|
||||
@@ -47,7 +47,7 @@ make install
|
||||
5. 基于 adb 工具部署 PP-LiteSeg 分割模型到晶晨 A311D，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp/build/install/
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo ppliteseg cityscapes_demo.png $DEVICE_ID
|
||||
```
|
||||
@@ -56,4 +56,4 @@ bash run_with_adb.sh infer_demo ppliteseg cityscapes_demo.png $DEVICE_ID
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/205544166-9b2719ff-ed82-4908-b90a-095de47392e1.png">
|
||||
|
||||
需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../../../../docs/cn/quantize.md)
|
||||
需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../quantize/README.md)
|
||||
6
examples/vision/segmentation/paddleseg/a311d/cpp/infer.cc → examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/infer.cc
Executable file → Normal file
6
examples/vision/segmentation/paddleseg/a311d/cpp/infer.cc → examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/infer.cc
Executable file → Normal file
@@ -24,13 +24,13 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
auto subgraph_file = model_dir + sep + "subgraph.txt";
|
||||
fastdeploy::vision::EnableFlyCV();
|
||||
fastdeploy::vision::EnableFlyCV();
|
||||
fastdeploy::RuntimeOption option;
|
||||
option.UseTimVX();
|
||||
option.SetLiteSubgraphPartitionPath(subgraph_file);
|
||||
|
||||
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file,option);
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
48
examples/vision/segmentation/paddleseg/cpu-gpu/README_CN.md
Normal file
48
examples/vision/segmentation/paddleseg/cpu-gpu/README_CN.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/deeplabv3/README.md)
|
||||
- [SegFormer系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/segformer/README.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting/)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
| [SegFormer_B0-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-with-argmax.tgz) \| [SegFormer_B0-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-without-argmax.tgz) | 15MB | 1024x1024 | 76.73% | 77.16% | - |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -1,7 +1,7 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleSeg C++ Deployment Example
|
||||
|
||||
This directory provides examples that `infer.cc` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT.
|
||||
This directory provides examples that `infer.cc` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT.
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
@@ -15,7 +15,7 @@ Taking the inference on Linux as an example, the compilation test can be complet
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download the FastDeploy precompiled library. Users can choose your appropriate version in the `FastDeploy Precompiled Library` mentioned above
|
||||
# Download the FastDeploy precompiled library. Users can choose your appropriate version in the `FastDeploy Precompiled Library` mentioned above
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
@@ -45,7 +45,7 @@ The visualized result after running is as follows
|
||||
The above command works for Linux or MacOS. For SDK use-pattern in Windows, refer to:
|
||||
- [How to use FastDeploy C++ SDK in Windows](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
## PaddleSeg C++ Interface
|
||||
## PaddleSeg C++ Interface
|
||||
|
||||
### PaddleSeg Class
|
||||
|
||||
@@ -62,7 +62,7 @@ PaddleSegModel model loading and initialization, among which model_file is the e
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
106
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/README_CN.md
Normal file
106
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/README_CN.md
Normal file
@@ -0,0 +1,106 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成PP-LiteSeg在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../../matting)
|
||||
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/cpp-gpu/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载PP-LiteSeg模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
|
||||
# CPU推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 0
|
||||
# GPU推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 1
|
||||
# GPU上TensorRT推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 2
|
||||
```
|
||||
|
||||
运行完成可视化结果如下图所示
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
> **注意：**
|
||||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||
- [如何在Windows中使用FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
## PaddleSeg C++接口
|
||||
|
||||
### PaddleSeg类
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
const string& model_file,
|
||||
const string& params_file = "",
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleSegModel模型加载和初始化，其中model_file为导出的Paddle模型格式。
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
#### Predict函数
|
||||
|
||||
> ```c++
|
||||
> PaddleSegModel::Predict(const cv::Mat &im, SegmentationResult *result)
|
||||
> ```
|
||||
>
|
||||
> 模型预测接口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **im**: 输入图像，注意需为HWC，BGR格式
|
||||
> > * **result**: 分割结果，包括分割预测的标签以及标签对应的概率值, SegmentationResult说明参考[SegmentationResult结构体介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
|
||||
### 类成员属性
|
||||
#### 预处理参数
|
||||
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
|
||||
|
||||
> > * **is_vertical_screen**(bool): PP-HumanSeg系列模型通过设置此参数为`true`表明输入图片是竖屏，即height大于width的图片
|
||||
|
||||
#### 后处理参数
|
||||
> > * **apply_softmax**(bool): 当模型导出时，并未指定`apply_softmax`参数，可通过此设置此参数为`true`，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理
|
||||
|
||||
## 快速链接
|
||||
- [PaddleSeg模型介绍](../../)
|
||||
- [Python部署](../python)
|
||||
|
||||
## 常见问题
|
||||
- [如何将模型预测结果SegmentationResult转为numpy格式](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
- [如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [PaddleSeg C++ API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1segmentation.html)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
131
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/infer.cc
Normal file
131
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/infer.cc
Normal file
@@ -0,0 +1,131 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void CpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseCpu();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void GpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void TrtInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
if (argc < 4) {
|
||||
std::cout
|
||||
<< "Usage: infer_demo path/to/model_dir path/to/image run_option, "
|
||||
"e.g ./infer_model ./ppseg_model_dir ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run "
|
||||
"with kunlunxin."
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (std::atoi(argv[3]) == 0) {
|
||||
CpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 1) {
|
||||
GpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 2) {
|
||||
TrtInfer(argv[1], argv[2]);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
@@ -10,7 +10,7 @@ Before deployment, two steps require confirmation
|
||||
|
||||
This directory provides examples that `infer.py` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT. The script is as follows
|
||||
```bash
|
||||
# Download the deployment example code
|
||||
# Download the deployment example code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python
|
||||
|
||||
@@ -34,7 +34,7 @@ The visualized result after running is as follows
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
## PaddleSegModel Python Interface
|
||||
## PaddleSegModel Python Interface
|
||||
|
||||
```python
|
||||
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
@@ -44,7 +44,7 @@ PaddleSeg model loading and initialization, among which model_file, params_file,
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
@@ -0,0 +1,88 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../../matting)
|
||||
|
||||
本目录下提供`infer.py`快速完成PP-LiteSeg在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/cpu-gpu/python
|
||||
|
||||
# 下载Unet模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# CPU推理
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu
|
||||
# GPU推理
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu
|
||||
# GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待）
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True
|
||||
```
|
||||
|
||||
运行完成可视化结果如下图所示
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
## PaddleSegModel Python接口
|
||||
|
||||
```python
|
||||
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleSeg模型加载和初始化，其中model_file, params_file以及config_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/docs/model_export_cn.md)
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
### predict函数
|
||||
|
||||
> ```python
|
||||
> PaddleSegModel.predict(input_image)
|
||||
> ```
|
||||
>
|
||||
> 模型预测结口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): 输入数据，注意需为HWC，BGR格式
|
||||
|
||||
> **返回**
|
||||
>
|
||||
> > 返回`fastdeploy.vision.SegmentationResult`结构体，结构体说明参考文档[SegmentationResult结构体介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
|
||||
### 类成员属性
|
||||
#### 预处理参数
|
||||
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
|
||||
|
||||
> > * **is_vertical_screen**(bool): PP-HumanSeg系列模型通过设置此参数为`true`表明输入图片是竖屏，即height大于width的图片
|
||||
|
||||
#### 后处理参数
|
||||
> > * **apply_softmax**(bool): 当模型导出时，并未指定`apply_softmax`参数，可通过此设置此参数为`true`，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理
|
||||
|
||||
## 其它文档
|
||||
|
||||
- [PaddleSeg 模型介绍](..)
|
||||
- [PaddleSeg C++部署](../cpp)
|
||||
|
||||
## 常见问题
|
||||
- [如何将模型预测结果SegmentationResult转为numpy格式](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
- [如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [PaddleSeg python API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/semantic_segmentation.html)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
57
examples/vision/segmentation/paddleseg/cpu-gpu/python/infer.py
Executable file
57
examples/vision/segmentation/paddleseg/cpu-gpu/python/infer.py
Executable file
@@ -0,0 +1,57 @@
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import os
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
import argparse
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--model", required=True, help="Path of PaddleSeg model.")
|
||||
parser.add_argument(
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
default='cpu',
|
||||
help="Type of inference device, support 'kunlunxin', 'cpu' or 'gpu'.")
|
||||
parser.add_argument(
|
||||
"--use_trt",
|
||||
type=ast.literal_eval,
|
||||
default=False,
|
||||
help="Wether to use tensorrt.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
option = fd.RuntimeOption()
|
||||
|
||||
if args.device.lower() == "gpu":
|
||||
option.use_gpu()
|
||||
|
||||
if args.use_trt:
|
||||
option.use_trt_backend()
|
||||
option.set_trt_input_shape("x", [1, 3, 256, 256], [1, 3, 1024, 1024],
|
||||
[1, 3, 2048, 2048])
|
||||
return option
|
||||
|
||||
|
||||
args = parse_arguments()
|
||||
|
||||
# 配置runtime，加载模型
|
||||
runtime_option = build_option(args)
|
||||
model_file = os.path.join(args.model, "model.pdmodel")
|
||||
params_file = os.path.join(args.model, "model.pdiparams")
|
||||
config_file = os.path.join(args.model, "deploy.yaml")
|
||||
model = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file, params_file, config_file, runtime_option=runtime_option)
|
||||
|
||||
# 预测图片分割结果
|
||||
im = cv2.imread(args.image)
|
||||
result = model.predict(im)
|
||||
print(result)
|
||||
|
||||
# 可视化结果
|
||||
vis_im = fd.vision.vis_segmentation(im, result, weight=0.5)
|
||||
cv2.imwrite("vis_img.png", vis_im)
|
||||
48
examples/vision/segmentation/paddleseg/kunlun/README_CN.md
Normal file
48
examples/vision/segmentation/paddleseg/kunlun/README_CN.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/deeplabv3/README.md)
|
||||
- [SegFormer系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/segformer/README.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting/)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
| [SegFormer_B0-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-with-argmax.tgz) \| [SegFormer_B0-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-without-argmax.tgz) | 15MB | 1024x1024 | 76.73% | 77.16% | - |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -0,0 +1,14 @@
|
||||
PROJECT(infer_demo C CXX)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||
|
||||
# 指定下载解压后的fastdeploy库路径
|
||||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||
|
||||
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
|
||||
|
||||
# 添加FastDeploy依赖头文件
|
||||
include_directories(${FASTDEPLOY_INCS})
|
||||
|
||||
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
|
||||
# 添加FastDeploy库依赖
|
||||
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
|
||||
96
examples/vision/segmentation/paddleseg/kunlun/cpp/README.md
Executable file
96
examples/vision/segmentation/paddleseg/kunlun/cpp/README.md
Executable file
@@ -0,0 +1,96 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleSeg C++ Deployment Example
|
||||
|
||||
This directory provides examples that `infer.cc` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT.
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Download the precompiled deployment library and samples code according to your development environment. Refer to [FastDeploy Precompiled Library](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【Attention】For the deployment of **PP-Matting**、**PP-HumanMatting** and **ModNet**, refer to [Matting Model Deployment](../../../matting)
|
||||
|
||||
Taking the inference on Linux as an example, the compilation test can be completed by executing the following command in this directory. FastDeploy version 1.0.0 or above (x.x.x>=1.0.0) is required to support this model.
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download the FastDeploy precompiled library. Users can choose your appropriate version in the `FastDeploy Precompiled Library` mentioned above
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# Download Unet model files and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf Unet_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
|
||||
# CPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 0
|
||||
# GPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 1
|
||||
# TensorRT inference on GPU
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 2
|
||||
# kunlunxin XPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 3
|
||||
```
|
||||
|
||||
The visualized result after running is as follows
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
The above command works for Linux or MacOS. For SDK use-pattern in Windows, refer to:
|
||||
- [How to use FastDeploy C++ SDK in Windows](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
## PaddleSeg C++ Interface
|
||||
|
||||
### PaddleSeg Class
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
const string& model_file,
|
||||
const string& params_file = "",
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleSegModel model loading and initialization, among which model_file is the exported Paddle model format.
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
#### Predict Function
|
||||
|
||||
> ```c++
|
||||
> PaddleSegModel::Predict(cv::Mat* im, DetectionResult* result)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output detection results.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **im**: Input images in HWC or BGR format
|
||||
> > * **result**: The segmentation result, including the predicted label of the segmentation and the corresponding probability of the label. Refer to [Vision Model Prediction Results](../../../../../docs/api/vision_results/) for the description of SegmentationResult
|
||||
|
||||
### Class Member Variable
|
||||
#### Pre-processing Parameter
|
||||
Users can modify the following pre-processing parameters to their needs, which affects the final inference and deployment results
|
||||
|
||||
> > * **is_vertical_screen**(bool): For PP-HumanSeg models, the input image is portrait, height greater than a width, by setting this parameter to`true`
|
||||
|
||||
#### Post-processing Parameter
|
||||
> > * **apply_softmax**(bool): The `apply_softmax` parameter is not specified when the model is exported. Set this parameter to `true` to normalize the probability result (score_map) of the predicted output segmentation label (label_map)
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Python Deployment](../python)
|
||||
- [Vision Model Prediction Results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
6
examples/vision/segmentation/paddleseg/cpp/infer.cc → examples/vision/segmentation/paddleseg/kunlun/cpp/infer.cc
Executable file → Normal file
6
examples/vision/segmentation/paddleseg/cpp/infer.cc → examples/vision/segmentation/paddleseg/kunlun/cpp/infer.cc
Executable file → Normal file
@@ -48,7 +48,8 @@ void CpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void KunlunXinInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
void KunlunXinInfer(const std::string& model_dir,
|
||||
const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
@@ -170,7 +171,8 @@ int main(int argc, char* argv[]) {
|
||||
"e.g ./infer_model ./ppseg_model_dir ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run with kunlunxin."
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run "
|
||||
"with kunlunxin."
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
82
examples/vision/segmentation/paddleseg/kunlun/python/README.md
Executable file
82
examples/vision/segmentation/paddleseg/kunlun/python/README.md
Executable file
@@ -0,0 +1,82 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleSeg Python Deployment Example
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Install FastDeploy Python whl package. Refer to [FastDeploy Python Installation](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【Attention】For the deployment of **PP-Matting**、**PP-HumanMatting** and **ModNet**, refer to [Matting Model Deployment](../../../matting)
|
||||
|
||||
This directory provides examples that `infer.py` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT. The script is as follows
|
||||
```bash
|
||||
# Download the deployment example code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python
|
||||
|
||||
# Download Unet model files and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf Unet_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# CPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu
|
||||
# GPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu
|
||||
# TensorRT inference on GPU（Attention: It is somewhat time-consuming for the operation of model serialization when running TensorRT inference for the first time. Please be patient.）
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True
|
||||
# kunlunxin XPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device kunlunxin
|
||||
```
|
||||
|
||||
The visualized result after running is as follows
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
## PaddleSegModel Python Interface
|
||||
|
||||
```python
|
||||
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleSeg model loading and initialization, among which model_file, params_file, and config_file are the Paddle inference files exported from the training model. Refer to [Model Export](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/docs/model_export_cn.md) for more information
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
### predict function
|
||||
|
||||
> ```python
|
||||
> PaddleSegModel.predict(input_image)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output detection results.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): Input data in HWC or BGR format
|
||||
|
||||
> **Return**
|
||||
>
|
||||
> > Return `fastdeploy.vision.SegmentationResult` structure. Refer to [Vision Model Prediction Results](../../../../../docs/api/vision_results/) for the description of the structure.
|
||||
|
||||
### Class Member Variable
|
||||
#### Pre-processing Parameter
|
||||
Users can modify the following pre-processing parameters to their needs, which affects the final inference and deployment results
|
||||
|
||||
> > * **is_vertical_screen**(bool): For PP-HumanSeg models, the input image is portrait with height greater than width by setting this parameter to `true`
|
||||
#### Post-processing Parameter
|
||||
> > * **apply_softmax**(bool): The `apply_softmax` parameter is not specified when the model is exported. Set this parameter to `true` to normalize the probability result (score_map) of the predicted output segmentation label (label_map) in softmax
|
||||
|
||||
## Other Documents
|
||||
|
||||
- [PaddleSeg Model Description](..)
|
||||
- [PaddleSeg C++ Deployment](../cpp)
|
||||
- [Model Prediction Results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
@@ -5,33 +5,22 @@ FastDeploy已支持部署量化模型,并提供一键模型自动化压缩的工
|
||||
|
||||
## FastDeploy一键模型自动化压缩工具
|
||||
FastDeploy 提供了一键模型自动化压缩工具, 能够简单地通过输入一个配置文件, 对模型进行量化.
|
||||
详细教程请见: [一键模型自动化压缩工具](../../../../../tools/common_tools/auto_compression/)
|
||||
注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的deploy.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
详细教程请见: [一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression)
|
||||
>> **注意**: 推理量化后的分类模型仍然需要FP32模型文件夹下的deploy.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
|
||||
## 下载量化完成的PaddleSeg模型
|
||||
## 量化完成的PaddleSeg模型
|
||||
用户也可以直接下载下表中的量化模型进行部署.(点击模型名字即可下载)
|
||||
|
||||
Benchmark表格说明:
|
||||
- Runtime时延为模型在各种Runtime上的推理时延,包含CPU->GPU数据拷贝,GPU推理,GPU->CPU数据拷贝时间. 不包含模型各自的前后处理时间.
|
||||
- 端到端时延为模型在实际推理场景中的时延, 包含模型的前后处理.
|
||||
- 所测时延均为推理1000次后求得的平均值, 单位是毫秒.
|
||||
- INT8 + FP16 为在推理INT8量化模型的同时, 给Runtime 开启FP16推理选项
|
||||
- INT8 + FP16 + PM, 为在推理INT8量化模型和开启FP16的同时, 开启使用Pinned Memory的选项,可加速GPU->CPU数据拷贝的速度
|
||||
- 最大加速比, 为FP32时延除以INT8推理的最快时延,得到最大加速比.
|
||||
- 策略为量化蒸馏训练时, 采用少量无标签数据集训练得到量化模型, 并在全量验证集上验证精度, INT8精度并不代表最高的INT8精度.
|
||||
- CPU为Intel(R) Xeon(R) Gold 6271C, 所有测试中固定CPU线程数为1. GPU为Tesla T4, TensorRT版本8.4.15.
|
||||
| 模型 | 量化方式 |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) |量化蒸馏训练 |
|
||||
|
||||
#### Runtime Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 Runtime时延 | INT8 Runtime时延 | INT8 + FP16 Runtime时延 | INT8+FP16+PM Runtime时延 | 最大加速比 | FP32 mIoU | INT8 mIoU | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) | Paddle Inference | CPU | 1138.04| 602.62 |None|None | 1.89 |77.37 | 71.62 |量化蒸馏训练 |
|
||||
量化后模型的Benchmark比较，请参考[量化模型 Benchmark](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/quantize.md)
|
||||
|
||||
#### 端到端 Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 End2End时延 | INT8 End2End时延 | INT8 + FP16 End2End时延 | INT8+FP16+PM End2End时延 | 最大加速比 | FP32 mIoU | INT8 mIoU | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) | Paddle Inference | CPU | 4726.65| 4134.91|None|None | 1.14 |77.37 | 71.62 |量化蒸馏训练 |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
## 支持部署量化模型的硬件
|
||||
FastDeploy 量化模型部署的过程大致都与FP32模型类似，只是模型量化与非量化的区别，如果硬件在量化模型部署过程有特殊处理，也会在文档中特别标明，因此量化模型部署可以参考如下硬件的链接
|
||||
- [NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU](../cpu-gpu)
|
||||
- [昆仑](../kunlun)
|
||||
- [升腾](../ascend)
|
||||
- [瑞芯微](../rockchip)
|
||||
- [晶晨](../amlogic)
|
||||
- [算能](../sophgo)
|
||||
|
||||
@@ -33,4 +33,4 @@ file(GLOB PADDLETOONNX_LIBS ${FASTDEPLOY_INSTALL_DIR}/third_libs/install/paddle2
|
||||
install(PROGRAMS ${PADDLETOONNX_LIBS} DESTINATION lib)
|
||||
|
||||
file(GLOB RKNPU2_LIBS ${FASTDEPLOY_INSTALL_DIR}/third_libs/install/rknpu2_runtime/RK3588/lib/*)
|
||||
install(PROGRAMS ${RKNPU2_LIBS} DESTINATION lib)
|
||||
install(PROGRAMS ${RKNPU2_LIBS} DESTINATION lib)
|
||||
@@ -40,7 +40,7 @@ mkdir thirdpartys
|
||||
Please refer to [How to Build RKNPU2 Deployment Environment](../../../../../../docs/en/build_and_install/rknpu2.md) to compile SDK.After compiling, the fastdeploy-0.0.3 directory will be created in the build directory, please move it to the thirdpartys directory.
|
||||
|
||||
### Copy model and configuration files to folder Model
|
||||
In the process of Paddle dynamic map model -> Paddle static map model -> ONNX mdoel, ONNX file and the corresponding yaml configuration file will be generated. Please move the configuration file to the folder model.
|
||||
In the process of Paddle dynamic map model -> Paddle static map model -> ONNX mdoel, ONNX file and the corresponding yaml configuration file will be generated. Please move the configuration file to the folder model.
|
||||
After converting to RKNN, the model file also needs to be copied to folder model. Run the following command to download and use (the model file is RK3588. RK3568 needs to be [reconverted to PPSeg RKNN model](../README.md)).
|
||||
|
||||
### Prepare Test Images to folder image
|
||||
@@ -16,7 +16,8 @@
|
||||
#include "fastdeploy/vision.h"
|
||||
|
||||
void ONNXInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::string model_file = model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer.onnx";
|
||||
std::string model_file =
|
||||
model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer.onnx";
|
||||
std::string params_file;
|
||||
std::string config_file = model_dir + "/deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
@@ -43,13 +44,12 @@ void ONNXInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
tc.PrintInfo("PPSeg in ONNX");
|
||||
|
||||
cv::imwrite("infer_onnx.jpg", vis_im);
|
||||
std::cout
|
||||
<< "Visualized result saved in ./infer_onnx.jpg"
|
||||
<< std::endl;
|
||||
std::cout << "Visualized result saved in ./infer_onnx.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::string model_file = model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer_rk3588.rknn";
|
||||
std::string model_file =
|
||||
model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer_rk3588.rknn";
|
||||
std::string params_file;
|
||||
std::string config_file = model_dir + "/deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
@@ -78,9 +78,7 @@ void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
tc.PrintInfo("PPSeg in RKNPU2");
|
||||
|
||||
cv::imwrite("infer_rknn.jpg", vis_im);
|
||||
std::cout
|
||||
<< "Visualized result saved in ./infer_rknn.jpg"
|
||||
<< std::endl;
|
||||
std::cout << "Visualized result saved in ./infer_rknn.jpg" << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
@@ -93,7 +91,6 @@ int main(int argc, char* argv[]) {
|
||||
}
|
||||
|
||||
RKNPU2Infer(argv[1], argv[2]);
|
||||
// ONNXInfer(argv[1], argv[2]);
|
||||
// ONNXInfer(argv[1], argv[2]);
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -78,4 +78,4 @@ Deploy:
|
||||
- 144
|
||||
type: Resize
|
||||
- type: Normalize
|
||||
```
|
||||
```
|
||||
@@ -78,4 +78,4 @@ Deploy:
|
||||
- 144
|
||||
type: Resize
|
||||
- type: Normalize
|
||||
```
|
||||
```
|
||||
@@ -1,6 +1,6 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# Deployment of PP-LiteSeg Quantification Model on RV1126
|
||||
Now FastDeploy allows deploying PP-LiteSeg quantization model to RV1126 based on Paddle Lite.
|
||||
Now FastDeploy allows deploying PP-LiteSeg quantization model to RV1126 based on Paddle Lite.
|
||||
|
||||
For model quantization and download of quantized models, refer to [Model Quantization](../quantize/README.md)
|
||||
|
||||
6
examples/vision/segmentation/paddleseg/rv1126/cpp/infer.cc → examples/vision/segmentation/paddleseg/rockchip/rv1126/cpp/infer.cc
Executable file → Normal file
6
examples/vision/segmentation/paddleseg/rv1126/cpp/infer.cc → examples/vision/segmentation/paddleseg/rockchip/rv1126/cpp/infer.cc
Executable file → Normal file
@@ -24,13 +24,13 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
auto subgraph_file = model_dir + sep + "subgraph.txt";
|
||||
fastdeploy::vision::EnableFlyCV();
|
||||
fastdeploy::vision::EnableFlyCV();
|
||||
fastdeploy::RuntimeOption option;
|
||||
option.UseTimVX();
|
||||
option.SetLiteSubgraphPartitionPath(subgraph_file);
|
||||
|
||||
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file,option);
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
@@ -1,68 +1,9 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSegmentation 服务化部署示例
|
||||
# 使用 FastDeploy 服务化部署 PaddleSeg 模型
|
||||
## FastDeploy 服务化部署介绍
|
||||
在线推理作为企业或个人线上部署模型的最后一环，是工业界必不可少的环节，其中最重要的就是服务化推理框架。FastDeploy 目前提供两种服务化部署方式：simple_serving和fastdeploy_serving。simple_serving 基于Flask框架具有简单高效的特点，可以快速验证线上部署模型的可行性。fastdeploy_serving基于Triton Inference Server框架，是一套完备且性能卓越的服务化部署框架，可用于实际生产。
|
||||
|
||||
在服务化部署前，需确认
|
||||
## 详细部署文档
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
|
||||
|
||||
## 启动服务
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/serving
|
||||
|
||||
#下载yolov5模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 将模型文件放入 models/runtime/1目录下
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdmodel models/runtime/1/
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdiparams models/runtime/1/
|
||||
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改paddleseg_grpc_client.py中的请求端口号
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
#下载测试图片
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient[all]
|
||||
|
||||
# 发送请求
|
||||
python3 paddleseg_grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在CPU上运行ONNXRuntime引擎， 如果要在GPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](../../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
- [fastdeploy serving](fastdeploy_serving)
|
||||
- [simple serving](simple_serving)
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg 服务化部署示例
|
||||
|
||||
在服务化部署前，需确认
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/README_CN.md)
|
||||
|
||||
|
||||
## 启动服务
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/serving/fastdeploy_serving
|
||||
|
||||
#下载PP-LiteSeg模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 将模型文件放入 models/runtime/1目录下
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdmodel models/runtime/1/
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdiparams models/runtime/1/
|
||||
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改paddleseg_grpc_client.py中的请求端口号
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
#下载测试图片
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient[all]
|
||||
|
||||
# 发送请求
|
||||
python3 paddleseg_grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
tm: name: "INPUT"
|
||||
datatype: "UINT8"
|
||||
shape: -1
|
||||
shape: -1
|
||||
shape: -1
|
||||
shape: 3
|
||||
|
||||
output_name: SEG_RESULT
|
||||
Only print the first 20 labels in label_map of SEG_RESULT
|
||||
{'label_map': [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], 'score_map': [], 'shape': [1024, 2048], 'contain_score_map': False}
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在CPU上运行ONNXRuntime引擎， 如果要在GPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
## 更多部署方式
|
||||
- [使用 VisualDL 进行 Serving 可视化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/vdl_management.md)
|
||||
|
||||
## 常见问题
|
||||
- [如何编写客户端 HTTP/GRPC 请求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/client.md)
|
||||
- [如何编译服务化部署镜像](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/compile.md)
|
||||
- [服务化部署原理及动态Batch介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/demo.md)
|
||||
- [模型仓库介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/model_repository.md)
|
||||
@@ -0,0 +1,36 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
# PaddleSegmentation Python Simple Serving Demo
|
||||
|
||||
|
||||
## Environment
|
||||
|
||||
- 1. Prepare environment and install FastDeploy Python whl, refer to [download_prebuilt_libraries](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
Server:
|
||||
```bash
|
||||
# Download demo code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# Download PP_LiteSeg model
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# Launch server, change the configurations in server.py to select hardware, backend, etc.
|
||||
# and use --host, --port to specify IP and port
|
||||
fastdeploy simple_serving --app server:app
|
||||
```
|
||||
|
||||
Client:
|
||||
```bash
|
||||
# Download demo code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# Download test image
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# Send request and get inference result (Please adapt the IP and port if necessary)
|
||||
python client.py
|
||||
```
|
||||
@@ -0,0 +1,32 @@
|
||||
简体中文 | [English](README.md)
|
||||
|
||||
# PaddleSeg Python轻量服务化部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
服务端：
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# 下载PP-LiteSeg模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 启动服务，可修改server.py中的配置项来指定硬件、后端等
|
||||
# 可通过--host、--port指定IP和端口号
|
||||
fastdeploy simple_serving --app server:app
|
||||
```
|
||||
|
||||
客户端：
|
||||
```bash
|
||||
# 下载测试图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
|
||||
|
||||
# 请求服务，获取推理结果（如有必要，请修改脚本中的IP和端口号）
|
||||
python client.py
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
import requests
|
||||
import json
|
||||
import cv2
|
||||
import fastdeploy as fd
|
||||
from fastdeploy.serving.utils import cv2_to_base64
|
||||
|
||||
if __name__ == '__main__':
|
||||
url = "http://127.0.0.1:8000/fd/ppliteseg"
|
||||
headers = {"Content-Type": "application/json"}
|
||||
|
||||
im = cv2.imread("cityscapes_demo.png")
|
||||
data = {"data": {"image": cv2_to_base64(im)}, "parameters": {}}
|
||||
|
||||
resp = requests.post(url=url, headers=headers, data=json.dumps(data))
|
||||
if resp.status_code == 200:
|
||||
r_json = json.loads(resp.json()["result"])
|
||||
result = fd.vision.utils.json_to_segmentation(r_json)
|
||||
vis_im = fd.vision.vis_segmentation(im, result, weight=0.5)
|
||||
cv2.imwrite("visualized_result.jpg", vis_im)
|
||||
print("Visualized result save in ./visualized_result.jpg")
|
||||
else:
|
||||
print("Error code:", resp.status_code)
|
||||
print(resp.text)
|
||||
@@ -0,0 +1,38 @@
|
||||
import fastdeploy as fd
|
||||
from fastdeploy.serving.server import SimpleServer
|
||||
import os
|
||||
import logging
|
||||
|
||||
logging.getLogger().setLevel(logging.INFO)
|
||||
|
||||
# Configurations
|
||||
model_dir = 'PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer'
|
||||
device = 'cpu'
|
||||
use_trt = False
|
||||
|
||||
# Prepare model
|
||||
model_file = os.path.join(model_dir, "model.pdmodel")
|
||||
params_file = os.path.join(model_dir, "model.pdiparams")
|
||||
config_file = os.path.join(model_dir, "deploy.yaml")
|
||||
|
||||
# Setup runtime option to select hardware, backend, etc.
|
||||
option = fd.RuntimeOption()
|
||||
if device.lower() == 'gpu':
|
||||
option.use_gpu()
|
||||
if use_trt:
|
||||
option.use_trt_backend()
|
||||
option.set_trt_cache_file('pp_lite_seg.trt')
|
||||
|
||||
# Create model instance
|
||||
model_instance = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file=model_file,
|
||||
params_file=params_file,
|
||||
config_file=config_file,
|
||||
runtime_option=option)
|
||||
|
||||
# Create server, setup REST API
|
||||
app = SimpleServer()
|
||||
app.register(
|
||||
task_name="fd/ppliteseg",
|
||||
model_handler=fd.serving.handler.VisionModelHandler,
|
||||
predictor=model_instance)
|
||||
@@ -18,7 +18,14 @@ Here we take [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.c
|
||||
|
||||
### Download PP-LiteSeg-B(STDC2)-cityscapes-without-argmax, and convert it to ONNX
|
||||
```shell
|
||||
https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
# Download Paddle2ONNX repository.
|
||||
git clone https://github.com/PaddlePaddle/Paddle2ONNX
|
||||
|
||||
# Download the Paddle static map model and fix the input shape.
|
||||
## Go to the directory where the input shape is fixed for the Paddle static map model.
|
||||
cd Paddle2ONNX/tools/paddle
|
||||
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
|
||||
# Modify the input shape of PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer model from dynamic input to constant input.
|
||||
|
||||
@@ -18,7 +18,14 @@ SOPHGO-TPU部署模型前需要将Paddle模型转换成bmodel模型，具体步
|
||||
|
||||
### 下载PP-LiteSeg-B(STDC2)-cityscapes-without-argmax模型,并转换为ONNX模型
|
||||
```shell
|
||||
https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
# 下载Paddle2ONNX仓库
|
||||
git clone https://github.com/PaddlePaddle/Paddle2ONNX
|
||||
|
||||
# 下载Paddle静态图模型并为Paddle静态图模型固定输入shape
|
||||
## 进入为Paddle静态图模型固定输入shape的目录
|
||||
cd Paddle2ONNX/tools/paddle
|
||||
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
|
||||
# 修改PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer模型的输入shape，由动态输入变成固定输入
|
||||
|
||||
Reference in New Issue
Block a user