mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-19 06:54:41 +08:00
Update PaddleSeg example directory
This commit is contained in:
felixhjh
@@ -16,3 +16,6 @@ FastDeploy根据视觉模型的任务类型，定义了不同的结构体(`fastd
|
||||
| OCRResult | [C++/Python文档](./ocr_result.md) | 文本框检测，分类和文本识别返回结果 | OCR系列模型等 |
|
||||
| MOTResult | [C++/Python文档](./mot_result.md) | 多目标跟踪返回结果 | pptracking系列模型等 |
|
||||
| HeadPoseResult | [C++/Python文档](./headpose_result.md) | 头部姿态估计返回结果 | FSANet系列模型等 |
|
||||
|
||||
## 常见问题
|
||||
- [如何将视觉模型预测结果转换为numpy格式](./faq_CN.md)
|
||||
|
||||
25
docs/api/vision_results/faq_CN.md
Normal file
25
docs/api/vision_results/faq_CN.md
Normal file
@@ -0,0 +1,25 @@
|
||||
[English](faq.md)| 简体中文
|
||||
# 视觉模型预测结果常见问题
|
||||
|
||||
## 将视觉模型预测结果转换为numpy格式
|
||||
|
||||
这里以[SegmentationResult](./segmentation_result_CN.md)为例，展示如何抽取SegmentationResult中的label_map或者score_map来转为numpy格式，同时也可以利用已有数据new SegmentationResult结构体
|
||||
```
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
model = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file, params_file, config_file)
|
||||
im = cv2.imread(image)
|
||||
result = model.predict(im)
|
||||
# convert label_map and score_map to numpy format
|
||||
numpy_label_map = np.array(result.label_map)
|
||||
numpy_score_map = np.array(result.score_map)
|
||||
|
||||
# create SegmentationResult object
|
||||
result = fd.C.vision.SegmentationResult()
|
||||
result.label_map = numpy_label_map.tolist()
|
||||
result.score_map = numpy_score_map.tolist()
|
||||
```
|
||||
>> **注意**: 以上为示例代码，具体请参考[PaddleSeg example](../../../examples/vision/segmentation/paddleseg/)
|
||||
@@ -14,6 +14,7 @@ struct SegmentationResult {
|
||||
std::vector<int64_t> shape;
|
||||
bool contain_score_map = false;
|
||||
void Clear();
|
||||
void Free();
|
||||
std::string Str();
|
||||
};
|
||||
```
|
||||
@@ -22,6 +23,7 @@ struct SegmentationResult {
|
||||
- **score_map**: 成员变量，与label_map一一对应的所预测的分割类别概率值(当导出模型时指定`--output_op argmax`)或者经过softmax归一化化后的概率值(当导出模型时指定`--output_op softmax`或者导出模型时指定`--output_op none`同时模型初始化的时候设置模型[类成员属性](../../../examples/vision/segmentation/paddleseg/cpp/)`apply_softmax=True`)
|
||||
- **shape**: 成员变量，表示输出图片的shape，为H\*W
|
||||
- **Clear()**: 成员函数，用于清除结构体中存储的结果
|
||||
- **Free()**: 成员函数，用于清除结构体中存储的结果并释放内存
|
||||
- **Str()**: 成员函数，将结构体中的信息以字符串形式输出（用于Debug）
|
||||
|
||||
## Python 定义
|
||||
|
||||
@@ -2,6 +2,16 @@
|
||||
|
||||
# 晶晨 A311D 部署环境编译安装
|
||||
|
||||
## 导航目录
|
||||
|
||||
* [简介以及编译选项](#简介以及编译选项)
|
||||
* [交叉编译环境搭建](#交叉编译环境搭建)
|
||||
* [基于 Paddle Lite 的 FastDeploy 交叉编译库编译](#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
* [准备设备运行环境](#准备设备运行环境)
|
||||
* [基于 FastDeploy 在 A311D 上的部署示例](#基于-fastdeploy-在-a311d-上的部署示例)
|
||||
|
||||
## 简介以及编译选项
|
||||
|
||||
FastDeploy 基于 Paddle Lite 后端支持在晶晨 NPU 上进行部署推理。
|
||||
更多详细的信息请参考：[Paddle Lite部署示例](https://www.paddlepaddle.org.cn/lite/develop/demo_guides/verisilicon_timvx.html)。
|
||||
|
||||
|
||||
@@ -2,6 +2,16 @@
|
||||
|
||||
# 瑞芯微 RV1126 部署环境编译安装
|
||||
|
||||
## 导航目录
|
||||
|
||||
* [简介以及编译选项](#简介以及编译选项)
|
||||
* [交叉编译环境搭建](#交叉编译环境搭建)
|
||||
* [基于 Paddle Lite 的 FastDeploy 交叉编译库编译](#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
* [准备设备运行环境](#准备设备运行环境)
|
||||
* [基于 FastDeploy 在 RV1126 上的部署示例](#基于-fastdeploy-在-rv1126-上的部署示例)
|
||||
|
||||
## 简介以及编译选项
|
||||
|
||||
FastDeploy基于 Paddle Lite 后端支持在瑞芯微（Rockchip）Soc 上进行部署推理。
|
||||
更多详细的信息请参考：[Paddle Lite部署示例](https://www.paddlepaddle.org.cn/lite/develop/demo_guides/verisilicon_timvx.html)。
|
||||
|
||||
|
||||
@@ -1,47 +1,23 @@
|
||||
# PaddleSeg 模型部署
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
## FastDeploy介绍
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具，使用FastDeploy可以简单高效的在10+款硬件上对PaddleSeg模型进行快速部署
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
## 详细文档
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/configs/deeplabv3/README.md)
|
||||
- [NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU](cpu-gpu)
|
||||
- [昆仑](kunlun)
|
||||
- [升腾](ascend)
|
||||
- [瑞芯微](rockchip)
|
||||
- [晶晨](amlogic)
|
||||
- [算能](sophgo)
|
||||
- [Android ARM CPU部署](android)
|
||||
- [服务化Serving部署](serving)
|
||||
- [模型自动化压缩工具](quantize)
|
||||
- [web部署](web)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
## 常见问题
|
||||
遇到问题可查看常见问题集合文档或搜索 FastDeploy issues，链接如下。若都无法解决，欢迎给 FastDeploy 提交新的issue
|
||||
[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq)
|
||||
[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues)
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PP-LiteSeg 量化模型在 A311D 上的部署
|
||||
目前 FastDeploy 已经支持基于 Paddle Lite 部署 PP-LiteSeg 量化模型到 A311D 上。
|
||||
|
||||
模型的量化和量化模型的下载请参考：[模型量化](../quantize/README.md)
|
||||
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
在 A311D 上只支持 C++ 的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -0,0 +1,20 @@
|
||||
[English](README.md) | 简体中文
|
||||
# 在晶晨A311D上使用FastDeploy部署PaddleSeg模型
|
||||
晶晨A311D是一款先进的AI应用处理器。目前，FastDeploy支持在A311D上基于Paddle-Lite部署PaddleSeg相关模型
|
||||
|
||||
## 晶晨A311D支持的PaddleSeg模型
|
||||
由于晶晨A311D的NPU仅支持INT8量化模型的部署，因此所支持的量化模型如下：
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型，开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes-without-argmax](https://bj.bcebos.com/fastdeploy/models/rk1/ppliteseg.tar.gz)| 31MB | 1024x512 | 77.04% | 77.73% | 77.46% |
|
||||
>> **注意**: FastDeploy模型量化的方法及一键自动化压缩工具可以参考[模型量化](../../../quantize/README.md)
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
目前，A311D上只支持C++的部署。
|
||||
|
||||
- [C++部署](cpp)
|
||||
@@ -1,31 +1,31 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PP-LiteSeg 量化模型 C++ 部署示例
|
||||
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PP-LiteSeg 量化模型在 A311D 上的部署推理加速。
|
||||
本目录下提供的 `infer.cc`，可以帮助用户快速完成 PP-LiteSeg 量化模型在 晶晨A311D 上的部署推理加速。
|
||||
|
||||
## 部署准备
|
||||
### FastDeploy 交叉编译环境准备
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](../../../../../../docs/cn/build_and_install/a311d.md#交叉编译环境搭建)
|
||||
1. 软硬件环境满足要求，以及交叉编译环境的准备，请参考：[FastDeploy 交叉编译环境准备](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/a311d.md#交叉编译环境搭建)
|
||||
|
||||
### 模型准备
|
||||
1. 用户可以直接使用由 FastDeploy 提供的量化模型进行部署。
|
||||
2. 用户可以使用 FastDeploy 提供的一键模型自动化压缩工具,自行进行模型量化, 并使用产出的量化模型进行部署.(注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的 deploy.yaml 文件, 自行量化的模型文件夹内不包含此 yaml 文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可.)
|
||||
3. 模型需要异构计算，异构计算文件可以参考：[异构计算](./../../../../../../docs/cn/faq/heterogeneous_computing_on_timvx_npu.md)，由于 FastDeploy 已经提供了模型，可以先测试我们提供的异构文件，验证精度是否符合要求。
|
||||
3. 模型需要异构计算，异构计算文件可以参考：[异构计算](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/heterogeneous_computing_on_timvx_npu.md)，由于 FastDeploy 已经提供了模型，可以先测试我们提供的异构文件，验证精度是否符合要求。
|
||||
|
||||
更多量化相关相关信息可查阅[模型量化](../../quantize/README.md)
|
||||
更多量化相关相关信息可查阅[模型量化](../../../quantize/README.md)
|
||||
|
||||
## 在 A311D 上部署量化后的 PP-LiteSeg 分割模型
|
||||
请按照以下步骤完成在 A311D 上部署 PP-LiteSeg 量化模型：
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](../../../../../../docs/cn/build_and_install/a311d.md#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
1. 交叉编译编译 FastDeploy 库，具体请参考：[交叉编译 FastDeploy](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/a311d.md#基于-paddle-lite-的-fastdeploy-交叉编译库编译)
|
||||
|
||||
2. 将编译后的库拷贝到当前目录，可使用如下命令：
|
||||
```bash
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
```
|
||||
|
||||
3. 在当前路径下载部署所需的模型和示例图片：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
mkdir models && mkdir images
|
||||
wget https://bj.bcebos.com/fastdeploy/models/rk1/ppliteseg.tar.gz
|
||||
tar -xvf ppliteseg.tar.gz
|
||||
@@ -36,7 +36,7 @@ cp -r cityscapes_demo.png images
|
||||
|
||||
4. 编译部署示例，可使入如下命令：
|
||||
```bash
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=arm64 ..
|
||||
make -j8
|
||||
@@ -47,7 +47,7 @@ make install
|
||||
5. 基于 adb 工具部署 PP-LiteSeg 分割模型到晶晨 A311D，可使用如下命令：
|
||||
```bash
|
||||
# 进入 install 目录
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/a311d/cpp/build/install/
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/build/install/
|
||||
# 如下命令表示：bash run_with_adb.sh 需要运行的demo 模型路径 图片路径 设备的DEVICE_ID
|
||||
bash run_with_adb.sh infer_demo ppliteseg cityscapes_demo.png $DEVICE_ID
|
||||
```
|
||||
@@ -56,4 +56,4 @@ bash run_with_adb.sh infer_demo ppliteseg cityscapes_demo.png $DEVICE_ID
|
||||
|
||||
<img width="640" src="https://user-images.githubusercontent.com/30516196/205544166-9b2719ff-ed82-4908-b90a-095de47392e1.png">
|
||||
|
||||
需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../../../../docs/cn/quantize.md)
|
||||
需要特别注意的是，在 A311D 上部署的模型需要是量化后的模型，模型的量化请参考：[模型量化](../../../quantize/README.md)
|
||||
2
examples/vision/segmentation/paddleseg/a311d/cpp/infer.cc → examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/infer.cc
Executable file → Normal file
2
examples/vision/segmentation/paddleseg/a311d/cpp/infer.cc → examples/vision/segmentation/paddleseg/amlogic/a311d/cpp/infer.cc
Executable file → Normal file
@@ -30,7 +30,7 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
option.SetLiteSubgraphPartitionPath(subgraph_file);
|
||||
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file,option);
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
48
examples/vision/segmentation/paddleseg/cpu-gpu/README_CN.md
Normal file
48
examples/vision/segmentation/paddleseg/cpu-gpu/README_CN.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/deeplabv3/README.md)
|
||||
- [SegFormer系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/segformer/README.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting/)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
| [SegFormer_B0-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-with-argmax.tgz) \| [SegFormer_B0-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-without-argmax.tgz) | 15MB | 1024x1024 | 76.73% | 77.16% | - |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
106
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/README_CN.md
Normal file
106
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/README_CN.md
Normal file
@@ -0,0 +1,106 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成PP-LiteSeg在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../../matting)
|
||||
|
||||
以Linux上推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/cpp-gpu/cpp
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载PP-LiteSeg模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
|
||||
# CPU推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 0
|
||||
# GPU推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 1
|
||||
# GPU上TensorRT推理
|
||||
./infer_demo PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer cityscapes_demo.png 2
|
||||
```
|
||||
|
||||
运行完成可视化结果如下图所示
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
> **注意：**
|
||||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||
- [如何在Windows中使用FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
## PaddleSeg C++接口
|
||||
|
||||
### PaddleSeg类
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
const string& model_file,
|
||||
const string& params_file = "",
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleSegModel模型加载和初始化，其中model_file为导出的Paddle模型格式。
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
#### Predict函数
|
||||
|
||||
> ```c++
|
||||
> PaddleSegModel::Predict(const cv::Mat &im, SegmentationResult *result)
|
||||
> ```
|
||||
>
|
||||
> 模型预测接口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **im**: 输入图像，注意需为HWC，BGR格式
|
||||
> > * **result**: 分割结果，包括分割预测的标签以及标签对应的概率值, SegmentationResult说明参考[SegmentationResult结构体介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
|
||||
### 类成员属性
|
||||
#### 预处理参数
|
||||
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
|
||||
|
||||
> > * **is_vertical_screen**(bool): PP-HumanSeg系列模型通过设置此参数为`true`表明输入图片是竖屏，即height大于width的图片
|
||||
|
||||
#### 后处理参数
|
||||
> > * **apply_softmax**(bool): 当模型导出时，并未指定`apply_softmax`参数，可通过此设置此参数为`true`，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理
|
||||
|
||||
## 快速链接
|
||||
- [PaddleSeg模型介绍](../../)
|
||||
- [Python部署](../python)
|
||||
|
||||
## 常见问题
|
||||
- [如何将模型预测结果SegmentationResult转为numpy格式](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
- [如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [PaddleSeg C++ API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1segmentation.html)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
131
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/infer.cc
Normal file
131
examples/vision/segmentation/paddleseg/cpu-gpu/cpp/infer.cc
Normal file
@@ -0,0 +1,131 @@
|
||||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "fastdeploy/vision.h"
|
||||
|
||||
#ifdef WIN32
|
||||
const char sep = '\\';
|
||||
#else
|
||||
const char sep = '/';
|
||||
#endif
|
||||
|
||||
void CpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseCpu();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void GpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void TrtInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
option.UseGpu();
|
||||
option.UseTrtBackend();
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
if (!model.Initialized()) {
|
||||
std::cerr << "Failed to initialize." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
auto im = cv::imread(image_file);
|
||||
|
||||
fastdeploy::vision::SegmentationResult res;
|
||||
if (!model.Predict(im, &res)) {
|
||||
std::cerr << "Failed to predict." << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
std::cout << res.Str() << std::endl;
|
||||
auto vis_im = fastdeploy::vision::VisSegmentation(im, res, 0.5);
|
||||
cv::imwrite("vis_result.jpg", vis_im);
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
if (argc < 4) {
|
||||
std::cout
|
||||
<< "Usage: infer_demo path/to/model_dir path/to/image run_option, "
|
||||
"e.g ./infer_model ./ppseg_model_dir ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run "
|
||||
"with kunlunxin."
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (std::atoi(argv[3]) == 0) {
|
||||
CpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 1) {
|
||||
GpuInfer(argv[1], argv[2]);
|
||||
} else if (std::atoi(argv[3]) == 2) {
|
||||
TrtInfer(argv[1], argv[2]);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../../matting)
|
||||
|
||||
本目录下提供`infer.py`快速完成PP-LiteSeg在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/cpu-gpu/python
|
||||
|
||||
# 下载Unet模型文件和测试图片
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# CPU推理
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu
|
||||
# GPU推理
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu
|
||||
# GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待）
|
||||
python infer.py --model PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True
|
||||
```
|
||||
|
||||
运行完成可视化结果如下图所示
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
## PaddleSegModel Python接口
|
||||
|
||||
```python
|
||||
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleSeg模型加载和初始化，其中model_file, params_file以及config_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/docs/model_export_cn.md)
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **config_file**(str): 推理部署配置文件
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
|
||||
### predict函数
|
||||
|
||||
> ```python
|
||||
> PaddleSegModel.predict(input_image)
|
||||
> ```
|
||||
>
|
||||
> 模型预测结口，输入图像直接输出检测结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): 输入数据，注意需为HWC，BGR格式
|
||||
|
||||
> **返回**
|
||||
>
|
||||
> > 返回`fastdeploy.vision.SegmentationResult`结构体，结构体说明参考文档[SegmentationResult结构体介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
|
||||
### 类成员属性
|
||||
#### 预处理参数
|
||||
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
|
||||
|
||||
> > * **is_vertical_screen**(bool): PP-HumanSeg系列模型通过设置此参数为`true`表明输入图片是竖屏，即height大于width的图片
|
||||
|
||||
#### 后处理参数
|
||||
> > * **apply_softmax**(bool): 当模型导出时，并未指定`apply_softmax`参数，可通过此设置此参数为`true`，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理
|
||||
|
||||
## 其它文档
|
||||
|
||||
- [PaddleSeg 模型介绍](..)
|
||||
- [PaddleSeg C++部署](../cpp)
|
||||
|
||||
## 常见问题
|
||||
- [如何将模型预测结果SegmentationResult转为numpy格式](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/api/vision_results/segmentation_result_CN.md)
|
||||
- [如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
|
||||
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
|
||||
- [PaddleSeg python API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/semantic_segmentation.html)
|
||||
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
|
||||
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
|
||||
57
examples/vision/segmentation/paddleseg/cpu-gpu/python/infer.py
Executable file
57
examples/vision/segmentation/paddleseg/cpu-gpu/python/infer.py
Executable file
@@ -0,0 +1,57 @@
|
||||
import fastdeploy as fd
|
||||
import cv2
|
||||
import os
|
||||
|
||||
|
||||
def parse_arguments():
|
||||
import argparse
|
||||
import ast
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--model", required=True, help="Path of PaddleSeg model.")
|
||||
parser.add_argument(
|
||||
"--image", type=str, required=True, help="Path of test image file.")
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
default='cpu',
|
||||
help="Type of inference device, support 'kunlunxin', 'cpu' or 'gpu'.")
|
||||
parser.add_argument(
|
||||
"--use_trt",
|
||||
type=ast.literal_eval,
|
||||
default=False,
|
||||
help="Wether to use tensorrt.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_option(args):
|
||||
option = fd.RuntimeOption()
|
||||
|
||||
if args.device.lower() == "gpu":
|
||||
option.use_gpu()
|
||||
|
||||
if args.use_trt:
|
||||
option.use_trt_backend()
|
||||
option.set_trt_input_shape("x", [1, 3, 256, 256], [1, 3, 1024, 1024],
|
||||
[1, 3, 2048, 2048])
|
||||
return option
|
||||
|
||||
|
||||
args = parse_arguments()
|
||||
|
||||
# 配置runtime，加载模型
|
||||
runtime_option = build_option(args)
|
||||
model_file = os.path.join(args.model, "model.pdmodel")
|
||||
params_file = os.path.join(args.model, "model.pdiparams")
|
||||
config_file = os.path.join(args.model, "deploy.yaml")
|
||||
model = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file, params_file, config_file, runtime_option=runtime_option)
|
||||
|
||||
# 预测图片分割结果
|
||||
im = cv2.imread(args.image)
|
||||
result = model.predict(im)
|
||||
print(result)
|
||||
|
||||
# 可视化结果
|
||||
vis_im = fd.vision.vis_segmentation(im, result, weight=0.5)
|
||||
cv2.imwrite("vis_img.png", vis_im)
|
||||
48
examples/vision/segmentation/paddleseg/kunlun/README_CN.md
Normal file
48
examples/vision/segmentation/paddleseg/kunlun/README_CN.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# 使用FastDeploy部署PaddleSeg模型
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleSeg develop](https://github.com/PaddlePaddle/PaddleSeg/tree/develop)
|
||||
|
||||
目前FastDeploy支持如下模型的部署
|
||||
|
||||
- [U-Net系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/unet/README.md)
|
||||
- [PP-LiteSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)
|
||||
- [PP-HumanSeg系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README.md)
|
||||
- [FCN系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/fcn/README.md)
|
||||
- [DeepLabV3系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/deeplabv3/README.md)
|
||||
- [SegFormer系列模型](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/segformer/README.md)
|
||||
|
||||
【注意】如你部署的为**PP-Matting**、**PP-HumanMatting**以及**ModNet**请参考[Matting模型部署](../../matting/)
|
||||
|
||||
## 准备PaddleSeg部署模型
|
||||
PaddleSeg模型导出，请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/model_export_cn.md)
|
||||
|
||||
**注意**
|
||||
- PaddleSeg导出的模型包含`model.pdmodel`、`model.pdiparams`和`deploy.yaml`三个文件，FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了PaddleSeg导出的部分模型
|
||||
- without-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op none`
|
||||
- with-argmax导出方式为：**不指定**`--input_shape`，**指定**`--output_op argmax`
|
||||
|
||||
开发者可直接下载使用。
|
||||
|
||||
| 模型 | 参数文件大小 |输入Shape | mIoU | mIoU (flip) | mIoU (ms+flip) |
|
||||
|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | :----- |
|
||||
| [Unet-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_with_argmax_infer.tgz) \| [Unet-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz) | 52MB | 1024x512 | 65.00% | 66.02% | 66.89% |
|
||||
| [PP-LiteSeg-B(STDC2)-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz) \| [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz) | 31MB | 1024x512 | 79.04% | 79.52% | 79.85% |
|
||||
|[PP-HumanSegV1-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV1_Lite_with_argmax_infer.tgz) \| [PP-HumanSegV1-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Lite_infer.tgz) | 543KB | 192x192 | 86.2% | - | - |
|
||||
|[PP-HumanSegV2-Lite-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Lite-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Lite_192x192_infer.tgz) | 12MB | 192x192 | 92.52% | - | - |
|
||||
| [PP-HumanSegV2-Mobile-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_with_argmax_infer.tgz) \| [PP-HumanSegV2-Mobile-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV2_Mobile_192x192_infer.tgz) | 29MB | 192x192 | 93.13% | - | - |
|
||||
|[PP-HumanSegV1-Server-with-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_with_argmax_infer.tgz) \| [PP-HumanSegV1-Server-without-argmax(通用人像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/PP_HumanSegV1_Server_infer.tgz) | 103MB | 512x512 | 96.47% | - | - |
|
||||
| [Portait-PP-HumanSegV2-Lite-with-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_with_argmax_infer.tgz) \| [Portait-PP-HumanSegV2-Lite-without-argmax(肖像分割模型)](https://bj.bcebos.com/paddlehub/fastdeploy/Portrait_PP_HumanSegV2_Lite_256x144_infer.tgz) | 3.6M | 256x144 | 96.63% | - | - |
|
||||
| [FCN-HRNet-W18-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_with_argmax_infer.tgz) \| [FCN-HRNet-W18-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/FCN_HRNet_W18_cityscapes_without_argmax_infer.tgz)(暂时不支持ONNXRuntime的GPU推理) | 37MB | 1024x512 | 78.97% | 79.49% | 79.74% |
|
||||
| [Deeplabv3-ResNet101-OS8-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_with_argmax_infer.tgz) \| [Deeplabv3-ResNet101-OS8-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/Deeplabv3_ResNet101_OS8_cityscapes_without_argmax_infer.tgz) | 150MB | 1024x512 | 79.90% | 80.22% | 80.47% |
|
||||
| [SegFormer_B0-cityscapes-with-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-with-argmax.tgz) \| [SegFormer_B0-cityscapes-without-argmax](https://bj.bcebos.com/paddlehub/fastdeploy/SegFormer_B0-cityscapes-without-argmax.tgz) | 15MB | 1024x1024 | 76.73% | 77.16% | - |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
@@ -0,0 +1,14 @@
|
||||
PROJECT(infer_demo C CXX)
|
||||
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||
|
||||
# 指定下载解压后的fastdeploy库路径
|
||||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||
|
||||
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
|
||||
|
||||
# 添加FastDeploy依赖头文件
|
||||
include_directories(${FASTDEPLOY_INCS})
|
||||
|
||||
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
|
||||
# 添加FastDeploy库依赖
|
||||
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
|
||||
96
examples/vision/segmentation/paddleseg/kunlun/cpp/README.md
Executable file
96
examples/vision/segmentation/paddleseg/kunlun/cpp/README.md
Executable file
@@ -0,0 +1,96 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleSeg C++ Deployment Example
|
||||
|
||||
This directory provides examples that `infer.cc` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT.
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Download the precompiled deployment library and samples code according to your development environment. Refer to [FastDeploy Precompiled Library](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【Attention】For the deployment of **PP-Matting**、**PP-HumanMatting** and **ModNet**, refer to [Matting Model Deployment](../../../matting)
|
||||
|
||||
Taking the inference on Linux as an example, the compilation test can be completed by executing the following command in this directory. FastDeploy version 1.0.0 or above (x.x.x>=1.0.0) is required to support this model.
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# Download the FastDeploy precompiled library. Users can choose your appropriate version in the `FastDeploy Precompiled Library` mentioned above
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# Download Unet model files and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf Unet_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
|
||||
# CPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 0
|
||||
# GPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 1
|
||||
# TensorRT inference on GPU
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 2
|
||||
# kunlunxin XPU inference
|
||||
./infer_demo Unet_cityscapes_without_argmax_infer cityscapes_demo.png 3
|
||||
```
|
||||
|
||||
The visualized result after running is as follows
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
The above command works for Linux or MacOS. For SDK use-pattern in Windows, refer to:
|
||||
- [How to use FastDeploy C++ SDK in Windows](../../../../../docs/cn/faq/use_sdk_on_windows.md)
|
||||
|
||||
## PaddleSeg C++ Interface
|
||||
|
||||
### PaddleSeg Class
|
||||
|
||||
```c++
|
||||
fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
const string& model_file,
|
||||
const string& params_file = "",
|
||||
const string& config_file,
|
||||
const RuntimeOption& runtime_option = RuntimeOption(),
|
||||
const ModelFormat& model_format = ModelFormat::PADDLE)
|
||||
```
|
||||
|
||||
PaddleSegModel model loading and initialization, among which model_file is the exported Paddle model format.
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
#### Predict Function
|
||||
|
||||
> ```c++
|
||||
> PaddleSegModel::Predict(cv::Mat* im, DetectionResult* result)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output detection results.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **im**: Input images in HWC or BGR format
|
||||
> > * **result**: The segmentation result, including the predicted label of the segmentation and the corresponding probability of the label. Refer to [Vision Model Prediction Results](../../../../../docs/api/vision_results/) for the description of SegmentationResult
|
||||
|
||||
### Class Member Variable
|
||||
#### Pre-processing Parameter
|
||||
Users can modify the following pre-processing parameters to their needs, which affects the final inference and deployment results
|
||||
|
||||
> > * **is_vertical_screen**(bool): For PP-HumanSeg models, the input image is portrait, height greater than a width, by setting this parameter to`true`
|
||||
|
||||
#### Post-processing Parameter
|
||||
> > * **apply_softmax**(bool): The `apply_softmax` parameter is not specified when the model is exported. Set this parameter to `true` to normalize the probability result (score_map) of the predicted output segmentation label (label_map)
|
||||
|
||||
- [Model Description](../../)
|
||||
- [Python Deployment](../python)
|
||||
- [Vision Model Prediction Results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
6
examples/vision/segmentation/paddleseg/cpp/infer.cc → examples/vision/segmentation/paddleseg/kunlun/cpp/infer.cc
Executable file → Normal file
6
examples/vision/segmentation/paddleseg/cpp/infer.cc → examples/vision/segmentation/paddleseg/kunlun/cpp/infer.cc
Executable file → Normal file
@@ -48,7 +48,8 @@ void CpuInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void KunlunXinInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
void KunlunXinInfer(const std::string& model_dir,
|
||||
const std::string& image_file) {
|
||||
auto model_file = model_dir + sep + "model.pdmodel";
|
||||
auto params_file = model_dir + sep + "model.pdiparams";
|
||||
auto config_file = model_dir + sep + "deploy.yaml";
|
||||
@@ -170,7 +171,8 @@ int main(int argc, char* argv[]) {
|
||||
"e.g ./infer_model ./ppseg_model_dir ./test.jpeg 0"
|
||||
<< std::endl;
|
||||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run with kunlunxin."
|
||||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run "
|
||||
"with kunlunxin."
|
||||
<< std::endl;
|
||||
return -1;
|
||||
}
|
||||
82
examples/vision/segmentation/paddleseg/kunlun/python/README.md
Executable file
82
examples/vision/segmentation/paddleseg/kunlun/python/README.md
Executable file
@@ -0,0 +1,82 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
# PaddleSeg Python Deployment Example
|
||||
|
||||
Before deployment, two steps require confirmation
|
||||
|
||||
- 1. Software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Install FastDeploy Python whl package. Refer to [FastDeploy Python Installation](../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
【Attention】For the deployment of **PP-Matting**、**PP-HumanMatting** and **ModNet**, refer to [Matting Model Deployment](../../../matting)
|
||||
|
||||
This directory provides examples that `infer.py` fast finishes the deployment of Unet on CPU/GPU and GPU accelerated by TensorRT. The script is as follows
|
||||
```bash
|
||||
# Download the deployment example code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python
|
||||
|
||||
# Download Unet model files and test images
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz
|
||||
tar -xvf Unet_cityscapes_without_argmax_infer.tgz
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# CPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu
|
||||
# GPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu
|
||||
# TensorRT inference on GPU（Attention: It is somewhat time-consuming for the operation of model serialization when running TensorRT inference for the first time. Please be patient.）
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True
|
||||
# kunlunxin XPU inference
|
||||
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device kunlunxin
|
||||
```
|
||||
|
||||
The visualized result after running is as follows
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/16222477/191712880-91ae128d-247a-43e0-b1e3-cafae78431e0.jpg", width=512px, height=256px />
|
||||
</div>
|
||||
|
||||
## PaddleSegModel Python Interface
|
||||
|
||||
```python
|
||||
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=ModelFormat.PADDLE)
|
||||
```
|
||||
|
||||
PaddleSeg model loading and initialization, among which model_file, params_file, and config_file are the Paddle inference files exported from the training model. Refer to [Model Export](https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.6/docs/model_export_cn.md) for more information
|
||||
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **config_file**(str): Inference deployment configuration file
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration. None by default, which is the default configuration
|
||||
> * **model_format**(ModelFormat): Model format. Paddle format by default
|
||||
|
||||
### predict function
|
||||
|
||||
> ```python
|
||||
> PaddleSegModel.predict(input_image)
|
||||
> ```
|
||||
>
|
||||
> Model prediction interface. Input images and output detection results.
|
||||
>
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **input_image**(np.ndarray): Input data in HWC or BGR format
|
||||
|
||||
> **Return**
|
||||
>
|
||||
> > Return `fastdeploy.vision.SegmentationResult` structure. Refer to [Vision Model Prediction Results](../../../../../docs/api/vision_results/) for the description of the structure.
|
||||
|
||||
### Class Member Variable
|
||||
#### Pre-processing Parameter
|
||||
Users can modify the following pre-processing parameters to their needs, which affects the final inference and deployment results
|
||||
|
||||
> > * **is_vertical_screen**(bool): For PP-HumanSeg models, the input image is portrait with height greater than width by setting this parameter to `true`
|
||||
#### Post-processing Parameter
|
||||
> > * **apply_softmax**(bool): The `apply_softmax` parameter is not specified when the model is exported. Set this parameter to `true` to normalize the probability result (score_map) of the predicted output segmentation label (label_map) in softmax
|
||||
|
||||
## Other Documents
|
||||
|
||||
- [PaddleSeg Model Description](..)
|
||||
- [PaddleSeg C++ Deployment](../cpp)
|
||||
- [Model Prediction Results](../../../../../docs/api/vision_results/)
|
||||
- [How to switch the model inference backend engine](../../../../../docs/cn/faq/how_to_change_backend.md)
|
||||
@@ -5,33 +5,22 @@ FastDeploy已支持部署量化模型,并提供一键模型自动化压缩的工
|
||||
|
||||
## FastDeploy一键模型自动化压缩工具
|
||||
FastDeploy 提供了一键模型自动化压缩工具, 能够简单地通过输入一个配置文件, 对模型进行量化.
|
||||
详细教程请见: [一键模型自动化压缩工具](../../../../../tools/common_tools/auto_compression/)
|
||||
注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的deploy.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
详细教程请见: [一键模型自动化压缩工具](https://github.com/PaddlePaddle/FastDeploy/tree/develop/tools/common_tools/auto_compression)
|
||||
>> **注意**: 推理量化后的分类模型仍然需要FP32模型文件夹下的deploy.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。
|
||||
|
||||
## 下载量化完成的PaddleSeg模型
|
||||
## 量化完成的PaddleSeg模型
|
||||
用户也可以直接下载下表中的量化模型进行部署.(点击模型名字即可下载)
|
||||
|
||||
Benchmark表格说明:
|
||||
- Runtime时延为模型在各种Runtime上的推理时延,包含CPU->GPU数据拷贝,GPU推理,GPU->CPU数据拷贝时间. 不包含模型各自的前后处理时间.
|
||||
- 端到端时延为模型在实际推理场景中的时延, 包含模型的前后处理.
|
||||
- 所测时延均为推理1000次后求得的平均值, 单位是毫秒.
|
||||
- INT8 + FP16 为在推理INT8量化模型的同时, 给Runtime 开启FP16推理选项
|
||||
- INT8 + FP16 + PM, 为在推理INT8量化模型和开启FP16的同时, 开启使用Pinned Memory的选项,可加速GPU->CPU数据拷贝的速度
|
||||
- 最大加速比, 为FP32时延除以INT8推理的最快时延,得到最大加速比.
|
||||
- 策略为量化蒸馏训练时, 采用少量无标签数据集训练得到量化模型, 并在全量验证集上验证精度, INT8精度并不代表最高的INT8精度.
|
||||
- CPU为Intel(R) Xeon(R) Gold 6271C, 所有测试中固定CPU线程数为1. GPU为Tesla T4, TensorRT版本8.4.15.
|
||||
| 模型 | 量化方式 |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) |量化蒸馏训练 |
|
||||
|
||||
#### Runtime Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 Runtime时延 | INT8 Runtime时延 | INT8 + FP16 Runtime时延 | INT8+FP16+PM Runtime时延 | 最大加速比 | FP32 mIoU | INT8 mIoU | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) | Paddle Inference | CPU | 1138.04| 602.62 |None|None | 1.89 |77.37 | 71.62 |量化蒸馏训练 |
|
||||
量化后模型的Benchmark比较，请参考[量化模型 Benchmark](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/quantize.md)
|
||||
|
||||
#### 端到端 Benchmark
|
||||
| 模型 |推理后端 |部署硬件 | FP32 End2End时延 | INT8 End2End时延 | INT8 + FP16 End2End时延 | INT8+FP16+PM End2End时延 | 最大加速比 | FP32 mIoU | INT8 mIoU | 量化方式 |
|
||||
| ------------------- | -----------------|-----------| -------- |-------- |-------- | --------- |-------- |----- |----- |----- |
|
||||
| [PP-LiteSeg-T(STDC1)-cityscapes](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar) | Paddle Inference | CPU | 4726.65| 4134.91|None|None | 1.14 |77.37 | 71.62 |量化蒸馏训练 |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
## 支持部署量化模型的硬件
|
||||
FastDeploy 量化模型部署的过程大致都与FP32模型类似，只是模型量化与非量化的区别，如果硬件在量化模型部署过程有特殊处理，也会在文档中特别标明，因此量化模型部署可以参考如下硬件的链接
|
||||
- [NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU](../cpu-gpu)
|
||||
- [昆仑](../kunlun)
|
||||
- [升腾](../ascend)
|
||||
- [瑞芯微](../rockchip)
|
||||
- [晶晨](../amlogic)
|
||||
- [算能](../sophgo)
|
||||
|
||||
@@ -16,7 +16,8 @@
|
||||
#include "fastdeploy/vision.h"
|
||||
|
||||
void ONNXInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::string model_file = model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer.onnx";
|
||||
std::string model_file =
|
||||
model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer.onnx";
|
||||
std::string params_file;
|
||||
std::string config_file = model_dir + "/deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
@@ -43,13 +44,12 @@ void ONNXInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
tc.PrintInfo("PPSeg in ONNX");
|
||||
|
||||
cv::imwrite("infer_onnx.jpg", vis_im);
|
||||
std::cout
|
||||
<< "Visualized result saved in ./infer_onnx.jpg"
|
||||
<< std::endl;
|
||||
std::cout << "Visualized result saved in ./infer_onnx.jpg" << std::endl;
|
||||
}
|
||||
|
||||
void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
std::string model_file = model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer_rk3588.rknn";
|
||||
std::string model_file =
|
||||
model_dir + "/Portrait_PP_HumanSegV2_Lite_256x144_infer_rk3588.rknn";
|
||||
std::string params_file;
|
||||
std::string config_file = model_dir + "/deploy.yaml";
|
||||
auto option = fastdeploy::RuntimeOption();
|
||||
@@ -78,9 +78,7 @@ void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
|
||||
tc.PrintInfo("PPSeg in RKNPU2");
|
||||
|
||||
cv::imwrite("infer_rknn.jpg", vis_im);
|
||||
std::cout
|
||||
<< "Visualized result saved in ./infer_rknn.jpg"
|
||||
<< std::endl;
|
||||
std::cout << "Visualized result saved in ./infer_rknn.jpg" << std::endl;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
@@ -93,7 +91,6 @@ int main(int argc, char* argv[]) {
|
||||
}
|
||||
|
||||
RKNPU2Infer(argv[1], argv[2]);
|
||||
// ONNXInfer(argv[1], argv[2]);
|
||||
// ONNXInfer(argv[1], argv[2]);
|
||||
return 0;
|
||||
}
|
||||
|
||||
2
examples/vision/segmentation/paddleseg/rv1126/cpp/infer.cc → examples/vision/segmentation/paddleseg/rockchip/rv1126/cpp/infer.cc
Executable file → Normal file
2
examples/vision/segmentation/paddleseg/rv1126/cpp/infer.cc → examples/vision/segmentation/paddleseg/rockchip/rv1126/cpp/infer.cc
Executable file → Normal file
@@ -30,7 +30,7 @@ void InitAndInfer(const std::string& model_dir, const std::string& image_file) {
|
||||
option.SetLiteSubgraphPartitionPath(subgraph_file);
|
||||
|
||||
auto model = fastdeploy::vision::segmentation::PaddleSegModel(
|
||||
model_file, params_file, config_file,option);
|
||||
model_file, params_file, config_file, option);
|
||||
|
||||
assert(model.Initialized());
|
||||
|
||||
@@ -1,68 +1,9 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSegmentation 服务化部署示例
|
||||
# 使用 FastDeploy 服务化部署 PaddleSeg 模型
|
||||
## FastDeploy 服务化部署介绍
|
||||
在线推理作为企业或个人线上部署模型的最后一环，是工业界必不可少的环节，其中最重要的就是服务化推理框架。FastDeploy 目前提供两种服务化部署方式：simple_serving和fastdeploy_serving。simple_serving 基于Flask框架具有简单高效的特点，可以快速验证线上部署模型的可行性。fastdeploy_serving基于Triton Inference Server框架，是一套完备且性能卓越的服务化部署框架，可用于实际生产。
|
||||
|
||||
在服务化部署前，需确认
|
||||
## 详细部署文档
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
|
||||
|
||||
## 启动服务
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/serving
|
||||
|
||||
#下载yolov5模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 将模型文件放入 models/runtime/1目录下
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdmodel models/runtime/1/
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdiparams models/runtime/1/
|
||||
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改paddleseg_grpc_client.py中的请求端口号
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
#下载测试图片
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient[all]
|
||||
|
||||
# 发送请求
|
||||
python3 paddleseg_grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在CPU上运行ONNXRuntime引擎， 如果要在GPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](../../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
- [fastdeploy serving](fastdeploy_serving)
|
||||
- [simple serving](simple_serving)
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
[English](README.md) | 简体中文
|
||||
# PaddleSeg 服务化部署示例
|
||||
|
||||
在服务化部署前，需确认
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/README_CN.md)
|
||||
|
||||
|
||||
## 启动服务
|
||||
|
||||
```bash
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/serving/fastdeploy_serving
|
||||
|
||||
#下载PP-LiteSeg模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 将模型文件放入 models/runtime/1目录下
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdmodel models/runtime/1/
|
||||
mv PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer/model.pdiparams models/runtime/1/
|
||||
|
||||
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
|
||||
# GPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
|
||||
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改paddleseg_grpc_client.py中的请求端口号
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
|
||||
在物理机器中执行以下命令，发送grpc请求并输出结果
|
||||
```
|
||||
#下载测试图片
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
#安装客户端依赖
|
||||
python3 -m pip install tritonclient[all]
|
||||
|
||||
# 发送请求
|
||||
python3 paddleseg_grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回json格式的检测结果并打印输出:
|
||||
```
|
||||
tm: name: "INPUT"
|
||||
datatype: "UINT8"
|
||||
shape: -1
|
||||
shape: -1
|
||||
shape: -1
|
||||
shape: 3
|
||||
|
||||
output_name: SEG_RESULT
|
||||
Only print the first 20 labels in label_map of SEG_RESULT
|
||||
{'label_map': [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], 'score_map': [], 'shape': [1024, 2048], 'contain_score_map': False}
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在CPU上运行ONNXRuntime引擎， 如果要在GPU或其他推理引擎上运行。 需要修改`models/runtime/config.pbtxt`中配置，详情请参考[配置文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
## 更多部署方式
|
||||
- [使用 VisualDL 进行 Serving 可视化部署](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/vdl_management.md)
|
||||
|
||||
## 常见问题
|
||||
- [如何编写客户端 HTTP/GRPC 请求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/client.md)
|
||||
- [如何编译服务化部署镜像](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/compile.md)
|
||||
- [服务化部署原理及动态Batch介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/demo.md)
|
||||
- [模型仓库介绍](https://github.com/PaddlePaddle/FastDeploy/blob/develop/serving/docs/zh_CN/model_repository.md)
|
||||
@@ -0,0 +1,36 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
# PaddleSegmentation Python Simple Serving Demo
|
||||
|
||||
|
||||
## Environment
|
||||
|
||||
- 1. Prepare environment and install FastDeploy Python whl, refer to [download_prebuilt_libraries](../../../../../../docs/en/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
Server:
|
||||
```bash
|
||||
# Download demo code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# Download PP_LiteSeg model
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# Launch server, change the configurations in server.py to select hardware, backend, etc.
|
||||
# and use --host, --port to specify IP and port
|
||||
fastdeploy simple_serving --app server:app
|
||||
```
|
||||
|
||||
Client:
|
||||
```bash
|
||||
# Download demo code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# Download test image
|
||||
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png
|
||||
|
||||
# Send request and get inference result (Please adapt the IP and port if necessary)
|
||||
python client.py
|
||||
```
|
||||
@@ -0,0 +1,32 @@
|
||||
简体中文 | [English](README.md)
|
||||
|
||||
# PaddleSeg Python轻量服务化部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
服务端：
|
||||
```bash
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/vision/segmentation/paddleseg/python/serving
|
||||
|
||||
# 下载PP-LiteSeg模型文件
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
tar -xvf PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer.tgz
|
||||
|
||||
# 启动服务，可修改server.py中的配置项来指定硬件、后端等
|
||||
# 可通过--host、--port指定IP和端口号
|
||||
fastdeploy simple_serving --app server:app
|
||||
```
|
||||
|
||||
客户端：
|
||||
```bash
|
||||
# 下载测试图片
|
||||
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
|
||||
|
||||
# 请求服务，获取推理结果（如有必要，请修改脚本中的IP和端口号）
|
||||
python client.py
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
import requests
|
||||
import json
|
||||
import cv2
|
||||
import fastdeploy as fd
|
||||
from fastdeploy.serving.utils import cv2_to_base64
|
||||
|
||||
if __name__ == '__main__':
|
||||
url = "http://127.0.0.1:8000/fd/ppliteseg"
|
||||
headers = {"Content-Type": "application/json"}
|
||||
|
||||
im = cv2.imread("cityscapes_demo.png")
|
||||
data = {"data": {"image": cv2_to_base64(im)}, "parameters": {}}
|
||||
|
||||
resp = requests.post(url=url, headers=headers, data=json.dumps(data))
|
||||
if resp.status_code == 200:
|
||||
r_json = json.loads(resp.json()["result"])
|
||||
result = fd.vision.utils.json_to_segmentation(r_json)
|
||||
vis_im = fd.vision.vis_segmentation(im, result, weight=0.5)
|
||||
cv2.imwrite("visualized_result.jpg", vis_im)
|
||||
print("Visualized result save in ./visualized_result.jpg")
|
||||
else:
|
||||
print("Error code:", resp.status_code)
|
||||
print(resp.text)
|
||||
@@ -0,0 +1,38 @@
|
||||
import fastdeploy as fd
|
||||
from fastdeploy.serving.server import SimpleServer
|
||||
import os
|
||||
import logging
|
||||
|
||||
logging.getLogger().setLevel(logging.INFO)
|
||||
|
||||
# Configurations
|
||||
model_dir = 'PP_LiteSeg_B_STDC2_cityscapes_with_argmax_infer'
|
||||
device = 'cpu'
|
||||
use_trt = False
|
||||
|
||||
# Prepare model
|
||||
model_file = os.path.join(model_dir, "model.pdmodel")
|
||||
params_file = os.path.join(model_dir, "model.pdiparams")
|
||||
config_file = os.path.join(model_dir, "deploy.yaml")
|
||||
|
||||
# Setup runtime option to select hardware, backend, etc.
|
||||
option = fd.RuntimeOption()
|
||||
if device.lower() == 'gpu':
|
||||
option.use_gpu()
|
||||
if use_trt:
|
||||
option.use_trt_backend()
|
||||
option.set_trt_cache_file('pp_lite_seg.trt')
|
||||
|
||||
# Create model instance
|
||||
model_instance = fd.vision.segmentation.PaddleSegModel(
|
||||
model_file=model_file,
|
||||
params_file=params_file,
|
||||
config_file=config_file,
|
||||
runtime_option=option)
|
||||
|
||||
# Create server, setup REST API
|
||||
app = SimpleServer()
|
||||
app.register(
|
||||
task_name="fd/ppliteseg",
|
||||
model_handler=fd.serving.handler.VisionModelHandler,
|
||||
predictor=model_instance)
|
||||
@@ -18,7 +18,14 @@ Here we take [PP-LiteSeg-B(STDC2)-cityscapes-without-argmax](https://bj.bcebos.c
|
||||
|
||||
### Download PP-LiteSeg-B(STDC2)-cityscapes-without-argmax, and convert it to ONNX
|
||||
```shell
|
||||
https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
# Download Paddle2ONNX repository.
|
||||
git clone https://github.com/PaddlePaddle/Paddle2ONNX
|
||||
|
||||
# Download the Paddle static map model and fix the input shape.
|
||||
## Go to the directory where the input shape is fixed for the Paddle static map model.
|
||||
cd Paddle2ONNX/tools/paddle
|
||||
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
|
||||
# Modify the input shape of PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer model from dynamic input to constant input.
|
||||
|
||||
@@ -18,7 +18,14 @@ SOPHGO-TPU部署模型前需要将Paddle模型转换成bmodel模型，具体步
|
||||
|
||||
### 下载PP-LiteSeg-B(STDC2)-cityscapes-without-argmax模型,并转换为ONNX模型
|
||||
```shell
|
||||
https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
# 下载Paddle2ONNX仓库
|
||||
git clone https://github.com/PaddlePaddle/Paddle2ONNX
|
||||
|
||||
# 下载Paddle静态图模型并为Paddle静态图模型固定输入shape
|
||||
## 进入为Paddle静态图模型固定输入shape的目录
|
||||
cd Paddle2ONNX/tools/paddle
|
||||
|
||||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
tar xvf PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer.tgz
|
||||
|
||||
# 修改PP_LiteSeg_B_STDC2_cityscapes_without_argmax_infer模型的输入shape，由动态输入变成固定输入
|
||||
|
||||
Reference in New Issue
Block a user