mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-05 16:48:03 +08:00
[Tutorial] Add ResNet Support and FastDeploy Tutorial for External Models (#347)
* first commit for yolov7 * pybind for yolov7 * CPP README.md * CPP README.md * modified yolov7.cc * README.md * python file modify * delete license in fastdeploy/ * repush the conflict part * README.md modified * README.md modified * file path modified * file path modified * file path modified * file path modified * file path modified * README modified * README modified * move some helpers to private * add examples for yolov7 * api.md modified * api.md modified * api.md modified * YOLOv7 * yolov7 release link * yolov7 release link * yolov7 release link * copyright * change some helpers to private * change variables to const and fix documents. * gitignore * Transfer some funtions to private member of class * Transfer some funtions to private member of class * Merge from develop (#9) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * first commit for yolor * for merge * Develop (#11) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * Yolor (#16) * Develop (#11) (#12) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * Develop (#13) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * Develop (#14) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <928090362@qq.com> * add is_dynamic for YOLO series (#22) * modify ppmatting backend and docs * modify ppmatting docs * fix the PPMatting size problem * fix LimitShort's log * retrigger ci * modify PPMatting docs * modify the way for dealing with LimitShort * first commit for ResNet and AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * modify AddModel.md * add explaination to code * modify add_new_model.md * move add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * add C++ Comment * add link to add_new_model.md * modify for doxygen * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * modify add_new_model.md * add resnet explaination and modify docs Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <928090362@qq.com>
This commit is contained in:
ziqi-jin
268
.new_docs/cn/faq/add_new_model.md
Normal file
268
.new_docs/cn/faq/add_new_model.md
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
# FastDeploy外部模型集成指引
|
||||||
|
|
||||||
|
在FastDeploy里面新增一个模型，包括增加C++/Python的部署支持。 本文以torchvision v0.12.0中的ResNet50模型为例，介绍使用FastDeploy做外部[模型集成](#modelsupport)，具体包括如下3步。
|
||||||
|
|
||||||
|
| 步骤 | 说明 | 创建或修改的文件 |
|
||||||
|
|:------:|:-------------------------------------:|:---------------------------------------------:|
|
||||||
|
| [1](#step2) | 在fastdeploy/vision相应任务模块增加模型实现 | resnet.h、resnet.cc、vision.h |
|
||||||
|
| [2](#step4) | 通过pybind完成Python接口绑定 | resnet_pybind.cc、classification_pybind.cc |
|
||||||
|
| [3](#step5) | 实现Python相应调用接口 | resnet.py、\_\_init\_\_.py |
|
||||||
|
|
||||||
|
在完成上述3步之后，一个外部模型就集成好了。
|
||||||
|
<br />
|
||||||
|
如果您想为FastDeploy贡献代码，还需要为新增模型添加测试代码、说明文档和代码注释，可在[测试](#test)中查看。
|
||||||
|
## 模型集成 <span id="modelsupport"></span>
|
||||||
|
|
||||||
|
### 模型准备 <span id="step1"></span>
|
||||||
|
|
||||||
|
|
||||||
|
在集成外部模型之前，先要将训练好的模型（.pt，.pdparams 等）转换成FastDeploy支持部署的模型格式（.onnx，.pdmodel）。多数开源仓库会提供模型转换脚本，可以直接利用脚本做模型的转换。由于torchvision没有提供转换脚本，因此手动编写转换脚本，本文中将 `torchvison.models.resnet50` 转换为 `resnet50.onnx`， 参考代码如下：
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
import torchvision.models as models
|
||||||
|
model = models.resnet50(pretrained=True)
|
||||||
|
batch_size = 1 #批处理大小
|
||||||
|
input_shape = (3, 224, 224) #输入数据,改成自己的输入shape
|
||||||
|
model.eval()
|
||||||
|
x = torch.randn(batch_size, *input_shape) # 生成张量
|
||||||
|
export_onnx_file = "resnet50.onnx" # 目的ONNX文件名
|
||||||
|
torch.onnx.export(model,
|
||||||
|
x,
|
||||||
|
export_onnx_file,
|
||||||
|
opset_version=12,
|
||||||
|
input_names=["input"], # 输入名
|
||||||
|
output_names=["output"], # 输出名
|
||||||
|
dynamic_axes={"input":{0:"batch_size"}, # 批处理变量
|
||||||
|
"output":{0:"batch_size"}})
|
||||||
|
```
|
||||||
|
执行上述脚本将会得到 `resnet50.onnx` 文件。
|
||||||
|
|
||||||
|
### C++部分 <span id="step2"></span>
|
||||||
|
* 创建`resnet.h`文件
|
||||||
|
* 创建位置
|
||||||
|
* FastDeploy/fastdeploy/vision/classification/contrib/resnet.h (FastDeploy/C++代码存放位置/视觉模型/任务名称/外部模型/模型名.h)
|
||||||
|
* 创建内容
|
||||||
|
* 首先在resnet.h中创建 ResNet类并继承FastDeployModel父类，之后声明`Predict`、`Initialize`、`Preprocess`、`Postprocess`和`构造函数`，以及必要的变量，具体的代码细节请参考[resnet.h](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-69128489e918f305c208476ba793d8167e77de2aa7cadf5dcbac30da448bd28e)。
|
||||||
|
|
||||||
|
```C++
|
||||||
|
class FASTDEPLOY_DECL ResNet : public FastDeployModel {
|
||||||
|
public:
|
||||||
|
ResNet(...);
|
||||||
|
virtual bool Predict(...);
|
||||||
|
private:
|
||||||
|
bool Initialize();
|
||||||
|
bool Preprocess(...);
|
||||||
|
bool Postprocess(...);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
* 创建`resnet.cc`文件
|
||||||
|
* 创建位置
|
||||||
|
* FastDeploy/fastdeploy/vision/classification/contrib/resnet.cc (FastDeploy/C++代码存放位置/视觉模型/任务名称/外部模型/模型名.cc)
|
||||||
|
* 创建内容
|
||||||
|
* 在`resnet.cc`中实现`resnet.h`中声明函数的具体逻辑，其中`PreProcess` 和 `PostProcess`需要参考源官方库的前后处理逻辑复现，ResNet每个函数具体逻辑如下，具体的代码请参考[resnet.cc](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-d229d702de28345253a53f2a5839fd2c638f3d32fffa6a7d04d23db9da13a871)。
|
||||||
|
|
||||||
|
```C++
|
||||||
|
ResNet::ResNet(...) {
|
||||||
|
// 构造函数逻辑
|
||||||
|
// 1. 指定 Backend 2. 设置RuntimeOption 3. 调用Initialize()函数
|
||||||
|
}
|
||||||
|
bool ResNet::Initialize() {
|
||||||
|
// 初始化逻辑
|
||||||
|
// 1. 全局变量赋值 2. 调用InitRuntime()函数
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
bool ResNet::Preprocess(Mat* mat, FDTensor* output) {
|
||||||
|
// 前处理逻辑
|
||||||
|
// 1. Resize 2. BGR2RGB 3. Normalize 4. HWC2CHW 5. 处理结果存入 FDTensor类中
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
bool ResNet::Postprocess(FDTensor& infer_result, ClassifyResult* result, int topk) {
|
||||||
|
//后处理逻辑
|
||||||
|
// 1. Softmax 2. Choose topk labels 3. 结果存入 ClassifyResult类
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
bool ResNet::Predict(cv::Mat* im, ClassifyResult* result, int topk) {
|
||||||
|
Preprocess(...)
|
||||||
|
Infer(...)
|
||||||
|
Postprocess(...)
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
<span id="step3"></span>
|
||||||
|
* 在`vision.h`文件中加入新增模型文件
|
||||||
|
* 修改位置
|
||||||
|
* FastDeploy/fastdeploy/vision.h
|
||||||
|
* 修改内容
|
||||||
|
|
||||||
|
```C++
|
||||||
|
#ifdef ENABLE_VISION
|
||||||
|
#include "fastdeploy/vision/classification/contrib/resnet.h"
|
||||||
|
#endif
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Pybind部分 <span id="step4"></span>
|
||||||
|
|
||||||
|
* 创建Pybind文件
|
||||||
|
* 创建位置
|
||||||
|
* FastDeploy/fastdeploy/vision/classification/contrib/resnet_pybind.cc (FastDeploy/C++代码存放位置/视觉模型/任务名称/外部模型/模型名_pybind.cc)
|
||||||
|
* 创建内容
|
||||||
|
* 利用Pybind将C++中的函数变量绑定到Python中，具体代码请参考[resnet_pybind.cc](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-270af0d65720310e2cfbd5373c391b2110d65c0f4efa547f7b7eeffcb958bdec)。
|
||||||
|
```C++
|
||||||
|
void BindResNet(pybind11::module& m) {
|
||||||
|
pybind11::class_<vision::classification::ResNet, FastDeployModel>(
|
||||||
|
m, "ResNet")
|
||||||
|
.def(pybind11::init<std::string, std::string, RuntimeOption, ModelFormat>())
|
||||||
|
.def("predict", ...)
|
||||||
|
.def_readwrite("size", &vision::classification::ResNet::size)
|
||||||
|
.def_readwrite("mean_vals", &vision::classification::ResNet::mean_vals)
|
||||||
|
.def_readwrite("std_vals", &vision::classification::ResNet::std_vals);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
* 调用Pybind函数
|
||||||
|
* 修改位置
|
||||||
|
* FastDeploy/fastdeploy/vision/classification/classification_pybind.cc (FastDeploy/C++代码存放位置/视觉模型/任务名称/任务名称}_pybind.cc)
|
||||||
|
* 修改内容
|
||||||
|
```C++
|
||||||
|
void BindResNet(pybind11::module& m);
|
||||||
|
void BindClassification(pybind11::module& m) {
|
||||||
|
auto classification_module =
|
||||||
|
m.def_submodule("classification", "Image classification models.");
|
||||||
|
BindResNet(classification_module);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Python部分 <span id="step5"></span>
|
||||||
|
|
||||||
|
|
||||||
|
* 创建`resnet.py`文件
|
||||||
|
* 创建位置
|
||||||
|
* FastDeploy/python/fastdeploy/vision/classification/contrib/resnet.py (FastDeploy/Python代码存放位置/fastdeploy/视觉模型/任务名称/外部模型/模型名.py)
|
||||||
|
* 创建内容
|
||||||
|
* 创建ResNet类继承自FastDeployModel，实现 `\_\_init\_\_`、Pybind绑定的函数（如`predict()`）、以及`对Pybind绑定的全局变量进行赋值和获取的函数`，具体代码请参考[resnet.py](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-a4dc5ec2d450e91f1c03819bf314c238b37ac678df56d7dea3aab7feac10a157)。
|
||||||
|
|
||||||
|
```python
|
||||||
|
|
||||||
|
class ResNet(FastDeployModel):
|
||||||
|
def __init__(self, ...):
|
||||||
|
self._model = C.vision.classification.ResNet(...)
|
||||||
|
def predict(self, input_image, topk=1):
|
||||||
|
return self._model.predict(input_image, topk)
|
||||||
|
@property
|
||||||
|
def size(self):
|
||||||
|
return self._model.size
|
||||||
|
@size.setter
|
||||||
|
def size(self, wh):
|
||||||
|
...
|
||||||
|
```
|
||||||
|
<span id="step6"></span>
|
||||||
|
* 导入ResNet类
|
||||||
|
* 修改位置
|
||||||
|
* FastDeploy/python/fastdeploy/vision/classification/\_\_init\_\_.py (FastDeploy/Python代码存放位置/fastdeploy/视觉模型/任务名称/\_\_init\_\_.py)
|
||||||
|
* 修改内容
|
||||||
|
|
||||||
|
```Python
|
||||||
|
from .contrib.resnet import ResNet
|
||||||
|
```
|
||||||
|
|
||||||
|
## 测试 <span id="test"></span>
|
||||||
|
### 编译
|
||||||
|
* C++
|
||||||
|
* 位置：FastDeploy/
|
||||||
|
|
||||||
|
```
|
||||||
|
mkdir build & cd build
|
||||||
|
cmake .. -DENABLE_ORT_BACKEND=ON -DENABLE_VISION=ON -DCMAKE_INSTALL_PREFIX=${PWD/fastdeploy-0.0.3
|
||||||
|
-DENABLE_PADDLE_BACKEND=ON -DENABLE_TRT_BACKEND=ON -DWITH_GPU=ON -DTRT_DIRECTORY=/PATH/TO/TensorRT/
|
||||||
|
make -j8
|
||||||
|
make install
|
||||||
|
```
|
||||||
|
|
||||||
|
编译会得到 build/fastdeploy-0.0.3/。
|
||||||
|
|
||||||
|
* Python
|
||||||
|
* 位置：FastDeploy/python/
|
||||||
|
|
||||||
|
```
|
||||||
|
export TRT_DIRECTORY=/PATH/TO/TensorRT/ # 如果用TensorRT 需要填写TensorRT所在位置，并开启 ENABLE_TRT_BACKEND
|
||||||
|
export ENABLE_TRT_BACKEND=ON
|
||||||
|
export WITH_GPU=ON
|
||||||
|
export ENABLE_PADDLE_BACKEND=ON
|
||||||
|
export ENABLE_OPENVINO_BACKEND=ON
|
||||||
|
export ENABLE_VISION=ON
|
||||||
|
export ENABLE_ORT_BACKEND=ON
|
||||||
|
python setup.py build
|
||||||
|
python setup.py bdist_wheel
|
||||||
|
cd dist
|
||||||
|
pip install fastdeploy_gpu_python-版本号-cpxx-cpxxm-系统架构.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
### 编写测试代码
|
||||||
|
* 创建位置: FastDeploy/examples/vision/classification/resnet/ (FastDeploy/示例目录/视觉模型/任务名称/模型名/)
|
||||||
|
* 创建目录结构
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── cpp
|
||||||
|
│ ├── CMakeLists.txt
|
||||||
|
│ ├── infer.cc // C++ 版本测试代码
|
||||||
|
│ └── README.md // C++版本使用文档
|
||||||
|
├── python
|
||||||
|
│ ├── infer.py // Python 版本测试代码
|
||||||
|
│ └── README.md // Python版本使用文档
|
||||||
|
└── README.md // ResNet 模型集成说明文档
|
||||||
|
```
|
||||||
|
|
||||||
|
* C++
|
||||||
|
* 编写CmakeLists文件、C++ 代码以及 README.md 内容请参考[cpp/](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-afcbe607b796509581f89e38b84190717f1eeda2df0419a2ac9034197ead5f96)。
|
||||||
|
* 编译 infer.cc
|
||||||
|
* 位置：FastDeploy/examples/vision/classification/resnet/cpp/
|
||||||
|
|
||||||
|
```
|
||||||
|
mkdir build & cd build
|
||||||
|
cmake .. -DFASTDEPLOY_INSTALL_DIR=/PATH/TO/FastDeploy/build/fastdeploy-0.0.3/
|
||||||
|
make
|
||||||
|
```
|
||||||
|
|
||||||
|
* Python
|
||||||
|
* Python 代码以及 README.md 内容请参考[python/](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-5a0d6be8c603a8b81454ac14c17fb93555288d9adf92bbe40454449309700135)。
|
||||||
|
|
||||||
|
### 为代码添加注释
|
||||||
|
为了方便用户理解代码，我们需要为新增代码添加注释，添加注释方法可参考如下示例。
|
||||||
|
- C++ 代码
|
||||||
|
您需要在resnet.h文件中为函数和变量增加注释，有如下三种注释方式，具体可参考[resnet.h](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-69128489e918f305c208476ba793d8167e77de2aa7cadf5dcbac30da448bd28e)。
|
||||||
|
|
||||||
|
```C++
|
||||||
|
/** \brief Predict for the input "im", the result will be saved in "result".
|
||||||
|
*
|
||||||
|
* \param[in] im Input image for inference.
|
||||||
|
* \param[in] result Saving the inference result.
|
||||||
|
* \param[in] topk The length of return values, e.g., if topk==2, the result will include the 2 most possible class label for input image.
|
||||||
|
*/
|
||||||
|
virtual bool Predict(cv::Mat* im, ClassifyResult* result, int topk = 1);
|
||||||
|
|
||||||
|
/// Tuple of (width, height)
|
||||||
|
std::vector<int> size;
|
||||||
|
/*! @brief Initialize for ResNet model, assign values to the global variables and call InitRuntime()
|
||||||
|
*/
|
||||||
|
bool Initialize();
|
||||||
|
```
|
||||||
|
- Python 代码
|
||||||
|
你需要为resnet.py文件中的函数和变量增加适当的注释，示例如下，具体可参考[resnet.py](https://github.com/PaddlePaddle/FastDeploy/pull/347/files#diff-a4dc5ec2d450e91f1c03819bf314c238b37ac678df56d7dea3aab7feac10a157)。
|
||||||
|
|
||||||
|
```python
|
||||||
|
def predict(self, input_image, topk=1):
|
||||||
|
"""Classify an input image
|
||||||
|
|
||||||
|
:param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format
|
||||||
|
:param topk: (int)The topk result by the classify confidence score, default 1
|
||||||
|
:return: ClassifyResult
|
||||||
|
"""
|
||||||
|
return self._model.predict(input_image, topk)
|
||||||
|
```
|
||||||
|
|
||||||
|
对于集成模型过程中的其他文件，您也可以对实现的细节添加适当的注释说明。
|
||||||
@@ -15,3 +15,11 @@
|
|||||||
:members:
|
:members:
|
||||||
:inherited-members:
|
:inherited-members:
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## fastdeploy.vision.classification.ResNet
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
|
.. autoclass:: fastdeploy.vision.classification.ResNet
|
||||||
|
:members:
|
||||||
|
:inherited-members:
|
||||||
|
```
|
||||||
|
|||||||
53
examples/vision/classification/resnet/README.md
Normal file
53
examples/vision/classification/resnet/README.md
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
# ResNet准备部署模型
|
||||||
|
|
||||||
|
- ResNet部署实现来自[Torchvision](https://github.com/pytorch/vision/tree/v0.12.0)的代码，和[基于ImageNet2012的预训练模型](https://github.com/pytorch/vision/tree/v0.12.0)。
|
||||||
|
|
||||||
|
- （1）[官方库](https://github.com/pytorch/vision/tree/v0.12.0)提供的*.pt通过[导出ONNX模型](#导出ONNX模型)操作后，可进行部署；
|
||||||
|
- （2）自己数据训练的ResNet模型，按照[导出ONNX模型](#%E5%AF%BC%E5%87%BAONNX%E6%A8%A1%E5%9E%8B)操作后，参考[详细部署文档](#详细部署文档)完成部署。
|
||||||
|
|
||||||
|
|
||||||
|
## 导出ONNX模型
|
||||||
|
|
||||||
|
|
||||||
|
导入[Torchvision](https://github.com/pytorch/vision/tree/v0.12.0)，加载预训练模型，并进行模型转换，具体转换步骤如下。
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
import torchvision.models as models
|
||||||
|
|
||||||
|
model = models.resnet50(pretrained=True)
|
||||||
|
batch_size = 1 #批处理大小
|
||||||
|
input_shape = (3, 224, 224) #输入数据,改成自己的输入shape
|
||||||
|
# #set the model to inference mode
|
||||||
|
model.eval()
|
||||||
|
x = torch.randn(batch_size, *input_shape) # 生成张量
|
||||||
|
export_onnx_file = "ResNet50.onnx" # 目的ONNX文件名
|
||||||
|
torch.onnx.export(model,
|
||||||
|
x,
|
||||||
|
export_onnx_file,
|
||||||
|
opset_version=12,
|
||||||
|
input_names=["input"], # 输入名
|
||||||
|
output_names=["output"], # 输出名
|
||||||
|
dynamic_axes={"input":{0:"batch_size"}, # 批处理变量
|
||||||
|
"output":{0:"batch_size"}})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 下载预训练ONNX模型
|
||||||
|
|
||||||
|
为了方便开发者的测试，下面提供了ResNet导出的各系列模型，开发者可直接下载使用。（下表中模型的精度来源于源官方库）
|
||||||
|
| 模型 | 大小 | 精度 |
|
||||||
|
|:---------------------------------------------------------------- |:----- |:----- |
|
||||||
|
| [ResNet-18](https://bj.bcebos.com/paddlehub/fastdeploy/resnet18.onnx) | 45MB | |
|
||||||
|
| [ResNet-34](https://bj.bcebos.com/paddlehub/fastdeploy/resnet34.onnx) | 84MB | |
|
||||||
|
| [ResNet-50](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50.onnx) | 98MB | |
|
||||||
|
| [ResNet-101](https://bj.bcebos.com/paddlehub/fastdeploy/resnet101.onnx) | 170MB | |
|
||||||
|
|
||||||
|
|
||||||
|
## 详细部署文档
|
||||||
|
|
||||||
|
- [Python部署](python)
|
||||||
|
- [C++部署](cpp)
|
||||||
|
|
||||||
|
## 版本说明
|
||||||
|
|
||||||
|
- 本版本文档和代码基于[Torchvision v0.12.0](https://github.com/pytorch/vision/tree/v0.12.0) 编写
|
||||||
14
examples/vision/classification/resnet/cpp/CMakeLists.txt
Normal file
14
examples/vision/classification/resnet/cpp/CMakeLists.txt
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
PROJECT(infer_demo C CXX)
|
||||||
|
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
|
||||||
|
|
||||||
|
# 指定下载解压后的fastdeploy库路径
|
||||||
|
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
|
||||||
|
|
||||||
|
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
|
||||||
|
|
||||||
|
# 添加FastDeploy依赖头文件
|
||||||
|
include_directories(${FASTDEPLOY_INCS})
|
||||||
|
|
||||||
|
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
|
||||||
|
# 添加FastDeploy库依赖
|
||||||
|
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
|
||||||
77
examples/vision/classification/resnet/cpp/README.md
Normal file
77
examples/vision/classification/resnet/cpp/README.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
# ResNet C++部署示例
|
||||||
|
|
||||||
|
本目录下提供`infer.cc`快速完成ResNet系列模型在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。
|
||||||
|
|
||||||
|
在部署前，需确认以下两个步骤
|
||||||
|
|
||||||
|
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/environment.md)
|
||||||
|
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../../docs/quick_start)
|
||||||
|
|
||||||
|
以Linux上 ResNet50 推理为例，在本目录执行如下命令即可完成编译测试
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#下载SDK，编译模型examples代码（SDK中包含了examples代码）
|
||||||
|
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-gpu-0.2.1.tgz
|
||||||
|
tar xvf fastdeploy-linux-x64-gpu-0.2.1.tgz
|
||||||
|
cd fastdeploy-linux-x64-gpu-0.2.1/examples/vision/classification/resnet/cpp
|
||||||
|
mkdir build
|
||||||
|
cd build
|
||||||
|
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../../../../../fastdeploy-linux-x64-gpu-0.2.1
|
||||||
|
make -j
|
||||||
|
|
||||||
|
# 下载ResNet模型文件和测试图片
|
||||||
|
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50.onnx
|
||||||
|
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||||
|
|
||||||
|
|
||||||
|
# CPU推理
|
||||||
|
./infer_demo resnet50.onnx ILSVRC2012_val_00000010.jpeg 0
|
||||||
|
# GPU推理
|
||||||
|
./infer_demo resnet50.onnx ILSVRC2012_val_00000010.jpeg 1
|
||||||
|
# GPU上TensorRT推理
|
||||||
|
./infer_demo resnet50.onnx ILSVRC2012_val_00000010.jpeg 2
|
||||||
|
```
|
||||||
|
|
||||||
|
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
|
||||||
|
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/compile/how_to_use_sdk_on_windows.md)
|
||||||
|
|
||||||
|
## ResNet C++接口
|
||||||
|
|
||||||
|
### ResNet类
|
||||||
|
|
||||||
|
```c++
|
||||||
|
|
||||||
|
fastdeploy::vision::classification::ResNet(
|
||||||
|
const std::string& model_file,
|
||||||
|
const std::string& params_file = "",
|
||||||
|
const RuntimeOption& custom_option = RuntimeOption(),
|
||||||
|
const ModelFormat& model_format = ModelFormat::ONNX)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
**参数**
|
||||||
|
|
||||||
|
> * **model_file**(str): 模型文件路径
|
||||||
|
> * **params_file**(str): 参数文件路径
|
||||||
|
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||||
|
> * **model_format**(ModelFormat): 模型格式，默认为ONNX格式
|
||||||
|
|
||||||
|
#### Predict函数
|
||||||
|
|
||||||
|
> ```c++
|
||||||
|
> ResNet::Predict(cv::Mat* im, ClassifyResult* result, int topk = 1)
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> 模型预测接口，输入图像直接输出检测结果。
|
||||||
|
>
|
||||||
|
> **参数**
|
||||||
|
>
|
||||||
|
> > * **im**: 输入图像，注意需为HWC，BGR格式
|
||||||
|
> > * **result**: 分类结果，包括label_id，以及相应的置信度, ClassifyResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||||
|
> > * **topk**(int):返回预测概率最高的topk个分类结果，默认为1
|
||||||
|
|
||||||
|
|
||||||
|
- [模型介绍](../../)
|
||||||
|
- [Python部署](../python)
|
||||||
|
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||||
|
- [如何切换模型推理后端引擎](../../../../../docs/runtime/how_to_change_backend.md)
|
||||||
94
examples/vision/classification/resnet/cpp/infer.cc
Normal file
94
examples/vision/classification/resnet/cpp/infer.cc
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
#include "fastdeploy/vision.h"
|
||||||
|
|
||||||
|
void CpuInfer(const std::string& model_file, const std::string& image_file) {
|
||||||
|
auto model = fastdeploy::vision::classification::ResNet(model_file);
|
||||||
|
if (!model.Initialized()) {
|
||||||
|
std::cerr << "Failed to initialize." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto im = cv::imread(image_file);
|
||||||
|
|
||||||
|
fastdeploy::vision::ClassifyResult res;
|
||||||
|
if (!model.Predict(&im, &res)) {
|
||||||
|
std::cerr << "Failed to predict." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
std::cout << res.Str() << std::endl;
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
void GpuInfer(const std::string& model_file, const std::string& image_file) {
|
||||||
|
auto option = fastdeploy::RuntimeOption();
|
||||||
|
option.UseGpu();
|
||||||
|
auto model = fastdeploy::vision::classification::ResNet(model_file, "", option);
|
||||||
|
if (!model.Initialized()) {
|
||||||
|
std::cerr << "Failed to initialize." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto im = cv::imread(image_file);
|
||||||
|
|
||||||
|
fastdeploy::vision::ClassifyResult res;
|
||||||
|
if (!model.Predict(&im, &res)) {
|
||||||
|
std::cerr << "Failed to predict." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
std::cout << res.Str() << std::endl;
|
||||||
|
}
|
||||||
|
|
||||||
|
void TrtInfer(const std::string& model_file, const std::string& image_file) {
|
||||||
|
auto option = fastdeploy::RuntimeOption();
|
||||||
|
option.UseGpu();
|
||||||
|
option.UseTrtBackend();
|
||||||
|
option.SetTrtInputShape("images", {1, 3, 224, 224});
|
||||||
|
auto model = fastdeploy::vision::classification::ResNet(model_file, "", option);
|
||||||
|
if (!model.Initialized()) {
|
||||||
|
std::cerr << "Failed to initialize." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto im = cv::imread(image_file);
|
||||||
|
|
||||||
|
fastdeploy::vision::ClassifyResult res;
|
||||||
|
if (!model.Predict(&im, &res)) {
|
||||||
|
std::cerr << "Failed to predict." << std::endl;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
std::cout << res.Str() << std::endl;
|
||||||
|
}
|
||||||
|
|
||||||
|
int main(int argc, char* argv[]) {
|
||||||
|
if (argc < 4) {
|
||||||
|
std::cout << "Usage: infer_demo path/to/model path/to/image run_option, "
|
||||||
|
"e.g ./infer_model ./resnet50.onnx ./test.jpeg 0"
|
||||||
|
<< std::endl;
|
||||||
|
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
|
||||||
|
"with gpu; 2: run with gpu and use tensorrt backend."
|
||||||
|
<< std::endl;
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (std::atoi(argv[3]) == 0) {
|
||||||
|
CpuInfer(argv[1], argv[2]);
|
||||||
|
} else if (std::atoi(argv[3]) == 1) {
|
||||||
|
GpuInfer(argv[1], argv[2]);
|
||||||
|
} else if (std::atoi(argv[3]) == 2) {
|
||||||
|
TrtInfer(argv[1], argv[2]);
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
72
examples/vision/classification/resnet/python/README.md
Normal file
72
examples/vision/classification/resnet/python/README.md
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
# ResNet模型 Python部署示例
|
||||||
|
|
||||||
|
在部署前，需确认以下两个步骤
|
||||||
|
|
||||||
|
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../../docs/environment.md)
|
||||||
|
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../../docs/quick_start)
|
||||||
|
|
||||||
|
本目录下提供`infer.py`快速完成ResNet50_vd在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#下载部署示例代码
|
||||||
|
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||||
|
cd FastDeploy/examples/vision/classification/resnet/python
|
||||||
|
|
||||||
|
# 下载ResNet50_vd模型文件和测试图片
|
||||||
|
wget https://bj.bcebos.com/paddlehub/fastdeploy/resnet50.onnx
|
||||||
|
wget https://gitee.com/paddlepaddle/PaddleClas/raw/release/2.4/deploy/images/ImageNet/ILSVRC2012_val_00000010.jpeg
|
||||||
|
|
||||||
|
# CPU推理
|
||||||
|
python infer.py --model resnet50.onnx --image ILSVRC2012_val_00000010.jpeg --device cpu --topk 1
|
||||||
|
# GPU推理
|
||||||
|
python infer.py --model resnet50.onnx --image ILSVRC2012_val_00000010.jpeg --device gpu --topk 1
|
||||||
|
# GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待）
|
||||||
|
python infer.py --model resnet50.onnx --image ILSVRC2012_val_00000010.jpeg --device gpu --use_trt True --topk 1
|
||||||
|
```
|
||||||
|
|
||||||
|
运行完成后返回结果如下所示
|
||||||
|
```bash
|
||||||
|
ClassifyResult(
|
||||||
|
label_ids: 332,
|
||||||
|
scores: 0.825349,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## ResNet Python接口
|
||||||
|
|
||||||
|
```python
|
||||||
|
fd.vision.classification.ResNet(model_file, params_file, runtime_option=None, model_format=ModelFormat.ONNX)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
**参数**
|
||||||
|
|
||||||
|
> * **model_file**(str): 模型文件路径
|
||||||
|
> * **params_file**(str): 参数文件路径
|
||||||
|
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||||
|
> * **model_format**(ModelFormat): 模型格式，默认为ONNX格式
|
||||||
|
|
||||||
|
### predict函数
|
||||||
|
|
||||||
|
> ```python
|
||||||
|
> ResNet.predict(input_image, topk=1)
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> 模型预测结口，输入图像直接输出检测结果。
|
||||||
|
>
|
||||||
|
> **参数**
|
||||||
|
>
|
||||||
|
> > * **input_image**(np.ndarray): 输入数据，注意需为HWC，BGR格式
|
||||||
|
> > * **topk**(int):返回预测概率最高的topk个分类结果，默认为1
|
||||||
|
|
||||||
|
> **返回**
|
||||||
|
>
|
||||||
|
> > 返回`fastdeploy.vision.ClassifyResult`结构体，结构体说明参考文档[视觉模型预测结果](../../../../../docs/api/vision_results/)
|
||||||
|
|
||||||
|
|
||||||
|
## 其它文档
|
||||||
|
|
||||||
|
- [ResNet 模型介绍](..)
|
||||||
|
- [ResNet C++部署](../cpp)
|
||||||
|
- [模型预测结果说明](../../../../../docs/api/vision_results/)
|
||||||
|
- [如何切换模型推理后端引擎](../../../../../docs/runtime/how_to_change_backend.md)
|
||||||
50
examples/vision/classification/resnet/python/infer.py
Normal file
50
examples/vision/classification/resnet/python/infer.py
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
import fastdeploy as fd
|
||||||
|
import cv2
|
||||||
|
import os
|
||||||
|
|
||||||
|
|
||||||
|
def parse_arguments():
|
||||||
|
import argparse
|
||||||
|
import ast
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument(
|
||||||
|
"--model", required=True, help="Path of PaddleClas model.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--image", type=str, required=True, help="Path of test image file.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--topk", type=int, default=1, help="Return topk results.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--device",
|
||||||
|
type=str,
|
||||||
|
default='cpu',
|
||||||
|
help="Type of inference device, support 'cpu' or 'gpu'.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--use_trt",
|
||||||
|
type=ast.literal_eval,
|
||||||
|
default=False,
|
||||||
|
help="Wether to use tensorrt.")
|
||||||
|
return parser.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
def build_option(args):
|
||||||
|
option = fd.RuntimeOption()
|
||||||
|
|
||||||
|
if args.device.lower() == "gpu":
|
||||||
|
option.use_gpu()
|
||||||
|
|
||||||
|
if args.use_trt:
|
||||||
|
option.use_trt_backend()
|
||||||
|
return option
|
||||||
|
|
||||||
|
|
||||||
|
args = parse_arguments()
|
||||||
|
|
||||||
|
# 配置runtime，加载模型
|
||||||
|
runtime_option = build_option(args)
|
||||||
|
|
||||||
|
model = fd.vision.classification.ResNet(
|
||||||
|
args.model, runtime_option=runtime_option)
|
||||||
|
# 预测图片分类结果
|
||||||
|
im = cv2.imread(args.image)
|
||||||
|
result = model.predict(im.copy(), args.topk)
|
||||||
|
print(result)
|
||||||
@@ -17,6 +17,7 @@
|
|||||||
#ifdef ENABLE_VISION
|
#ifdef ENABLE_VISION
|
||||||
#include "fastdeploy/vision/classification/contrib/yolov5cls.h"
|
#include "fastdeploy/vision/classification/contrib/yolov5cls.h"
|
||||||
#include "fastdeploy/vision/classification/ppcls/model.h"
|
#include "fastdeploy/vision/classification/ppcls/model.h"
|
||||||
|
#include "fastdeploy/vision/classification/contrib/resnet.h"
|
||||||
#include "fastdeploy/vision/detection/contrib/nanodet_plus.h"
|
#include "fastdeploy/vision/detection/contrib/nanodet_plus.h"
|
||||||
#include "fastdeploy/vision/detection/contrib/scaledyolov4.h"
|
#include "fastdeploy/vision/detection/contrib/scaledyolov4.h"
|
||||||
#include "fastdeploy/vision/detection/contrib/yolor.h"
|
#include "fastdeploy/vision/detection/contrib/yolor.h"
|
||||||
|
|||||||
@@ -18,11 +18,12 @@ namespace fastdeploy {
|
|||||||
|
|
||||||
void BindYOLOv5Cls(pybind11::module& m);
|
void BindYOLOv5Cls(pybind11::module& m);
|
||||||
void BindPaddleClas(pybind11::module& m);
|
void BindPaddleClas(pybind11::module& m);
|
||||||
|
void BindResNet(pybind11::module& m);
|
||||||
void BindClassification(pybind11::module& m) {
|
void BindClassification(pybind11::module& m) {
|
||||||
auto classification_module =
|
auto classification_module =
|
||||||
m.def_submodule("classification", "Image classification models.");
|
m.def_submodule("classification", "Image classification models.");
|
||||||
BindYOLOv5Cls(classification_module);
|
BindYOLOv5Cls(classification_module);
|
||||||
BindPaddleClas(classification_module);
|
BindPaddleClas(classification_module);
|
||||||
|
BindResNet(classification_module);
|
||||||
}
|
}
|
||||||
} // namespace fastdeploy
|
} // namespace fastdeploy
|
||||||
|
|||||||
134
fastdeploy/vision/classification/contrib/resnet.cc
Normal file
134
fastdeploy/vision/classification/contrib/resnet.cc
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
#include "fastdeploy/vision/classification/contrib/resnet.h"
|
||||||
|
#include "fastdeploy/vision/utils/utils.h"
|
||||||
|
#include "fastdeploy/utils/perf.h"
|
||||||
|
|
||||||
|
namespace fastdeploy {
|
||||||
|
namespace vision {
|
||||||
|
namespace classification {
|
||||||
|
|
||||||
|
ResNet::ResNet(const std::string& model_file,
|
||||||
|
const std::string& params_file,
|
||||||
|
const RuntimeOption& custom_option,
|
||||||
|
const ModelFormat& model_format) {
|
||||||
|
// In constructor, the 3 steps below are necessary.
|
||||||
|
// 1. set the Backend 2. set RuntimeOption 3. call Initialize()
|
||||||
|
|
||||||
|
if (model_format == ModelFormat::ONNX) {
|
||||||
|
valid_cpu_backends = {Backend::ORT, Backend::OPENVINO};
|
||||||
|
valid_gpu_backends = {Backend::ORT, Backend::TRT};
|
||||||
|
} else {
|
||||||
|
valid_cpu_backends = {Backend::PDINFER};
|
||||||
|
valid_gpu_backends = {Backend::PDINFER};
|
||||||
|
}
|
||||||
|
runtime_option = custom_option;
|
||||||
|
runtime_option.model_format = model_format;
|
||||||
|
runtime_option.model_file = model_file;
|
||||||
|
runtime_option.params_file = params_file;
|
||||||
|
initialized = Initialize();
|
||||||
|
}
|
||||||
|
|
||||||
|
bool ResNet::Initialize() {
|
||||||
|
|
||||||
|
// In this function, the 3 steps below are necessary.
|
||||||
|
// 1. assign values to the global variables 2. call InitRuntime()
|
||||||
|
|
||||||
|
size = {224, 224};
|
||||||
|
mean_vals = {0.485f, 0.456f, 0.406f};
|
||||||
|
std_vals = {0.229f, 0.224f, 0.225f};
|
||||||
|
|
||||||
|
if (!InitRuntime()) {
|
||||||
|
FDERROR << "Failed to initialize fastdeploy backend." << std::endl;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
bool ResNet::Preprocess(Mat* mat, FDTensor* output) {
|
||||||
|
|
||||||
|
// In this function, the preprocess need be implemented according to the original Repos,
|
||||||
|
// The result of preprocess has to be saved in FDTensor variable, because the input of Infer() need to be std::vector<FDTensor>.
|
||||||
|
// 1. Resize 2. BGR2RGB 3. Normalize 4. HWC2CHW 5. Put the result into FDTensor variable.
|
||||||
|
|
||||||
|
if (mat->Height()!=size[0] || mat->Width()!=size[1]){
|
||||||
|
int interp = cv::INTER_LINEAR;
|
||||||
|
Resize::Run(mat, size[1], size[0], -1, -1, interp);
|
||||||
|
}
|
||||||
|
|
||||||
|
BGR2RGB::Run(mat);
|
||||||
|
Normalize::Run(mat, mean_vals, std_vals);
|
||||||
|
|
||||||
|
HWC2CHW::Run(mat);
|
||||||
|
Cast::Run(mat, "float");
|
||||||
|
mat->ShareWithTensor(output);
|
||||||
|
output->shape.insert(output->shape.begin(), 1); // reshape to n, h, w, c
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool ResNet::Postprocess(FDTensor& infer_result,
|
||||||
|
ClassifyResult* result, int topk) {
|
||||||
|
|
||||||
|
// In this function, the postprocess need be implemented according to the original Repos,
|
||||||
|
// Finally the reslut of postprocess should be saved in ClassifyResult variable.
|
||||||
|
// 1. Softmax 2. Choose topk labels 3. Put the result into ClassifyResult variable.
|
||||||
|
|
||||||
|
int num_classes = infer_result.shape[1];
|
||||||
|
Softmax(infer_result, &infer_result);
|
||||||
|
const float* infer_result_buffer = reinterpret_cast<float*>(infer_result.Data());
|
||||||
|
topk = std::min(num_classes, topk);
|
||||||
|
result->label_ids =
|
||||||
|
utils::TopKIndices(infer_result_buffer, num_classes, topk);
|
||||||
|
result->scores.resize(topk);
|
||||||
|
for (int i = 0; i < topk; ++i) {
|
||||||
|
result->scores[i] = *(infer_result_buffer + result->label_ids[i]);
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool ResNet::Predict(cv::Mat* im, ClassifyResult* result, int topk) {
|
||||||
|
|
||||||
|
// In this function, the Preprocess(), Infer(), and Postprocess() are called sequentially.
|
||||||
|
|
||||||
|
Mat mat(*im);
|
||||||
|
std::vector<FDTensor> processed_data(1);
|
||||||
|
if (!Preprocess(&mat, &(processed_data[0]))) {
|
||||||
|
FDERROR << "Failed to preprocess input data while using model:"
|
||||||
|
<< ModelName() << "." << std::endl;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
processed_data[0].name = InputInfoOfRuntime(0).name;
|
||||||
|
|
||||||
|
std::vector<FDTensor> output_tensors;
|
||||||
|
if (!Infer(processed_data, &output_tensors)) {
|
||||||
|
FDERROR << "Failed to inference while using model:" << ModelName() << "."
|
||||||
|
<< std::endl;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!Postprocess(output_tensors[0], result, topk)) {
|
||||||
|
FDERROR << "Failed to postprocess while using model:" << ModelName() << "."
|
||||||
|
<< std::endl;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
} // namespace classification
|
||||||
|
} // namespace vision
|
||||||
|
} // namespace fastdeploy
|
||||||
74
fastdeploy/vision/classification/contrib/resnet.h
Normal file
74
fastdeploy/vision/classification/contrib/resnet.h
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
#include "fastdeploy/fastdeploy_model.h"
|
||||||
|
#include "fastdeploy/vision/common/processors/transform.h"
|
||||||
|
#include "fastdeploy/vision/common/result.h"
|
||||||
|
|
||||||
|
// The namespace shoulde be
|
||||||
|
// fastdeploy::vision::classification (fastdeploy::vision::${task})
|
||||||
|
namespace fastdeploy {
|
||||||
|
namespace vision {
|
||||||
|
/** \brief All object classification model APIs are defined inside this namespace
|
||||||
|
*
|
||||||
|
*/
|
||||||
|
namespace classification {
|
||||||
|
/*! @brief ResNet series model
|
||||||
|
*/
|
||||||
|
class FASTDEPLOY_DECL ResNet : public FastDeployModel {
|
||||||
|
public:
|
||||||
|
/** \brief Set path of model file and the configuration of runtime.
|
||||||
|
*
|
||||||
|
* \param[in] model_file Path of model file, e.g ./resnet50.onnx
|
||||||
|
* \param[in] params_file Path of parameter file, e.g ppyoloe/model.pdiparams, if the model format is ONNX, this parameter will be ignored
|
||||||
|
* \param[in] custom_option RuntimeOption for inference, the default will use cpu, and choose the backend defined in "valid_cpu_backends"
|
||||||
|
* \param[in] model_format Model format of the loaded model, default is ONNX format
|
||||||
|
*/
|
||||||
|
ResNet(const std::string& model_file,
|
||||||
|
const std::string& params_file = "",
|
||||||
|
const RuntimeOption& custom_option = RuntimeOption(),
|
||||||
|
const ModelFormat& model_format = ModelFormat::ONNX);
|
||||||
|
|
||||||
|
virtual std::string ModelName() const { return "ResNet"; }
|
||||||
|
/** \brief Predict for the input "im", the result will be saved in "result".
|
||||||
|
*
|
||||||
|
* \param[in] im Input image for inference.
|
||||||
|
* \param[in] result Saving the inference result.
|
||||||
|
* \param[in] topk The length of return values, e.g., if topk==2, the result will include the 2 most possible class label for input image.
|
||||||
|
*/
|
||||||
|
virtual bool Predict(cv::Mat* im, ClassifyResult* result, int topk = 1);
|
||||||
|
|
||||||
|
/// Tuple of (width, height)
|
||||||
|
std::vector<int> size;
|
||||||
|
/// Mean parameters for normalize
|
||||||
|
std::vector<float> mean_vals;
|
||||||
|
/// Std parameters for normalize
|
||||||
|
std::vector<float> std_vals;
|
||||||
|
|
||||||
|
|
||||||
|
private:
|
||||||
|
/*! @brief Initialize for ResNet model, assign values to the global variables and call InitRuntime()
|
||||||
|
*/
|
||||||
|
bool Initialize();
|
||||||
|
/// PreProcessing for the input "mat", the result will be saved in "outputs".
|
||||||
|
bool Preprocess(Mat* mat, FDTensor* outputs);
|
||||||
|
/*! @brief PostProcessing for the input "infer_result", the result will be saved in "result".
|
||||||
|
*/
|
||||||
|
bool Postprocess(FDTensor& infer_result, ClassifyResult* result,
|
||||||
|
int topk = 1);
|
||||||
|
};
|
||||||
|
} // namespace classification
|
||||||
|
} // namespace vision
|
||||||
|
} // namespace fastdeploy
|
||||||
40
fastdeploy/vision/classification/contrib/resnet_pybind.cc
Normal file
40
fastdeploy/vision/classification/contrib/resnet_pybind.cc
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
#include "fastdeploy/pybind/main.h"
|
||||||
|
// namespace should be `fastdeploy`
|
||||||
|
namespace fastdeploy {
|
||||||
|
// the name of Pybind function should be Bind${model_name}
|
||||||
|
void BindResNet(pybind11::module& m) {
|
||||||
|
// the constructor and the predict funtion are necessary
|
||||||
|
// the constructor is used to initialize the python model class.
|
||||||
|
// the necessary public functions and variables like `size`, `mean_vals` should also be binded.
|
||||||
|
pybind11::class_<vision::classification::ResNet, FastDeployModel>(
|
||||||
|
m, "ResNet")
|
||||||
|
.def(pybind11::init<std::string, std::string, RuntimeOption,
|
||||||
|
ModelFormat>())
|
||||||
|
.def("predict",
|
||||||
|
[](vision::classification::ResNet& self, pybind11::array& data,
|
||||||
|
int topk = 1) {
|
||||||
|
auto mat = PyArrayToCvMat(data);
|
||||||
|
vision::ClassifyResult res;
|
||||||
|
self.Predict(&mat, &res, topk);
|
||||||
|
return res;
|
||||||
|
})
|
||||||
|
.def_readwrite("size", &vision::classification::ResNet::size)
|
||||||
|
.def_readwrite("mean_vals", &vision::classification::ResNet::mean_vals)
|
||||||
|
.def_readwrite("std_vals", &vision::classification::ResNet::std_vals);
|
||||||
|
}
|
||||||
|
} // namespace fastdeploy
|
||||||
|
|
||||||
@@ -15,7 +15,7 @@ from __future__ import absolute_import
|
|||||||
|
|
||||||
from .contrib.yolov5cls import YOLOv5Cls
|
from .contrib.yolov5cls import YOLOv5Cls
|
||||||
from .ppcls import PaddleClasModel
|
from .ppcls import PaddleClasModel
|
||||||
|
from .contrib.resnet import ResNet
|
||||||
PPLCNet = PaddleClasModel
|
PPLCNet = PaddleClasModel
|
||||||
PPLCNetv2 = PaddleClasModel
|
PPLCNetv2 = PaddleClasModel
|
||||||
EfficientNet = PaddleClasModel
|
EfficientNet = PaddleClasModel
|
||||||
|
|||||||
96
python/fastdeploy/vision/classification/contrib/resnet.py
Normal file
96
python/fastdeploy/vision/classification/contrib/resnet.py
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
from __future__ import absolute_import
|
||||||
|
import logging
|
||||||
|
from .... import FastDeployModel, ModelFormat
|
||||||
|
from .... import c_lib_wrap as C
|
||||||
|
|
||||||
|
|
||||||
|
class ResNet(FastDeployModel):
|
||||||
|
def __init__(self,
|
||||||
|
model_file,
|
||||||
|
params_file="",
|
||||||
|
runtime_option=None,
|
||||||
|
model_format=ModelFormat.ONNX):
|
||||||
|
"""Load a image classification model exported by ResNet.
|
||||||
|
|
||||||
|
:param model_file: (str)Path of model file, e.g resnet/resnet50.onnx
|
||||||
|
:param params_file: (str)Path of parameters file, if the model_fomat is ModelFormat.ONNX, this param will be ignored, can be set as empty string
|
||||||
|
:param runtime_option: (fastdeploy.RuntimeOption)RuntimeOption for inference this model, if it's None, will use the default backend on CPU
|
||||||
|
:param model_format: (fastdeploy.ModelForamt)Model format of the loaded model, default is ONNX
|
||||||
|
"""
|
||||||
|
|
||||||
|
# call super() to initialize the backend_option
|
||||||
|
# the result of initialization will be saved in self._runtime_option
|
||||||
|
super(ResNet, self).__init__(runtime_option)
|
||||||
|
|
||||||
|
self._model = C.vision.classification.ResNet(
|
||||||
|
model_file, params_file, self._runtime_option, model_format)
|
||||||
|
# self.initialized shows the initialization of the model is successful or not
|
||||||
|
|
||||||
|
assert self.initialized, "ResNet initialize failed."
|
||||||
|
|
||||||
|
# Predict and return the inference result of "input_image".
|
||||||
|
def predict(self, input_image, topk=1):
|
||||||
|
"""Classify an input image
|
||||||
|
|
||||||
|
:param input_image: (numpy.ndarray)The input image data, 3-D array with layout HWC, BGR format
|
||||||
|
:param topk: (int)The topk result by the classify confidence score, default 1
|
||||||
|
:return: ClassifyResult
|
||||||
|
"""
|
||||||
|
return self._model.predict(input_image, topk)
|
||||||
|
|
||||||
|
# Implement the setter and getter method for variables
|
||||||
|
@property
|
||||||
|
def size(self):
|
||||||
|
"""
|
||||||
|
Returns the preprocess image size
|
||||||
|
"""
|
||||||
|
return self._model.size
|
||||||
|
|
||||||
|
@property
|
||||||
|
def mean_vals(self):
|
||||||
|
"""
|
||||||
|
Returns the mean value of normlization
|
||||||
|
"""
|
||||||
|
return self._model.mean_vals
|
||||||
|
|
||||||
|
@property
|
||||||
|
def std_vals(self):
|
||||||
|
"""
|
||||||
|
Returns the std value of normlization
|
||||||
|
"""
|
||||||
|
return self._model.std_vals
|
||||||
|
|
||||||
|
@size.setter
|
||||||
|
def size(self, wh):
|
||||||
|
assert isinstance(wh, (list, tuple)),\
|
||||||
|
"The value to set `size` must be type of tuple or list."
|
||||||
|
assert len(wh) == 2,\
|
||||||
|
"The value to set `size` must contatins 2 elements means [width, height], but now it contains {} elements.".format(
|
||||||
|
len(wh))
|
||||||
|
self._model.size = wh
|
||||||
|
|
||||||
|
@mean_vals.setter
|
||||||
|
def mean_vals(self, value):
|
||||||
|
assert isinstance(
|
||||||
|
value, list), "The value to set `mean_vals` must be type of list."
|
||||||
|
self._model.mean_vals = value
|
||||||
|
|
||||||
|
@std_vals.setter
|
||||||
|
def std_vals(self, value):
|
||||||
|
assert isinstance(
|
||||||
|
value, list), "The value to set `std_vals` must be type of list."
|
||||||
|
self._model.std_vals = value
|
||||||
Reference in New Issue
Block a user