apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-06 00:57:33 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Tools] Add x2paddle tool (#631)

Browse Source 
* add onnx_ort_runtime demo

* rm in requirements

* support batch eval

* fixed MattingResults bug

* move assignment for DetectionResult

* integrated x2paddle

* add model convert readme

* update readme

* re-lint

Co-authored-by: Jason <jiangjiajun@baidu.com>
This commit is contained in:
WJJ1995
2022-11-21 11:01:28 +08:00
committed by GitHub
parent c7ec14de95
commit 053537f4b9
5 changed files with 186 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/cn/build_and_install/download_prebuilt_libraries.md
View File

@@ -3,8 +3,10 @@
FastDeploy提供各平台预编译库，供开发者直接下载安装使用。当然FastDeploy编译也非常容易，开发者也可根据自身需求编译FastDeploy。
本文分为两部分：
- [1.GPU部署环境](##GPU部署环境)
- [2.CPU部署环境](##CPU部署环境)
- [1.GPU部署环境](#1)
- [2.CPU部署环境](#2)
<p id="1"></p>
## GPU部署环境
@@ -49,6 +51,8 @@ Develop版本（Nightly build）
| Linux x64 | [fastdeploy-linux-x64-gpu-0.0.0.tgz](https://fastdeploy.bj.bcebos.com/dev/cpp/fastdeploy-linux-x64-gpu-0.0.0.tgz) | g++ 8.2, CUDA 11.2, cuDNN 8.2编译产出 |
| Windows x64 | [fastdeploy-win-x64-gpu-0.0.0.zip](https://fastdeploy.bj.bcebos.com/dev/cpp/fastdeploy-win-x64-gpu-0.0.0.zip) | Visual Studio 16 2019, CUDA 11.2, cuDNN 8.2编译产出 |
<p id="2"></p>
## CPU部署环境
### 环境要求

8
docs/en/build_and_install/download_prebuilt_libraries.md
View File

@@ -3,8 +3,10 @@
FastDeploy provides pre-built libraries for developers to download and install directly. Meanwhile, FastDeploy also offers easy access to compile so that developers can compile FastDeploy according to their own needs.
This article is divided into two parts:
- [1.GPU Deployment Environment](#gpu-deployment-environment)
- [2.CPU Deployment Environment](#cpu-deployment-environment)
- [1.GPU Deployment Environment](#1)
- [2.CPU Deployment Environment](#2)
<p id="1"></p>
## GPU Deployment Environment
@@ -53,6 +55,8 @@ Install the Develop version（Nightly build）
| Linux x64 | [fastdeploy-linux-x64-gpu-0.0.0.tgz](https://fastdeploy.bj.bcebos.com/dev/cpp/fastdeploy-linux-x64-gpu-0.0.0.tgz) | g++ 8.2, CUDA 11.2, cuDNN 8.2 |
| Windows x64 | [fastdeploy-win-x64-gpu-0.0.0.zip](https://fastdeploy.bj.bcebos.com/dev/cpp/fastdeploy-win-x64-gpu-0.0.0.zip) | Visual Studio 16 2019, CUDA 11.2, cuDNN 8.2 |
<p id="2"></p>
## CPU Deployment Environment
### Environment Requirement

47
tools/README.md Normal file → Executable file
View File

@@ -1,8 +1,14 @@
# FastDeploy 工具包
FastDeploy提供了一系列高效易用的工具优化部署体验, 提升推理性能.
例如, FastDeploy基于PaddleSlim的Auto Compression Toolkit(ACT), 给用户提供了一键模型自动化压缩的工具, 用户可以轻松地通过一行命令对模型进行自动化压缩, 并在FastDeploy上部署压缩后的模型, 提升推理速度. 本文档将以FastDeploy一键模型自动化压缩工具为例, 介绍如何安装此工具, 并提供相应的使用文档.
## FastDeploy一键模型自动化压缩工具
- [1.自动压缩工具包](#1)
- [2.模型转换工具包](#2)
<p id="1"></p>
## 一键模型自动化压缩工具
FastDeploy基于PaddleSlim的Auto Compression Toolkit(ACT), 给用户提供了一键模型自动化压缩的工具, 用户可以轻松地通过一行命令对模型进行自动化压缩, 并在FastDeploy上部署压缩后的模型, 提升推理速度. 本文档将以FastDeploy一键模型自动化压缩工具为例, 介绍如何安装此工具, 并提供相应的使用文档.
### 环境准备
1.用户参考PaddlePaddle官网, 安装develop版本
@@ -33,3 +39,40 @@ python setup.py install
fastdeploy --auto_compress --config_path=./configs/detection/yolov5s_quant.yaml --method='PTQ' --save_dir='./yolov5s_ptq_model/'
```
详细使用文档请参考[FastDeploy一键模型自动化压缩工具](./auto_compression/README.md)
<p id="2"></p>
## 模型转换工具
FastDeploy 基于 X2Paddle 为用户提供了模型转换的工具, 用户可以轻松地通过一行命令将外部框架模型快速迁移至飞桨框架，目前支持 ONNX、TensorFlow 以及 Caffe，支持大部分主流的CV和NLP的模型转换。
### 环境准备
1. PaddlePaddle 安装，可参考如下文档快速安装
```
https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html
```
2. X2Paddle 安装
如需使用稳定版本，可通过pip方式安装X2Paddle：
```shell
pip install x2paddle
```
如需体验最新功能，可使用源码安装方式：
```shell
git clone https://github.com/PaddlePaddle/X2Paddle.git
cd X2Paddle
python setup.py install
```
### 使用方式
按照以上步骤成功安装后,即可使用 FastDeploy 一键转换工具, 示例如下:
```bash
fastdeploy --convert --framework onnx --model yolov5s.onnx --save_dir pd_model
```
更多详细内容可参考[X2Paddle](https://github.com/PaddlePaddle/X2Paddle)

46
tools/README_EN.md Normal file → Executable file
View File

@@ -1,9 +1,14 @@
# FastDeploy Toolkit
FastDeploy provides a series of efficient and easy-to-use tools to optimize the deployment experience and improve inference performance.
For example, based on PaddleSlim's Auto Compression Toolkit (ACT), FastDeploy provides users with a one-click model automation compression tool that allows users to easily compress the model with a single command. This document will take FastDeploy's one-click model automation compression tool as an example, introduce how to install the tool, and provide the corresponding documentation for usage.
- [1.Auto Compression Tool](#1)
- [2.Model Conversion Tool](#2)
## FastDeploy One-Click Model Auto Compression Tool
<p id="1"></p>
## One-Click Model Auto Compression Tool
Based on PaddleSlim's Auto Compression Toolkit (ACT), FastDeploy provides users with a one-click model automation compression tool that allows users to easily compress the model with a single command. This document will take FastDeploy's one-click model automation compression tool as an example, introduce how to install the tool, and provide the corresponding documentation for usage.
### Environmental Preparation
1.Install PaddlePaddle develop version
@@ -33,3 +38,40 @@ After the above steps are successfully installed, you can use FastDeploy one-cli
fastdeploy --auto_compress --config_path=./configs/detection/yolov5s_quant.yaml --method='PTQ' --save_dir='./yolov5s_ptq_model/'
```
For detailed documentation, please refer to [FastDeploy One-Click Model Auto Compression Tool](./auto_compression/README.md)
<p id="2"></p>
## Model Conversion Tool
Based on X2Paddle, FastDeploy provides users with a model conversion tool. Users can easily migrate external framework models to the Paddle framework with one line of commands. Currently, ONNX, TensorFlow and Caffe are supported, and most mainstream CV and NLP model conversions are supported.
### Environmental Preparation
1. Install PaddlePaddle, refer to the following documents for quick installation
```
https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html
```
2. Install X2Paddle
To use the stable version, install X2Paddle via pip:
```shell
pip install x2paddle
```
To experience the latest features, you can use the source installation method:
```shell
git clone https://github.com/PaddlePaddle/X2Paddle.git
cd X2Paddle
python setup.py install
```
### How to use
After successful installation according to the above steps, you can use the FastDeploy one-click conversion tool. The example is as follows:
```bash
fastdeploy --convert --framework onnx --model yolov5s.onnx --save_dir pd_model
```
For more details, please refer to[X2Paddle](https://github.com/PaddlePaddle/X2Paddle)

93
tools/common_tools/common_tools.py Normal file → Executable file
View File

@@ -1,8 +1,8 @@
import argparse
import ast
def argsparser():
parser = argparse.ArgumentParser(description=__doc__)
## argumentments for auto compression
parser.add_argument('--auto_compress', default=False, action='store_true')
@@ -10,31 +10,71 @@ def argsparser():
'--config_path',
type=str,
default=None,
help="path of compression strategy config.",
required=True)
help="path of compression strategy config.")
parser.add_argument(
'--method',
type=str,
default=None,
help="choose PTQ or QAT as quantization method",
required=True)
help="choose PTQ or QAT as quantization method")
parser.add_argument(
'--save_dir',
type=str,
default='./output',
help="directory to save compressed model.")
help="directory to save model.")
parser.add_argument(
'--devices',
type=str,
default='gpu',
help="which device used to compress.")
## arguments for other x2paddle
parser.add_argument('--convert', default=False, action='store_true')
parser.add_argument(
'--framework',
type=str,
default=None,
help="define which deeplearning framework(tensorflow/caffe/onnx)")
parser.add_argument(
'--model',
type=str,
default=None,
help="define model file path for tensorflow or onnx")
parser.add_argument(
"--prototxt",
"-p",
type=str,
default=None,
help="prototxt file of caffe model")
parser.add_argument(
"--weight",
"-w",
type=str,
default=None,
help="weight file of caffe model")
parser.add_argument(
"--caffe_proto",
"-c",
type=str,
default=None,
help="optional: the .py file compiled by caffe proto file of caffe model"
)
parser.add_argument(
"--input_shape_dict",
"-isd",
type=str,
default=None,
help="define input shapes, e.g --input_shape_dict=\"{'image':[1, 3, 608, 608]}\" or" \
"--input_shape_dict=\"{'image':[1, 3, 608, 608], 'im_shape': [1, 2], 'scale_factor': [1, 2]}\"")
parser.add_argument(
"--enable_code_optim",
"-co",
type=ast.literal_eval,
default=False,
help="Turn on code optimization")
## arguments for other tools
return parser
def main():
args = argsparser().parse_args()
if args.auto_compress == True:
try:
@@ -45,6 +85,43 @@ def main():
print(
"Can not start auto compresssion successfully! Please check if you have installed it!"
)
if args.convert == True:
try:
import platform
import logging
v0, v1, v2 = platform.python_version().split('.')
if not (int(v0) >= 3 and int(v1) >= 5):
logging.info("[ERROR] python>=3.5 is required")
return
import paddle
v0, v1, v2 = paddle.__version__.split('.')
logging.info("paddle.__version__ = {}".format(paddle.__version__))
if v0 == '0' and v1 == '0' and v2 == '0':
logging.info(
"[WARNING] You are use develop version of paddlepaddle")
elif int(v0) != 2 or int(v1) < 0:
logging.info("[ERROR] paddlepaddle>=2.0.0 is required")
return
from x2paddle.convert import tf2paddle, caffe2paddle, onnx2paddle
if args.framework == "tensorflow":
assert args.model is not None, "--model should be defined while convert tensorflow model"
tf2paddle(args.model, args.save_dir)
elif args.framework == "caffe":
assert args.prototxt is not None and args.weight is not None, "--prototxt and --weight should be defined while convert caffe model"
caffe2paddle(args.prototxt, args.weight, args.save_dir,
args.caffe_proto)
elif args.framework == "onnx":
assert args.model is not None, "--model should be defined while convert onnx model"
onnx2paddle(
args.model,
args.save_dir,
input_shape_dict=args.input_shape_dict)
else:
raise Exception(
"--framework only support tensorflow/caffe/onnx now")
except ImportError:
print(
"Model convert failed! Please check if you have installed it!")
if __name__ == '__main__':