apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-05 16:48:03 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
Files
c8db4b442ad0868fb7135826fa5290b9b39ced56
FastDeploy/examples/runtime/python
History
WJJ1995 c8db4b442a [Runtime] Add Poros Backend Runtime demo for c++/python (#915) 
* add onnx_ort_runtime demo

* rm in requirements

* support batch eval

* fixed MattingResults bug

* move assignment for DetectionResult

* integrated x2paddle

* add model convert readme

* update readme

* re-lint

* add processor api

* Add MattingResult Free

* change valid_cpu_backends order

* add ppocr benchmark

* mv bs from 64 to 32

* fixed quantize.md

* fixed quantize bugs

* Add Monitor for benchmark

* update mem monitor

* Set trt_max_batch_size default 1

* fixed ocr benchmark bug

* support yolov5 in serving

* Fixed yolov5 serving

* Fixed postprocess

* update yolov5 to 7.0

* add poros runtime demos

* update readme

* Support poros abi=1

* rm useless note

* deal with comments

Co-authored-by: Jason <jiangjiajun@baidu.com>
2022-12-20 19:03:14 +08:00
..
infer_onnx_onnxruntime.py
[Other] Add onnx_ort_runtime cpp/python demos (#565)
2022-11-11 12:47:06 +08:00
infer_onnx_openvino.py
Rename pybind/fastdeploy_runtime.cc to pybind/runtime.cc (#273)
2022-09-23 11:16:02 +08:00
infer_onnx_tensorrt.py
Rename pybind/fastdeploy_runtime.cc to pybind/runtime.cc (#273)
2022-09-23 11:16:02 +08:00
infer_paddle_onnxruntime.py
Optimize ocr system code (#209)
2022-09-14 09:46:03 +08:00
infer_paddle_openvino.py
Optimize ocr system code (#209)
2022-09-14 09:46:03 +08:00
infer_paddle_paddle_inference.py
Rename PaddleBackend to PaddleInferBackend (#728)
2022-11-28 21:29:09 +08:00
infer_paddle_tensorrt.py
[Doc] Add runtime demo in quick_start (#524)
2022-11-08 19:37:30 +08:00
infer_torchscript_poros.py
[Runtime] Add Poros Backend Runtime demo for c++/python (#915)
2022-12-20 19:03:14 +08:00
README.md
[Doc] Fix dead links (#584)
2022-11-14 18:44:33 +08:00

README.md

Python推理

在运行demo前，需确认以下两个步骤

本文档以 PaddleClas 分类模型 MobileNetV2 为例展示 CPU 上的推理示例

1. 获取模型

import fastdeploy as fd

model_url = "https://bj.bcebos.com/fastdeploy/models/mobilenetv2.tgz"
fd.download_and_decompress(model_url, path=".")

2. 配置后端

option = fd.RuntimeOption()

option.set_model_path("mobilenetv2/inference.pdmodel",
                      "mobilenetv2/inference.pdiparams")

# **** CPU 配置 ****
option.use_cpu()
option.use_ort_backend()
option.set_cpu_thread_num(12)

# 初始化构造runtime
runtime = fd.Runtime(option)

# 获取模型输入名
input_name = runtime.get_input_info(0).name

# 构造随机数据进行推理
results = runtime.infer({
    input_name: np.random.rand(1, 3, 224, 224).astype("float32")
})

print(results[0].shape)

加载完成，会输出提示如下，说明初始化的后端，以及运行的硬件设备

[INFO] fastdeploy/fastdeploy_runtime.cc(283)::Init	Runtime initialized with Backend::OrtBackend in device Device::CPU.

其它文档