apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-05 16:48:03 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
1,731 Commits 23 Branches 7 Tags
f73a538f619e23725af4898e3354858e1b2db15b
Commit Graph

33 Commits

Author SHA1 Message Date
DefTruth
f73a538f61 [Backend] support bechmark mode for runtime and backend (#1201) 
* [backend] support bechmark mode for runtime and backend

* [backend] support bechmark mode for runtime and backend

* [pybind11] add benchmark methods pybind

* [pybind11] add benchmark methods pybind

* [Other] Update build scripts

* [Other] Update cmake/summary.cmake

* [Other] update build scripts

* [Other] add ENABLE_BENCHMARK option -> setup.py

* optimize backend time recording

* optimize backend time recording

* optimize trt backend time record

* [backend] optimze backend_time recording for trt

* [benchmark] remove redundant logs

* fixed ov_backend confilct

* [benchmark] fixed paddle_backend conflicts

* [benchmark] fixed paddle_backend conflicts

* [benchmark] fixed paddle_backend conflicts

* [benchmark] remove use_gpu option from ort backend option

* [benchmark] update benchmark_ppdet.py

* [benchmark] update benchmark_ppcls.py

* fixed lite backend conflicts

* [Lite] fixed lite xpu

* add benchmark macro

* add RUNTIME_PROFILE_LOOP macros

* add comments for RUNTIME_PROFILE macros

* add comments for new apis

* add comments for new apis

* update benchmark_ppdet.py

* afixed bugs

* remove unused codes

* optimize RUNTIME_PROFILE_LOOP macros

* optimize RUNTIME_PROFILE_LOOP macros

* add comments for benchmark option and result

* add docs for benchmark namespace
 2023-02-06 14:29:35 +08:00
huangjianhui
76df90afc3 [Other] FastDeploy TensorRT && ONNX backend support to load model form memory (#1130) 
* Update all backends load model from buffer

* Delete redundant code

* Format code style

* Format code style

* Delete redundant code

* Delete redundant code

* Add some FDASSERTs

* Update load model form memory when cloning engine

* Update clone engine code

* Update set_model_buffer api parameters with char pointer

* Release memory buffer variables after finish init backends

* Fix conflict

* Fix bug
 2023-02-01 11:36:09 +08:00
Dantès
34bea7649d [Backend] add sophgo backend (#1015) 
* Add Sophgo Device

add sophgo backend in fastdeploy

add resnet50, yolov5s, liteseg examples.

* replace sophgo lib with download links; fix model.cc bug

* modify CodeStyle

* remove unuseful files;change the names of sophgo device and sophgo
backend

* sophgo support python and add python examples

* remove unuseful rows in cmake according pr

Co-authored-by: Zilong Xing <zilong.xing@sophgo.com>
 2023-01-04 15:49:17 +08:00
yeliang2258
45865c8724 [Other] Change all XPU to KunlunXin (#973) 
* [FlyCV] Bump up FlyCV -> official release 1.0.0

* XPU to KunlunXin

* update

* update model link

* update doc

* update device

* update code

* useless code

Co-authored-by: DefTruth <qiustudent_r@163.com>
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2022-12-27 10:02:02 +08:00
yunyaoXYY
d45382e3cc [Backend] Add Huawei Ascend NPU deploy using PaddleLite CANN. (#757) 
* Add Huawei Ascend NPU deploy through PaddleLite CANN

* Add NNAdapter interface for paddlelite

* Modify Huawei Ascend Cmake

* Update way for compiling Huawei Ascend NPU deployment

* remove UseLiteBackend in UseCANN

* Support compile python whlee

* Change names of nnadapter API

* Add nnadapter pybind and remove useless API

* Support Python deployment on Huawei Ascend NPU

* Add models suppor for ascend

* Add PPOCR rec reszie for ascend

* fix conflict for ascend

* Rename CANN to Ascend

* Rename CANN to Ascend

* Improve ascend

* fix ascend bug

* improve ascend docs

* improve ascend docs

* improve ascend docs

* Improve Ascend

* Improve Ascend

* Move ascend python demo

* Imporve ascend

* Improve ascend

* Improve ascend

* Improve ascend

* Improve ascend

* Imporve ascend

* Imporve ascend

* Improve ascend
 2022-12-26 10:18:34 +08:00
huangjianhui
291db315c8 [Other]Fastdeploy supports set_model_buffer function for encrypted model (#930) 
* Update keypointdetection result docs

* Update im.copy() to im in examples

* Update new Api, fastdeploy::vision::Visualize to fastdeploy::vision

* Update SwapBackgroundSegmentation && SwapBackgroundMatting to SwapBackground

* Update README_CN.md

* Update README_CN.md

* Support set_model_buffer function
 2022-12-21 14:21:28 +08:00
Jason
2d998223ac [Backend] Add KunlunXin XPU deploy support (#894) 
Revert "Revert "[Backend] Add KunlunXin XPU deploy support" (#893)"

This reverts commit 0990ab9b50.
 2022-12-16 11:22:30 +08:00
Jason
0990ab9b50 Revert "[Backend] Add KunlunXin XPU deploy support" (#893) 
Revert "[Backend] Add KunlunXin XPU deploy support (#747)"

This reverts commit 5be839b322.
 2022-12-15 21:17:59 +08:00
yeliang2258
5be839b322 [Backend] Add KunlunXin XPU deploy support (#747) 
* add xpu support

* fix docs

* update code

* update doc

* update code

* update yolov5

* update cmake

* add int64_t data support

* fix

* update download links

* add en doc

* update code

* update xpu options

* update doc

* update doc

* update doc

* update lib links

* update doc

* update code

* update lite xpu link

* update xpu lib

* update doc

* update en doc
 2022-12-15 21:17:14 +08:00
Jack Zhou
8c2d582925 [Backend]Add DisablePaddleTrtOPs (#788) 
* Add DisablePaddleTrtOPs

* Add delete_paddle_backend_pass disable_paddle_trt_ops pybind
 2022-12-05 10:03:52 +08:00
heliqi
5dca3a45c9 [backend][Serving]Fix paddle backend get outout tensor error (#741) 
fix paddle backend no_copy_infer
 2022-11-29 18:34:56 +08:00
Jason
03947d7b37 [Python] Fix bug while inference with non contiguous data (#713) 
Fix runtime infer
 2022-11-28 14:32:41 +08:00
Jason
5b3fd9dd88 [Backend] Support Intel GPU with heterogeneous mode (#701) 
* Add some comments for python api

* support openvino gpu

* Add cpu operators

* add interface to specify hetero operators

* remove useless dir

* format code

* remove debug code

* Support GPU for ONNX
 2022-11-25 19:40:56 +08:00
Jason
dbe96cd049 [Backend] Support Intel GPU with OpenVINO (#472) 
* Update ov_backend.cc

* Update ov_backend.cc

* support set openvino device
 2022-11-17 17:08:49 +08:00
Jason
f2e492c55b [Other] Add function alias (#571) 
Add function alias
 2022-11-12 14:53:39 +08:00
yunyaoXYY
4706a7c32a [Backend] Add SetTrtMaxBatchSize function for TensorRT. (#526) 
* Imporve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Add Initialize function to PP-OCR

* Add Initialize function to PP-OCR

* Add Initialize function to PP-OCR

* Make all the model links come from PaddleOCR

* Improve OCR readme

* Improve OCR readme

* Improve OCR readme

* Improve OCR readme

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add comments to create API docs

* Improve OCR comments

* Rename OCR and add comments

* Make sure previous python example works

* Make sure previous python example works

* Fix Rec model bug

* Fix Rec model bug

* Fix rec model bug

* Add SetTrtMaxBatchSize function for TensorRT

* Add SetTrtMaxBatchSize Pybind

* Add set_trt_max_batch_size python function

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-09 13:57:36 +08:00
Zheng_Bicheng
4ffcfbe726 [Backend] Add RKNPU2 backend support (#456) 
* 10-29/14:05
* 新增cmake
* 新增rknpu2 backend

* 10-29/14:43
* Runtime fd_type新增RKNPU代码

* 10-29/15:02
* 新增ppseg RKNPU2推理代码

* 10-29/15:46
* 新增ppseg RKNPU2 cpp example代码

* 10-29/15:51
* 新增README文档

* 10-29/15:51
* 按照要求修改部分注释以及变量名称

* 10-29/15:51
* 修复重命名之后，cc文件中的部分代码还用旧函数名的bug

* 10-29/22:32
* str(Device::NPU)将输出NPU而不是UNKOWN
* 修改runtime文件中的注释格式
* 新增Building Summary ENABLE_RKNPU2_BACKEND输出
* pybind新增支持rknpu2
* 新增python编译选项
* 新增PPSeg Python代码
* 新增以及更新各种文档

* 10-30/14:11
* 尝试修复编译cuda时产生的错误

* 10-30/19:27
* 修改CpuName和CoreMask层级
* 修改ppseg rknn推理层级
* 图片将移动到网络进行下载

* 10-30/19:39
* 更新文档

* 10-30/19:39
* 更新文档
* 更新ppseg rknpu2 example中的函数命名方式
* 更新ppseg rknpu2 example为一个cc文件
* 修复disable_normalize_and_permute部分的逻辑错误
* 移除rknpu2初始化时的无用参数

* 10-30/19:39
* 尝试重置python代码

* 10-30/10:16
* rknpu2_config.h文件不再包含rknn_api头文件防止出现导入错误的问题

* 10-31/14:31
* 修改pybind，支持最新的rknpu2 backends
* 再次支持ppseg python推理
* 移动cpuname 和 coremask的层级

* 10-31/15:35
* 尝试修复rknpu2导入错误

* 10-31/19:00
* 新增RKNPU2模型导出代码以及其对应的文档
* 更新大量文档错误

* 10-31/19:00
* 现在编译完fastdeploy仓库后无需重新设置RKNN2_TARGET_SOC

* 10-31/19:26
* 修改部分错误文档

* 10-31/19:26
* 修复错误删除的部分
* 修复各种错误文档
* 修复FastDeploy.cmake在设置RKNN2_TARGET_SOC错误时，提示错误的信息
* 修复rknpu2_backend.cc中存在的中文注释

* 10-31/20:45
* 删除无用的注释

* 10-31/20:45
* 按照要求修改Device::NPU为Device::RKNPU，硬件将共用valid_hardware_backends
* 删除无用注释以及debug代码

* 11-01/09:45
* 更新变量命名方式

* 11-01/10:16
* 修改部分文档，修改函数命名方式

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-01 11:14:05 +08:00
czr-gc
ede59af857 [Backend] support ipu in paddle inference backend. (#437) 
* feat(ipu): add ipu support for paddle_infer backend.

* fix(): remove unused env.

* fix(ipu): simplify user API for IPU.

* fix(cmake): fix merge conflict error in CMakeList.

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-30 18:59:59 +08:00
Wang Xinyu
43d86114d8 [Backend] TRT backend & PP-Infer backend support pinned memory (#403) 
* TRT backend use pinned memory

* refine fd tensor pinned memory logic

* TRT enable pinned memory configurable

* paddle inference support pinned memory

* pinned memory pybindings

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-21 18:51:36 +08:00
Jack Zhou
dccb737d8d [Backend] Add collect shape for pp-trt backend (#372) 
* Add collect_shape attr

* add EnableTunedTensorRtDynamicShape

* Add collect shape python api

* Fix quant model not set trt dynamic shape

* Add shape info print

* Fix shape print

* Use CopyFromCpu instead of ShareExternalData

* Add ENABLE_TRT_BACKEND macro

* Add shared data with
 2022-10-20 17:02:56 +08:00
WJJ1995
f5c94e5471 Support Poros Backend (#188) 
* Add poros backend

* Add torch lib

* Add python3 lib

* set c++ 14 for poros

* fixed bugs

* fixed grammer bugs

* fixed grammer bugs

* fixed code bugs

* fixed code bugs

* fixed CreatePorosValue bug

* Add AtType2String for Log

* fixed trt_option

* fixed poros.cmake path

* fixed grammer bug

* fixed grammer bug

* fixed ambiguous reference

* fixed ambiguous reference

* fixed reference error

* fixed include files

* rm ENABLE_TRT_BACKEND in poros

* update CMakeLists.txt

* fixed CMakeLists.txt

* Add libtorch.so in CMakeLists.txt

* Fixed CMakeLists.txt

* Fixed CMakeLists.txt

* Fixed copy bug

* Fixed copy bug

* Fixed copy bug

* Fixed Cmake

* Fixed Cmake

* debug

* debug

* debug

* debug

* debug

* debug

* debug utils

* debug utils

* copy to cpu

* rm log info

* test share mem

* test share mem

* test share mem

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* time cost

* time cost

* fixed bug

* time collect

* mem copy

* mem copy

* rm time log

* rm share mem

* fixed multi inputs bug

* add set_input_dtypes func

* add SetInputDtypes

* fixed bug

* fixed bug

* fixed prewarm data order

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* fixed bug

* Add compile func

* Add compile func

* Add compile func

* Add is_dynamic option

* Add is_dynamic option

* Add is_dynamic option

* Add is_dynamic option

* rm infer log

* add cuda11.6 poros lib

* fixed bug

* fixed bug

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* rm logs

* test

* test

* test

* add test log

* add test log

* add test log

* add test log

* support cpu

* support cpu

* support cpu

* support cpu

* support member variable definition

* rm useless log

* fixed name

* resolve conflict

* resolve conflict

* resolve conflict

* fixed cmake

* add GetInputInfos&GetOutputInfos

* add GetInputInfos&GetOutputInfos

* fixed bug

* fixed runtime.py

* add compile func

* add np

* deal with comments

* rm to_inter func

* add property
 2022-10-17 15:28:12 +08:00
Jason
945e197bd1 Fix bug of get input/output information from PaddleBackend (#339) 
* Fix bug of get input/output information from PaddleBackend

* Support Paddle Inference with TensorRT (#340)

* Fix bug
 2022-10-12 11:37:26 +08:00
yeliang2258
2a68a23baf Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend (#257) 
* add quantize model support for trt and paddle

* fix bugs

* fix

* update paddle2onnx version

* update version

* add quantize test

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-09 20:00:05 +08:00
Jason
5d4372955f Add some comments for python api (#327) 
* Add some comments for python api

* Update setup.py

* Update runtime.py
 2022-10-09 10:05:18 +08:00
heliqi
a3fa5989d2 Fix set_paddle_mkldnn python interface (#328) 
* fd serving add dockerfile

* fix enable_paddle_mkldnn

* delete disable_paddle_mkldnn

* fix python set_paddle_mkldnn

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-08 16:49:40 +08:00
heliqi
d57e997fa0 Fd serving add docker images correlation and docs (#311) 
* fd serving add dockerfile

* fix enable_paddle_mkldnn

* delete disable_paddle_mkldnn

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-08 16:08:07 +08:00
DefTruth
c4288a92e1 [lite] add more options for lite backend (#313) 
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk

* [cmake] fixed patchelf download on mac and android

* [lite] Add threads and power_mode option support

* [pybind] update runtime pybind for lite power mode

* [python] Add set_lite_power_mode api to runtime

* [Lite] add lite enable_fp16 option support

* [lite] add more options for lite backend.

* [cmake] fixed Paddle Lite typo

* [runtime] format LitePowerMode enum comments

* [runtime] format lite option comments
 2022-10-04 17:31:48 +08:00
Jack Zhou
dd365fb721 Add trt max workspace setting (#308) 
* add trt max workspace setting

* fix set trt max workspace
 2022-09-30 09:54:34 +08:00
DefTruth
c5f85de356 [lite] Add threads and power_mode option support (#298) 
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk

* [cmake] fixed patchelf download on mac and android

* [lite] Add threads and power_mode option support

* [pybind] update runtime pybind for lite power mode

* [python] Add set_lite_power_mode api to runtime
 2022-09-28 18:09:35 +08:00
Jason
2d3e61582d Rename pybind/fastdeploy_runtime.cc to pybind/runtime.cc (#273) 
* fix yaml

* Rename pybind/fastdeploy_runtime.cc to runtime.cc
 2022-09-23 11:16:02 +08:00
heliqi
a8e447f225 yolov5 servitization optimization (#262) 
* yolov5 split pre and post process

* yolov5 postprocess

* yolov5 postprocess
 2022-09-21 18:22:39 +08:00
Jason
addce837bc Add paddlelite backend support (#260) 
* Add paddlelite backend support

* Update CMakeLists.txt

* Update __init__.py
 2022-09-21 13:22:34 +08:00
Jason
68523be411 Modify file structure to separate python and cpp code (#223) 
Modify code structure
 2022-09-14 15:44:13 +08:00