apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-11-01 04:12:58 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
1,308 Commits 19 Branches 8 Tags
aa9f107e70b7e7bbcaedb7da21d557957e4abf4d
Commit Graph

55 Commits

Author SHA1 Message Date
heliqi
42f1888bb0 [Serving][Backend] Backend support zero_copy_infer and Serving reduce the output memory copy (#703) 
* backend add zero copy infer interface

* fix bug

* fix bug

* fix bug

* paddle ipu
 2022-11-28 14:07:53 +08:00
Jason
5b3fd9dd88 [Backend] Support Intel GPU with heterogeneous mode (#701) 
* Add some comments for python api

* support openvino gpu

* Add cpu operators

* add interface to specify hetero operators

* remove useless dir

* format code

* remove debug code

* Support GPU for ONNX
 2022-11-25 19:40:56 +08:00
yeliang2258
d14828cb18 [Backend] Add AdaptivePool2d for TensorRT plugin (#668) 
* add adaptivepool2d for tensorrt plugin

* update code

* update code

* update code to fix bug
 2022-11-25 17:36:59 +08:00
yeliang2258
7206094ddd [Backend] Add ORT CPU and CUDA kernel for adaptive_pool OP (#609) 
* add ORT CPU and CUDA kernel for adaptive_pool

* update code

* update paddle2onnx version
 2022-11-24 10:39:29 +08:00
Jason
dbe96cd049 [Backend] Support Intel GPU with OpenVINO (#472) 
* Update ov_backend.cc

* Update ov_backend.cc

* support set openvino device
 2022-11-17 17:08:49 +08:00
yeliang2258
fa807340be [Backend] Add Rockchip RV1126 deploy support through PaddleLite TIM-VX (#439) 
* add rk1126 support

* update lib

* fix compile bugs

* update doc

* fix complie bug

* update doc

* update doc

* update code

* support model bigger than 2G

* update code

* update code

* update code

* update doc

* update code

* fix bug

* update code

* update code

* update code

* update doc

* update info

* code style check

* update code

* update doc

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-10 10:05:47 +08:00
Jason
f2fed7959b [Other] Add namespace for functions (#538) 
Add namespace for functions
 2022-11-09 13:57:53 +08:00
yunyaoXYY
4706a7c32a [Backend] Add SetTrtMaxBatchSize function for TensorRT. (#526) 
* Imporve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Improve OCR Readme

* Add Initialize function to PP-OCR

* Add Initialize function to PP-OCR

* Add Initialize function to PP-OCR

* Make all the model links come from PaddleOCR

* Improve OCR readme

* Improve OCR readme

* Improve OCR readme

* Improve OCR readme

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add Readme for vision results

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add check for label file in postprocess of Rec model

* Add comments to create API docs

* Improve OCR comments

* Rename OCR and add comments

* Make sure previous python example works

* Make sure previous python example works

* Fix Rec model bug

* Fix Rec model bug

* Fix rec model bug

* Add SetTrtMaxBatchSize function for TensorRT

* Add SetTrtMaxBatchSize Pybind

* Add set_trt_max_batch_size python function

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-09 13:57:36 +08:00
yeliang2258
8d31fe59e8 [Backend] Support the deployment of models which is larger than 2G in ORT and TRT backend (#514) 
* support model bigger than 2G

* update code

* update code

* fix bug

* update code
 2022-11-09 11:14:15 +08:00
heliqi
277bec38c7 [Backend & Serving] Serving and Runtime support Clone (#464) 
* Add Serving and Runtime use Clone

* support TRT, OpenVINO and Paddle Backend

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-04 17:16:40 +08:00
Jason
9fa612c24b [Backend] Enable TensorRT BatchedNMSDynamic_TRT plugin (#449) 
* Enable TensorRT EfficientNMS plugin

* remove some temporary code

* Update trt_backend.cc

* Update utils.h
 2022-11-04 11:46:29 +08:00
Zheng_Bicheng
ce828ecb38 [Backend And DOC] 改进ppseg文档 + 为RKNPU2后端新增对多输入模型的支持 (#491) 
* 11-02/14:35
* 新增输入数据format错误判断
* 优化推理过程，减少内存分配次数
* 支持多输入rknn模型
* rknn模型输出shape为三维时，输出将被强制对齐为4纬。现在将直接抹除rknn补充的shape，方便部分对输出shape进行判断的模型进行正确的后处理。

* 11-03/17:25
* 支持导出多输入RKNN模型
* 更新各种文档
* ppseg改用Fastdeploy中的模型进行转换

* 11-03/17:25
* 新增开源头

* 11-03/21:48
* 删除无用debug代码，补充注释
 2022-11-04 09:39:23 +08:00
DefTruth
d8aa11efc8 [Backend] fix lite backend save model error (#474) 
* [Backend] fix lite backend save model error

* [Backend] fixed typos
 2022-11-02 10:01:02 +08:00
Zheng_Bicheng
bc089f47be [Bug Fix] fix pre-commit error (#471) 
11-01/10:16
* 修复commit错误
 2022-11-01 16:29:31 +08:00
Zheng_Bicheng
4ffcfbe726 [Backend] Add RKNPU2 backend support (#456) 
* 10-29/14:05
* 新增cmake
* 新增rknpu2 backend

* 10-29/14:43
* Runtime fd_type新增RKNPU代码

* 10-29/15:02
* 新增ppseg RKNPU2推理代码

* 10-29/15:46
* 新增ppseg RKNPU2 cpp example代码

* 10-29/15:51
* 新增README文档

* 10-29/15:51
* 按照要求修改部分注释以及变量名称

* 10-29/15:51
* 修复重命名之后，cc文件中的部分代码还用旧函数名的bug

* 10-29/22:32
* str(Device::NPU)将输出NPU而不是UNKOWN
* 修改runtime文件中的注释格式
* 新增Building Summary ENABLE_RKNPU2_BACKEND输出
* pybind新增支持rknpu2
* 新增python编译选项
* 新增PPSeg Python代码
* 新增以及更新各种文档

* 10-30/14:11
* 尝试修复编译cuda时产生的错误

* 10-30/19:27
* 修改CpuName和CoreMask层级
* 修改ppseg rknn推理层级
* 图片将移动到网络进行下载

* 10-30/19:39
* 更新文档

* 10-30/19:39
* 更新文档
* 更新ppseg rknpu2 example中的函数命名方式
* 更新ppseg rknpu2 example为一个cc文件
* 修复disable_normalize_and_permute部分的逻辑错误
* 移除rknpu2初始化时的无用参数

* 10-30/19:39
* 尝试重置python代码

* 10-30/10:16
* rknpu2_config.h文件不再包含rknn_api头文件防止出现导入错误的问题

* 10-31/14:31
* 修改pybind，支持最新的rknpu2 backends
* 再次支持ppseg python推理
* 移动cpuname 和 coremask的层级

* 10-31/15:35
* 尝试修复rknpu2导入错误

* 10-31/19:00
* 新增RKNPU2模型导出代码以及其对应的文档
* 更新大量文档错误

* 10-31/19:00
* 现在编译完fastdeploy仓库后无需重新设置RKNN2_TARGET_SOC

* 10-31/19:26
* 修改部分错误文档

* 10-31/19:26
* 修复错误删除的部分
* 修复各种错误文档
* 修复FastDeploy.cmake在设置RKNN2_TARGET_SOC错误时，提示错误的信息
* 修复rknpu2_backend.cc中存在的中文注释

* 10-31/20:45
* 删除无用的注释

* 10-31/20:45
* 按照要求修改Device::NPU为Device::RKNPU，硬件将共用valid_hardware_backends
* 删除无用注释以及debug代码

* 11-01/09:45
* 更新变量命名方式

* 11-01/10:16
* 修改部分文档，修改函数命名方式

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-11-01 11:14:05 +08:00
czr-gc
ede59af857 [Backend] support ipu in paddle inference backend. (#437) 
* feat(ipu): add ipu support for paddle_infer backend.

* fix(): remove unused env.

* fix(ipu): simplify user API for IPU.

* fix(cmake): fix merge conflict error in CMakeList.

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-30 18:59:59 +08:00
Wang Xinyu
caa369f64a [Backend] TRT cast GPU input from int64 to int32, output from int32 to int64, and Windows support building CUDA files (#426) 
* TRT cast int64 to int32

* windows cmake build cuda src

* fix windows cmake error when build cuda src

* add a notice in windows gpu build doc

* cmake add cuda std=11

* TRT cast output from int32 to int64

* nits

* trt get original input output dtype
 2022-10-28 13:38:06 +08:00
DefTruth
a51e5a6e55 [Android] Add android aar package (#416) 
* [Android] Add Android build docs and demo (#26)

* [Backend] Add override flag to lite backend

* [Docs] Add Android C++ SDK build docs

* [Doc] fix android_build_docs typos

* Update CMakeLists.txt

* Update android.md

* [Doc] Add PicoDet Android demo docs

* [Doc] Update PicoDet Andorid demo docs

* [Doc] Update PaddleClasModel Android demo docs

* [Doc] Update fastdeploy android jni docs

* [Doc] Update fastdeploy android jni usage docs

* [Android] init fastdeploy android jar package

* [Backend] support int8 option for lite backend

* [Model] add Backend::Lite to paddle model

* [Backend] use CopyFromCpu for lite backend.

* [Android] package jni srcs and java api into aar

* Update infer.cc

* Update infer.cc

* [Android] Update package build.gradle

* [Android] Update android app examples

* [Android] update android detection app
 2022-10-26 17:01:14 +08:00
heliqi
b064ddf7ed [Serving][backend]serving support multi stream and backend support external stream (#431) 
* serving support multi stream

* pybind add external stream

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-26 14:46:13 +08:00
WJJ1995
718698a32a [Model] add RobustVideoMatting model (#400) 
* add yolov5cls

* fixed bugs

* fixed bugs

* fixed preprocess bug

* add yolov5cls readme

* deal with comments

* Add YOLOv5Cls Note

* add yolov5cls test

* add rvm support

* support rvm model

* add rvm demo

* fixed bugs

* add rvm readme

* add TRT support

* add trt support

* add rvm test

* add EXPORT.md

* rename export.md

* rm poros doxyen

* deal with comments

* deal with comments

* add rvm video_mode note

Co-authored-by: Jason <jiangjiajun@baidu.com>
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2022-10-26 14:30:04 +08:00
Jack Zhou
8af35e5114 [Backend] Enable Memory Optim for paddle backend (#417) 
Enable Memory Optim for paddle backend

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-24 19:30:24 +08:00
Wang Xinyu
43d86114d8 [Backend] TRT backend & PP-Infer backend support pinned memory (#403) 
* TRT backend use pinned memory

* refine fd tensor pinned memory logic

* TRT enable pinned memory configurable

* paddle inference support pinned memory

* pinned memory pybindings

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-21 18:51:36 +08:00
huangjianhui
b565c15bf7 [Model] Add tinypose single && pipeline model (#177) 
* Add tinypose model

* Add PPTinypose python API

* Fix picodet preprocess bug && Add Tinypose examples

* Update tinypose example code

* Update ppseg preprocess if condition

* Update ppseg backend support type

* Update permute.h

* Update README.md

* Update code with comments

* Move files dir

* Delete premute.cc

* Add single model pptinypose

* Delete pptinypose old code in ppdet

* Code format

* Add ppdet + pptinypose pipeline model

* Fix bug for posedetpipeline

* Change Frontend to ModelFormat

* Change Frontend to ModelFormat in __init__.py

* Add python posedetpipeline/

* Update pptinypose example dir name

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Create keypointdetection_result.md

* Create README.md

* Create README.md

* Create README.md

* Update README.md

* Update README.md

* Create README.md

* Fix det_keypoint_unite_infer.py bug

* Create README.md

* Update PP-Tinypose by comment

* Update by comment

* Add pipeline directory

* Add pptinypose dir

* Update pptinypose to align accuracy

* Addd warpAffine processor

* Update GetCpuMat to  GetOpenCVMat

* Add comment for pptinypose && pipline

* Update docs/main_page.md

* Add README.md for pptinypose

* Add README for det_keypoint_unite

* Remove ENABLE_PIPELINE option

* Remove ENABLE_PIPELINE option

* Change pptinypose default backend

* PP-TinyPose Pipeline support multi PP-Detection models

* Update pp-tinypose comment

* Update by comments

* Add single test example

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-21 09:28:23 +08:00
Jack Zhou
dccb737d8d [Backend] Add collect shape for pp-trt backend (#372) 
* Add collect_shape attr

* add EnableTunedTensorRtDynamicShape

* Add collect shape python api

* Fix quant model not set trt dynamic shape

* Add shape info print

* Fix shape print

* Use CopyFromCpu instead of ShareExternalData

* Add ENABLE_TRT_BACKEND macro

* Add shared data with
 2022-10-20 17:02:56 +08:00
DefTruth
c28f4d6019 [Backend] Add override flag to lite backend (#402) 2022-10-20 17:01:56 +08:00
WJJ1995
f5c94e5471 Support Poros Backend (#188) 
* Add poros backend

* Add torch lib

* Add python3 lib

* set c++ 14 for poros

* fixed bugs

* fixed grammer bugs

* fixed grammer bugs

* fixed code bugs

* fixed code bugs

* fixed CreatePorosValue bug

* Add AtType2String for Log

* fixed trt_option

* fixed poros.cmake path

* fixed grammer bug

* fixed grammer bug

* fixed ambiguous reference

* fixed ambiguous reference

* fixed reference error

* fixed include files

* rm ENABLE_TRT_BACKEND in poros

* update CMakeLists.txt

* fixed CMakeLists.txt

* Add libtorch.so in CMakeLists.txt

* Fixed CMakeLists.txt

* Fixed CMakeLists.txt

* Fixed copy bug

* Fixed copy bug

* Fixed copy bug

* Fixed Cmake

* Fixed Cmake

* debug

* debug

* debug

* debug

* debug

* debug

* debug utils

* debug utils

* copy to cpu

* rm log info

* test share mem

* test share mem

* test share mem

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* test multi outputs

* time cost

* time cost

* fixed bug

* time collect

* mem copy

* mem copy

* rm time log

* rm share mem

* fixed multi inputs bug

* add set_input_dtypes func

* add SetInputDtypes

* fixed bug

* fixed bug

* fixed prewarm data order

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* fixed bug

* Add compile func

* Add compile func

* Add compile func

* Add is_dynamic option

* Add is_dynamic option

* Add is_dynamic option

* Add is_dynamic option

* rm infer log

* add cuda11.6 poros lib

* fixed bug

* fixed bug

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* fixed multi outputs

* rm logs

* test

* test

* test

* add test log

* add test log

* add test log

* add test log

* support cpu

* support cpu

* support cpu

* support cpu

* support member variable definition

* rm useless log

* fixed name

* resolve conflict

* resolve conflict

* resolve conflict

* fixed cmake

* add GetInputInfos&GetOutputInfos

* add GetInputInfos&GetOutputInfos

* fixed bug

* fixed runtime.py

* add compile func

* add np

* deal with comments

* rm to_inter func

* add property
 2022-10-17 15:28:12 +08:00
Jason
24c8fdc27d Integrate FalconCV to boost image processing (#332) 
* Rename GetCpuMat to GetOpenCVMat

* refine code structure
 2022-10-16 14:49:57 +08:00
jiangjiajun
7d9c47753f Fix dead link in notice 2022-10-16 06:24:47 +00:00
heliqi
831ef42207 fix paddle-mkldnn bug (#366) 
* fix input name bug

* fix paddle-mkldnn bug
 2022-10-13 21:37:31 +08:00
Jason
945e197bd1 Fix bug of get input/output information from PaddleBackend (#339) 
* Fix bug of get input/output information from PaddleBackend

* Support Paddle Inference with TensorRT (#340)

* Fix bug
 2022-10-12 11:37:26 +08:00
Jason
548777a283 Add document describes some tricks while using TensorRT backend (#343) 
* add tensorrt tricks doc

* add quick start
 2022-10-11 21:19:55 +08:00
yeliang2258
2a68a23baf Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend (#257) 
* add quantize model support for trt and paddle

* fix bugs

* fix

* update paddle2onnx version

* update version

* add quantize test

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-09 20:00:05 +08:00
ChaoII
116265d239 add override mark (#312) 
Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-10-07 11:38:31 +08:00
Jason
a8e08c9c1a Merge branch 'develop' into add_float16_dtype 2022-10-05 10:47:52 +08:00
DefTruth
c4288a92e1 [lite] add more options for lite backend (#313) 
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk

* [cmake] fixed patchelf download on mac and android

* [lite] Add threads and power_mode option support

* [pybind] update runtime pybind for lite power mode

* [python] Add set_lite_power_mode api to runtime

* [Lite] add lite enable_fp16 option support

* [lite] add more options for lite backend.

* [cmake] fixed Paddle Lite typo

* [runtime] format LitePowerMode enum comments

* [runtime] format lite option comments
 2022-10-04 17:31:48 +08:00
zhoushunjie
faf04b49a6 Add float16 tests 2022-10-02 09:21:00 +00:00
zhoushunjie
ec8b183d9c remote HOSTDEVICE modifier 2022-10-02 04:28:30 +00:00
Jason
8a1e2c6143 Add some comments for runtime (#309) 
* Add some notes for runtime

* modify note

* modify note

* add some comment

* Add comment for runtime

* Add comment for runtime

* Add comment for runtime

* Add comment for runtime
 2022-09-30 14:43:44 +08:00
Jack Zhou
dd365fb721 Add trt max workspace setting (#308) 
* add trt max workspace setting

* fix set trt max workspace
 2022-09-30 09:54:34 +08:00
DefTruth
c5f85de356 [lite] Add threads and power_mode option support (#298) 
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk

* [cmake] fixed patchelf download on mac and android

* [lite] Add threads and power_mode option support

* [pybind] update runtime pybind for lite power mode

* [python] Add set_lite_power_mode api to runtime
 2022-09-28 18:09:35 +08:00
Jason
360bee46b5 Support build on jetson (#276) 
* support jetson build

* Update allocate.cc

* update setup

* Update CMakeLists.txt

* Update setup.py
 2022-09-23 13:11:11 +08:00
DefTruth
5edd9fccd3 [TRT] fix tensorrt serialize file problem (#274) 
fix tensorrt serialize file problem
 2022-09-23 00:40:17 +08:00
Jason
addce837bc Add paddlelite backend support (#260) 
* Add paddlelite backend support

* Update CMakeLists.txt

* Update __init__.py
 2022-09-21 13:22:34 +08:00
Jason
5046574b95 Fix bug for part of detection model (#243) 
* fix error for part of detection model

* fix error for part of detection model

* add patch paddle inference
 2022-09-16 16:31:34 +08:00
heliqi
0805ead0ed add 'GetOutputInfos' and 'GetInputInfos' interface (#232) 
add GetOutputInfos GetInputInfos
 2022-09-15 13:09:31 +08:00
Jason
68523be411 Modify file structure to separate python and cpp code (#223) 
Modify code structure
 2022-09-14 15:44:13 +08:00
Jason
ffbc5cc42d Move cpp code to directory csrcs (#42) 
* move cpp code to csrcs

* move cpp code to csrcs
 2022-07-26 17:59:02 +08:00
Jason
17e4dc6b5e Support remove multiclass_nms to enable ppyoloe to tensorrt (#40) 
* Add custom operator for onnxruntime ans fix paddle backend

* Polish cmake files and runtime apis

* Remove copy libraries

* fix some issue

* fix bug

* fix bug

* Support remove multiclass_nms to enable paddledetection run tensorrt

* Support remove multiclass_nms to enable paddledetection run tensorrt

* Support remove multiclass_nms to enable paddledetection run tensorrt

* Support remove multiclass_nms to enable paddledetection run tensorrt

* add common operator multiclassnms

* fix compile problem

Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com>
 2022-07-26 11:16:01 +08:00
Jason
279c993483 Polish cmake files and runtime apis (#36) 
* Add custom operator for onnxruntime ans fix paddle backend

* Polish cmake files and runtime apis

* Remove copy libraries

* fix some issue

* fix bug

* fix bug
 2022-07-25 08:59:53 +08:00
Jason
4b681581b1 Add custom operator for onnxruntime and fix paddle backend (#35) 
Add custom operator for onnxruntime ans fix paddle backend
 2022-07-23 22:21:36 +08:00