apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-13 04:13:58 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
2,417 Commits 16 Branches 8 Tags
75b0d3498b51d0cec7bc8ab9bcbb9ccc19549eda
Commit Graph

92 Commits

Author SHA1 Message Date
DefTruth
cfebd24dfc [Backend] fix ort backend windows build error (#2269) 
* support ort offline graph optimize option

* support ort offline graph optimize option

* [Backend] fix windows build error
 2023-11-02 12:51:46 +08:00
DefTruth
6a8cd4d759 [Backend] support ort offline graph optimize option (#2268) 
* support ort offline graph optimize option

* support ort offline graph optimize option
 2023-11-02 09:19:33 +08:00
ChaoII
771dcf4b91 [Backend]Update sophgo_backend.cc for FP16 data type (#2161) 2023-08-16 14:12:58 +08:00
DefTruth
681ccc4c24 [Sync][Internal] sync some internal paddle3d codes (#2108) 2023-07-13 22:06:28 +08:00
DefTruth
d5ad3d9c8d [Bug Fix] fixed paddle custom ops windows build error (#2103) 
* [cmake] upgrade windows paddle inference -> 2.5.0

* [cmake] upgrade windows paddle inference -> 2.5.0

* fix paddle custom ops bug on windows

* [Backend] refactor paddle custom ops

* [Bug Fix] fixed paddle custom ops windows build error
 2023-07-13 14:04:01 +08:00
DefTruth
99c2b6592d [Backend] refactor paddle custom ops -> fastdeploy::paddle_custom_ops (#2101) 
* [cmake] upgrade windows paddle inference -> 2.5.0

* [cmake] upgrade windows paddle inference -> 2.5.0

* fix paddle custom ops bug on windows

* [Backend] refactor paddle custom ops
 2023-07-13 09:00:03 +08:00
DefTruth
2542a75b61 [cmake] upgrade windows paddle inference -> 2.5.0 (#2100) 
* [cmake] upgrade windows paddle inference -> 2.5.0

* [cmake] upgrade windows paddle inference -> 2.5.0

* fix paddle custom ops bug on windows
 2023-07-12 18:39:06 +08:00
DefTruth
cf1ff2077d [Bug Fix] fix trt backend page-locked error (#2095) 
* [Bug Fix] fix trt backend page-locked error

* Update trt_backend.cc
 2023-07-11 13:49:47 +08:00
yeliang2258
ad1f46f7d9 Add ORT fp16 support in server (#2069) 
* add ort fp16 support in server

* update paddle2onnx url

* update ort fp16 api

* add disable_ort_fp16_op_types in serving
 2023-07-05 17:50:00 +08:00
DefTruth
b2426aefa9 [Backend] add paddle custom ops compatible policy (#2070) 
* Add centerpoint

* fix postprocess op file name

* [Backend] add paddle custom ops compatible policy

* [Backend] add paddle custom ops compatible policy

* [Backend] add paddle custom ops compatible policy

* upgrade linx paddle gpu -> 2.5

* add custom op compatible policy for paddle 2.5

* add custom op compatible policy for paddle 2.5

* add custom op compatible policy for paddle 2.5

* add collect_trt_shape_by_device option for paddle backend

* add collect_trt_shape_by_device option for paddle backend

* add custom op option for python build

* fix python build bugs

* update paddle linux x86 cpu only lib

* update paddle linux gpu lib

* update patchelf cmake

* fix paddle backend option pybind

* update paddle_inference.cmake

* add cuda sm_80 support (A100)

---------

Co-authored-by: zengshao0622 <peter_z96@163.com>
Co-authored-by: qiuyanjun <qiuyanjun@baidu.com>
 2023-06-29 22:32:14 +08:00
JugendTraum
4c3e7030e1 Update rknpu2_backend.cc (#2064) 
Update rknpu2_backend.cc  Signed-off-by: JugendTraum <443248173@qq.com>

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-06-26 04:58:12 -07:00
DefTruth
dea3795930 [Backend] Add backward compatible for paddle inference 2.4.x (#2062) 
* [Backend] Add backward compatiable for paddle infernence 2.4.x

* [Backend] Add backward compatiable for paddle infernence 2.4.x
 2023-06-25 19:27:36 +08:00
DefTruth
f5bea8bf37 [Bug Fix] fixed paddlex ppseg pp-trt infer error (#2049) 
* [benchmark] fixed paddlex benchmark for picodet 320

* [Bug Fix] fixed paddlex ppseg pp-trt infer error
 2023-06-20 23:24:31 +08:00
Horror Proton
7191d2da20 [Backend] Remove deprecated ort api (#2034) 
* [Backend] Remove deprecated ort api

`Ort::CustomOpApi` deprecated in ONNXRuntime 1.15

fix https://github.com/PaddlePaddle/FastDeploy/issues/2033

* Improve compatibly with ort older than 1.14

* Update fastdeploy/runtime/backends/ort/ops/multiclass_nms.cc

Co-authored-by: Horror Proton <107091537+horror-proton@users.noreply.github.com>

* Fix double free and wrong attrib type in ort/ops

---------

Co-authored-by: Jason <928090362@qq.com>
 2023-06-15 21:35:58 +08:00
Wang Bojun
7b58387d3c add option for inference precision and cutlass (#2017) 
* add option for inference precision and cutlass

* code clean

* code refine

* code refine
 2023-06-09 10:20:48 +08:00
Zheng-Bicheng
8d357814e8 [Backend] Add pybind & PaddleDetection example for TVM (#1998) 
* update

* update

* Update infer_ppyoloe_demo.cc

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-06-04 13:26:47 +08:00
DefTruth
284b1b46c8 [Serving] Support XPU encrypt & auth server (#2007) 
* [patchelf] fix patchelf error for inference xpu

* [serving] add xpu dockerfile and support fd server

* [serving] add xpu dockerfile and support fd server

* [Serving] support XPU + Tritron

* [Serving] support XPU + Tritron

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] add comments for xpu tritron dockerfile

* [Doruntime] fix xpu infer error

* [Doruntime] fix xpu infer error

* [XPU] update xpu dockerfile

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* [XPU] Update XPU L3 Cache setting docs

* [XPU] Add Encryption and AUTH support for XPU Server

* [XPU] Add Encryption and AUTH support for XPU Server

* [Bug Fix] fix paddle reader error

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Triton] switch TAG 22.12 -> TAG 21.10wq

* update xpu auth server script
 2023-06-01 21:36:44 +08:00
DefTruth
387c5695b3 [XPU] Update XPU L3 Cache setting docs (#2001) 
* [patchelf] fix patchelf error for inference xpu

* [serving] add xpu dockerfile and support fd server

* [serving] add xpu dockerfile and support fd server

* [Serving] support XPU + Tritron

* [Serving] support XPU + Tritron

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] add comments for xpu tritron dockerfile

* [Doruntime] fix xpu infer error

* [Doruntime] fix xpu infer error

* [XPU] update xpu dockerfile

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* [XPU] Update XPU L3 Cache setting docs
 2023-05-30 11:21:04 +08:00
Zheng-Bicheng
643730bf5f [Hackathon 181] Add TVM support for FastDeploy on macOS (#1969) 
* update for tvm backend

* update third_party

* update third_party

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-25 19:59:02 +08:00
DefTruth
49c033a828 [XPU] Support XPU via Paddle Inference backend (#1987) 
* [backend] Support XPU via Paddle Inference backend

* [backend] Support XPU via Paddle Inference backend

* [backend] Support XPU via Paddle Inference backend

* [XPU] support XPU benchmark via paddle inference

* [XPU] support XPU benchmark via paddle inference

* [benchmark] add xpu paddle h2d config files
 2023-05-25 14:13:40 +08:00
heliqi
3e7cb88049 [Serving]support 22.12 (#1974) 
support 22.12
 2023-05-22 22:27:13 +08:00
linkk08
c8ff8b63e8 [XPU] Add docs for gm_default_size and fix type (#1962) 
Add description of gm_default_size and change kunlunxin_gm_default_size‘s type to int64_t
 2023-05-18 20:30:06 +08:00
DefTruth
1f66d5d9d3 [benchmark] support lite light api & optimize benchmark flags (#1950) 
* [benchmark] support lite light api & optimize benchmark flags

* [backend] remove un-used option member

* [backend] remove un-used option member
 2023-05-17 18:51:32 +08:00
linkk08
9ed8d18fcc [XPU] Add gm_default_size -> Backend::LITE (#1934) 
* add gm_default_size

* add gm_default_size

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-16 14:31:22 +08:00
unseenme
3fd21c935e [Hackthon_4th 244] Added Paddle Lite GPU Backend (#1907) 
* [improved] enum; ConfigureGpu();

* [improved] init()

* [improved] valid place; model dir; is valid;

* [added] WITH_OPENCL in cmake

* [improved] set model; valid place; cmake url; cmake option;

* Update runtime_option.cc

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-12 19:44:27 +08:00
seyosum
df8dd3e3ac 【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy (#1822) 
* add horizon backend and PPYOLOE examples

* 更改horizon头文件编码规范

* 更改horizon头文件编码规范

* 更改horizon头文件编码规范

* 增加horizon packages下载及自动安装

* Add UseHorizonNPUBackend Method

* 删除编译FD SDK后多余的头文件,同时更改部分规范

* Update horizon.md

* Update horizon.md

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-06 16:10:37 +08:00
DefTruth
652024d2f6 Revert "Remove Paddle Reader" (#1860) 
Revert "Remove Paddle Reader (#1813)"

This reverts commit f3d44785c4.
 2023-04-23 23:16:31 +08:00
thunder95
51be3fea78 [Hackthon_4th 177] Support PP-YOLOE-R with BM1684 (#1809) 
* first draft

* add robx iou

* add benchmark for ppyoloe_r

* remove trash code

* fix bugs

* add pybind nms rotated option

* add missing head file

* fix bug

* fix bug2

* fix shape bug

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-21 10:48:05 +08:00
Jason
f3d44785c4 Remove Paddle Reader (#1813) 
* Remove Paddle Reader

* support pp-infer c++14

* disable trt cache

---------

Co-authored-by: wang-xinyu <wangxinyu_es@163.com>
 2023-04-20 21:12:43 +08:00
DefTruth
326715569d [Bug Fix] fixed paddle inference headers include policy (#1827) 
* Update __init__.py

* [Python] Update python whl setup.py

* [cmake] support fd & paddle inference custom libs on win/linux

* [cmake] support fd & paddle inference custom libs on win/linux

* [Bug Fix] fixed paddle inference headers include policy

* [Bug Fix] fixed paddle inference headers include policy

* [Bug Fix] fixed paddle inference headers include policy
 2023-04-19 14:32:51 +08:00
yeliang2258
a509dd8ec1 [Model] Add Paddle3D smoke model (#1766) 
* add smoke model

* add 3d vis

* update code

* update doc

* mv paddle3d from detection to perception

* update result for velocity

* update code for CI

* add set input data for TRT backend

* add serving support for smoke model

* update code

* update code

* update code

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-14 16:30:56 +08:00
Zheng-Bicheng
b30f62af36 [Backend] Refactoring RKNPU2 Backend code (#1772) 
* update rknpu2 runtime

* update rknpu2 runtime

* update rknpu2 runtime

* update for rknpu2 backend

* update for rknpu2 backend

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-13 16:37:36 +08:00
Yi-sir
9e20dab0d6 [Example] Merge Download Paddle Model, Paddle->ONNX->MLIR->BModel (#1643) 
* fix infer.py and README

* [Example] Merge Download Paddle Model, Paddle->Onnx->Mlir->Bmodel and
inference into infer.py. Modify README.md

* modify pp_liteseg sophgo infer.py and README.md

* fix PPOCR,PPYOLOE,PICODET,LITESEG sophgo infer.py and README.md

* fix memory overflow problem while inferring with sophgo backend

* fix memory overflow problem while inferring with sophgo backend

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
Co-authored-by: xuyizhou <yizhou.xu@sophgo.com>
 2023-03-31 15:08:01 +08:00
DefTruth
7d1b706caf [Benchmark] Add pure runtime benchmark bin (#1731) 
* [Benchmark] Add pure runtime benchmark bin

* [Benchmark] add cpu/gpu memory collect -> benchmark bin

* [Backend] update trt max_batch_size policy

* [backend] Update trt backend max_batch_size policy

* [Benchmark] Add more model format support -> benchmark bin
 2023-03-30 19:03:30 +08:00
WJJ1995
409c5dc056 [Backend]Add default value for TRT SetShape() (#1712) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

* add macros

* deal with comments

* deal with comments

* deal with comments

* re-lint

* rm pmap and use mem api

* rm pmap and use mem api

* add mem api

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* deal with comments

* fixed enable_paddle_to_trt

* add log for paddle_trt

* support ppcls benchmark

* use new trt option api

* update benchmark info

* simplify benchmark.cc

* simplify benchmark.cc

* deal with comments

* Add ppseg && ppocr benchmark

* add OCR rec img

* add ocr benchmark

* fixed trt shape

* add trt shape

* resolve conflict

* add ENABLE_BENCHMARK define

* Add ClassifyDiff

* Add Resize for ClassifyResult

* deal with comments

* add convert info script

* resolve conflict

* Add SaveBenchmarkResult func

* fixed bug

* fixed bug

* fixed bug

* add config.txt for option

* fixed bug

* fixed bug

* fixed bug

* add benchmark.sh

* mv thread_nums from 8 to 1

* deal with comments

* deal with comments

* fixed readme

* deal with comments

* add all platform shell

* Update config.arm.txt

* Update config.gpu.txt

* Update config.x86.txt

* fixed printinfo bug

* rm proxy

* add more model support

* all backend config.txt

* deal with comments

* Add MattingDiff compare

* fixed predict bug

* adjust warmup/repeat times

* add e2e/mem configs

* fixed typo

* open collect_mem

* fixed typo

* add trt cache option

* fixed bug

* fixed repeat times

* test for benchmark

* test for det benchmark

* for benchmark

* fixed for x86

* add h2d and d2h config

* renmae txt file

* add dynamic shape for pp_trt

* fixed typo

* Update option.h

* add collect shape

* add default value for SetShape()

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-03-27 14:03:47 +08:00
WJJ1995
5c70db176f [Backend]Add switch_ir_debug for Paddle Backend (#1700) 
* avoid mem copy for cpp benchmark

* set CMAKE_BUILD_TYPE to Release

* Add SegmentationDiff

* change pointer to reference

* fixed bug

* cast uint8 to int32

* Add diff compare for OCR

* Add diff compare for OCR

* rm ppocr pipeline

* Add yolov5 diff compare

* Add yolov5 diff compare

* deal with comments

* deal with comments

* fixed bug

* fixed bug

* fixed thread nums

* Add Failed log

* optimizer x86 pipeline

* Add switch_ir_debug for paddle backend

* fixed for ci
 2023-03-24 17:29:31 +08:00
yeliang2258
cae341e6c5 [Backend] Add fp16 support for ONNXRuntime-GPU (#1239) 
* add fp16 support for ort-gpu

* add enable_ort_fp16 option

* fix

* fix bugs

* use fp16 for test

* update code

* update code

* fix windows bug
 2023-03-23 18:23:13 +08:00
yunyaoXYY
b3e16e9966 ORT support UINT8 and INT8 input and output 2023-03-21 02:23:23 +00:00
Jack Zhou
012c7771c1 [Serving] Add collect shape and fix serving infer (#1658) 
Add collect shape and fix serving infer
 2023-03-20 19:55:30 +08:00
DefTruth
6af3691568 [XPU] Update Lite XPU lib and fixed some bugs (#1647) 2023-03-17 21:39:39 +08:00
Jason
3b1343c726 [Bug] Fix loadding big model loadding problem (#1636) 
Fix loadding big model loadding problem
 2023-03-17 10:25:26 +08:00
Ethan Yang
4f48d84c27 [Backend] optimize cpu usage on openvino backend (#1531) 
* optimiza cpu usage on openvino backend

conver the hint varible to string

* update with affinity mode set-up

---------

Co-authored-by: Ethan <yicheng@intel.com>
Co-authored-by: Jason <jiangjiajun@baidu.com>
 2023-03-17 10:24:25 +08:00
Jason
3d36e26b8a [Backend] Remove some deprecated api (#1574) 
Remove some deprecated api
 2023-03-13 13:42:43 +08:00
Jack Zhou
f4736e7931 Merge pull request #1552 from joey12300/fix_delete_pass 
[Backend] Fix delete pass of paddle inference
 2023-03-08 19:57:08 +08:00
Jason
6be2c0367b [Example] Update runtime examples (#1542) 
* Add notes for tensors

* Optimize some apis

* move some warnings
 2023-03-08 16:56:04 +08:00
zhoushunjie
384eca14fd Fix delete pass 2023-03-08 06:32:27 +00:00
Zheng-Bicheng
0687d3b0ad [Model] Add Solov2 For PaddleDetection (#1435) 
* update solov2

* Repair note

* update solov2 postprocess

* update

* update solov2

* update solov2

* fixed bug

* fixed bug

* update solov2

* update solov2

* fix build android bug

* update docs

* update docs

* update docs

* update

* update

* update arch and docs

* update

* update

* update solov2 python

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-03-08 10:01:32 +08:00
Zheng-Bicheng
458523fc29 update pybind 2023-03-07 15:50:49 +08:00
Zheng-Bicheng
610b4c532b [Bug Fix] fixed rknn bugs (#1468) 
fixed rknn bugs
 2023-03-01 10:21:58 +08:00
Jack Zhou
524c85745b [Backend] Add fixed size optimization for transformer model (#1430) 
Add enable_fixed_size_opt flag
 2023-02-24 09:45:04 +08:00