Jack Zhou
8c2d582925
[Backend]Add DisablePaddleTrtOPs ( #788 )
...
* Add DisablePaddleTrtOPs
* Add delete_paddle_backend_pass disable_paddle_trt_ops pybind
2022-12-05 10:03:52 +08:00
heliqi
42f1888bb0
[Serving][Backend] Backend support zero_copy_infer and Serving reduce the output memory copy ( #703 )
...
* backend add zero copy infer interface
* fix bug
* fix bug
* fix bug
* paddle ipu
2022-11-28 14:07:53 +08:00
heliqi
277bec38c7
[Backend & Serving] Serving and Runtime support Clone ( #464 )
...
* Add Serving and Runtime use Clone
* support TRT, OpenVINO and Paddle Backend
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-11-04 17:16:40 +08:00
czr-gc
ede59af857
[Backend] support ipu in paddle inference backend. ( #437 )
...
* feat(ipu): add ipu support for paddle_infer backend.
* fix(): remove unused env.
* fix(ipu): simplify user API for IPU.
* fix(cmake): fix merge conflict error in CMakeList.
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-30 18:59:59 +08:00
heliqi
b064ddf7ed
[Serving][backend]serving support multi stream and backend support external stream ( #431 )
...
* serving support multi stream
* pybind add external stream
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-26 14:46:13 +08:00
Wang Xinyu
43d86114d8
[Backend] TRT backend & PP-Infer backend support pinned memory ( #403 )
...
* TRT backend use pinned memory
* refine fd tensor pinned memory logic
* TRT enable pinned memory configurable
* paddle inference support pinned memory
* pinned memory pybindings
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-21 18:51:36 +08:00
Jack Zhou
dccb737d8d
[Backend] Add collect shape for pp-trt backend ( #372 )
...
* Add collect_shape attr
* add EnableTunedTensorRtDynamicShape
* Add collect shape python api
* Fix quant model not set trt dynamic shape
* Add shape info print
* Fix shape print
* Use CopyFromCpu instead of ShareExternalData
* Add ENABLE_TRT_BACKEND macro
* Add shared data with
2022-10-20 17:02:56 +08:00
Jason
945e197bd1
Fix bug of get input/output information from PaddleBackend ( #339 )
...
* Fix bug of get input/output information from PaddleBackend
* Support Paddle Inference with TensorRT (#340 )
* Fix bug
2022-10-12 11:37:26 +08:00
yeliang2258
2a68a23baf
Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend ( #257 )
...
* add quantize model support for trt and paddle
* fix bugs
* fix
* update paddle2onnx version
* update version
* add quantize test
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-09 20:00:05 +08:00
ChaoII
116265d239
add override mark ( #312 )
...
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-07 11:38:31 +08:00
heliqi
0805ead0ed
add 'GetOutputInfos' and 'GetInputInfos' interface ( #232 )
...
add GetOutputInfos GetInputInfos
2022-09-15 13:09:31 +08:00
Jason
68523be411
Modify file structure to separate python and cpp code ( #223 )
...
Modify code structure
2022-09-14 15:44:13 +08:00
Jason
ffbc5cc42d
Move cpp code to directory csrcs ( #42 )
...
* move cpp code to csrcs
* move cpp code to csrcs
2022-07-26 17:59:02 +08:00
Jason
51ecb407d4
Add Paddle Inference as backend ( #33 )
...
* Add Paddle Inference as backend
* Update CMakeLists.txt
* Fix default backend option
* Fix some log bug and backend choosing strategy
* Add version of g++
2022-07-22 09:53:46 +08:00
