heliqi
b064ddf7ed
[Serving][backend]serving support multi stream and backend support external stream ( #431 )
...
* serving support multi stream
* pybind add external stream
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-26 14:46:13 +08:00
Jack Zhou
8af35e5114
[Backend] Enable Memory Optim for paddle backend ( #417 )
...
Enable Memory Optim for paddle backend
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-24 19:30:24 +08:00
Wang Xinyu
43d86114d8
[Backend] TRT backend & PP-Infer backend support pinned memory ( #403 )
...
* TRT backend use pinned memory
* refine fd tensor pinned memory logic
* TRT enable pinned memory configurable
* paddle inference support pinned memory
* pinned memory pybindings
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-21 18:51:36 +08:00
Jack Zhou
dccb737d8d
[Backend] Add collect shape for pp-trt backend ( #372 )
...
* Add collect_shape attr
* add EnableTunedTensorRtDynamicShape
* Add collect shape python api
* Fix quant model not set trt dynamic shape
* Add shape info print
* Fix shape print
* Use CopyFromCpu instead of ShareExternalData
* Add ENABLE_TRT_BACKEND macro
* Add shared data with
2022-10-20 17:02:56 +08:00
heliqi
831ef42207
fix paddle-mkldnn bug ( #366 )
...
* fix input name bug
* fix paddle-mkldnn bug
2022-10-13 21:37:31 +08:00
Jason
945e197bd1
Fix bug of get input/output information from PaddleBackend ( #339 )
...
* Fix bug of get input/output information from PaddleBackend
* Support Paddle Inference with TensorRT (#340 )
* Fix bug
2022-10-12 11:37:26 +08:00
yeliang2258
2a68a23baf
Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend ( #257 )
...
* add quantize model support for trt and paddle
* fix bugs
* fix
* update paddle2onnx version
* update version
* add quantize test
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-09 20:00:05 +08:00
heliqi
0805ead0ed
add 'GetOutputInfos' and 'GetInputInfos' interface ( #232 )
...
add GetOutputInfos GetInputInfos
2022-09-15 13:09:31 +08:00
Jason
68523be411
Modify file structure to separate python and cpp code ( #223 )
...
Modify code structure
2022-09-14 15:44:13 +08:00
Jason
ffbc5cc42d
Move cpp code to directory csrcs ( #42 )
...
* move cpp code to csrcs
* move cpp code to csrcs
2022-07-26 17:59:02 +08:00
Jason
51ecb407d4
Add Paddle Inference as backend ( #33 )
...
* Add Paddle Inference as backend
* Update CMakeLists.txt
* Fix default backend option
* Fix some log bug and backend choosing strategy
* Add version of g++
2022-07-22 09:53:46 +08:00
