heliqi
42f1888bb0
[Serving][Backend] Backend support zero_copy_infer and Serving reduce the output memory copy ( #703 )
...
* backend add zero copy infer interface
* fix bug
* fix bug
* fix bug
* paddle ipu
2022-11-28 14:07:53 +08:00
yeliang2258
d14828cb18
[Backend] Add AdaptivePool2d for TensorRT plugin ( #668 )
...
* add adaptivepool2d for tensorrt plugin
* update code
* update code
* update code to fix bug
2022-11-25 17:36:59 +08:00
Jason
f2fed7959b
[Other] Add namespace for functions ( #538 )
...
Add namespace for functions
2022-11-09 13:57:53 +08:00
yeliang2258
8d31fe59e8
[Backend] Support the deployment of models which is larger than 2G in ORT and TRT backend ( #514 )
...
* support model bigger than 2G
* update code
* update code
* fix bug
* update code
2022-11-09 11:14:15 +08:00
heliqi
277bec38c7
[Backend & Serving] Serving and Runtime support Clone ( #464 )
...
* Add Serving and Runtime use Clone
* support TRT, OpenVINO and Paddle Backend
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-11-04 17:16:40 +08:00
Jason
9fa612c24b
[Backend] Enable TensorRT BatchedNMSDynamic_TRT plugin ( #449 )
...
* Enable TensorRT EfficientNMS plugin
* remove some temporary code
* Update trt_backend.cc
* Update utils.h
2022-11-04 11:46:29 +08:00
Wang Xinyu
caa369f64a
[Backend] TRT cast GPU input from int64 to int32, output from int32 to int64, and Windows support building CUDA files ( #426 )
...
* TRT cast int64 to int32
* windows cmake build cuda src
* fix windows cmake error when build cuda src
* add a notice in windows gpu build doc
* cmake add cuda std=11
* TRT cast output from int32 to int64
* nits
* trt get original input output dtype
2022-10-28 13:38:06 +08:00
heliqi
b064ddf7ed
[Serving][backend]serving support multi stream and backend support external stream ( #431 )
...
* serving support multi stream
* pybind add external stream
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-26 14:46:13 +08:00
WJJ1995
718698a32a
[Model] add RobustVideoMatting model ( #400 )
...
* add yolov5cls
* fixed bugs
* fixed bugs
* fixed preprocess bug
* add yolov5cls readme
* deal with comments
* Add YOLOv5Cls Note
* add yolov5cls test
* add rvm support
* support rvm model
* add rvm demo
* fixed bugs
* add rvm readme
* add TRT support
* add trt support
* add rvm test
* add EXPORT.md
* rename export.md
* rm poros doxyen
* deal with comments
* deal with comments
* add rvm video_mode note
Co-authored-by: Jason <jiangjiajun@baidu.com >
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2022-10-26 14:30:04 +08:00
Wang Xinyu
43d86114d8
[Backend] TRT backend & PP-Infer backend support pinned memory ( #403 )
...
* TRT backend use pinned memory
* refine fd tensor pinned memory logic
* TRT enable pinned memory configurable
* paddle inference support pinned memory
* pinned memory pybindings
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-21 18:51:36 +08:00
Jason
24c8fdc27d
Integrate FalconCV to boost image processing ( #332 )
...
* Rename GetCpuMat to GetOpenCVMat
* refine code structure
2022-10-16 14:49:57 +08:00
jiangjiajun
7d9c47753f
Fix dead link in notice
2022-10-16 06:24:47 +00:00
Jason
548777a283
Add document describes some tricks while using TensorRT backend ( #343 )
...
* add tensorrt tricks doc
* add quick start
2022-10-11 21:19:55 +08:00
yeliang2258
2a68a23baf
Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend ( #257 )
...
* add quantize model support for trt and paddle
* fix bugs
* fix
* update paddle2onnx version
* update version
* add quantize test
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-09 20:00:05 +08:00
Jack Zhou
dd365fb721
Add trt max workspace setting ( #308 )
...
* add trt max workspace setting
* fix set trt max workspace
2022-09-30 09:54:34 +08:00
Jason
360bee46b5
Support build on jetson ( #276 )
...
* support jetson build
* Update allocate.cc
* update setup
* Update CMakeLists.txt
* Update setup.py
2022-09-23 13:11:11 +08:00
DefTruth
5edd9fccd3
[TRT] fix tensorrt serialize file problem ( #274 )
...
fix tensorrt serialize file problem
2022-09-23 00:40:17 +08:00
heliqi
0805ead0ed
add 'GetOutputInfos' and 'GetInputInfos' interface ( #232 )
...
add GetOutputInfos GetInputInfos
2022-09-15 13:09:31 +08:00
Jason
68523be411
Modify file structure to separate python and cpp code ( #223 )
...
Modify code structure
2022-09-14 15:44:13 +08:00
Jason
ffbc5cc42d
Move cpp code to directory csrcs ( #42 )
...
* move cpp code to csrcs
* move cpp code to csrcs
2022-07-26 17:59:02 +08:00
Jason
17e4dc6b5e
Support remove multiclass_nms to enable ppyoloe to tensorrt ( #40 )
...
* Add custom operator for onnxruntime ans fix paddle backend
* Polish cmake files and runtime apis
* Remove copy libraries
* fix some issue
* fix bug
* fix bug
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* add common operator multiclassnms
* fix compile problem
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com >
2022-07-26 11:16:01 +08:00
Jason
279c993483
Polish cmake files and runtime apis ( #36 )
...
* Add custom operator for onnxruntime ans fix paddle backend
* Polish cmake files and runtime apis
* Remove copy libraries
* fix some issue
* fix bug
* fix bug
2022-07-25 08:59:53 +08:00
Jason
90061e11f5
Fix outputs order of tensorrt ( #18 )
...
* fix trt output order
* Update trt_backend.cc
2022-07-14 19:19:56 +08:00
DefTruth
a45f189b56
fixed missing trt_backend option & remove un-need data layout check in Cast ( #14 )
...
* update .gitignore
* Added checking for cmake include dir
* fixed missing trt_backend option bug when init from trt
* remove un-need data layout and add pre-check for dtype
* changed RGB2BRG to BGR2RGB in ppcls model
* Update CMakeLists.txt
2022-07-12 15:14:30 +08:00
Jason
2b742d37e0
fix some bugs ( #12 )
2022-07-08 19:36:38 +08:00
jiangjiajun
9d87046d78
first commit
2022-07-05 09:30:15 +00:00
