Wang Xinyu
caa369f64a
[Backend] TRT cast GPU input from int64 to int32, output from int32 to int64, and Windows support building CUDA files ( #426 )
...
* TRT cast int64 to int32
* windows cmake build cuda src
* fix windows cmake error when build cuda src
* add a notice in windows gpu build doc
* cmake add cuda std=11
* TRT cast output from int32 to int64
* nits
* trt get original input output dtype
2022-10-28 13:38:06 +08:00
DefTruth
a51e5a6e55
[Android] Add android aar package ( #416 )
...
* [Android] Add Android build docs and demo (#26 )
* [Backend] Add override flag to lite backend
* [Docs] Add Android C++ SDK build docs
* [Doc] fix android_build_docs typos
* Update CMakeLists.txt
* Update android.md
* [Doc] Add PicoDet Android demo docs
* [Doc] Update PicoDet Andorid demo docs
* [Doc] Update PaddleClasModel Android demo docs
* [Doc] Update fastdeploy android jni docs
* [Doc] Update fastdeploy android jni usage docs
* [Android] init fastdeploy android jar package
* [Backend] support int8 option for lite backend
* [Model] add Backend::Lite to paddle model
* [Backend] use CopyFromCpu for lite backend.
* [Android] package jni srcs and java api into aar
* Update infer.cc
* Update infer.cc
* [Android] Update package build.gradle
* [Android] Update android app examples
* [Android] update android detection app
2022-10-26 17:01:14 +08:00
heliqi
b064ddf7ed
[Serving][backend]serving support multi stream and backend support external stream ( #431 )
...
* serving support multi stream
* pybind add external stream
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-26 14:46:13 +08:00
WJJ1995
718698a32a
[Model] add RobustVideoMatting model ( #400 )
...
* add yolov5cls
* fixed bugs
* fixed bugs
* fixed preprocess bug
* add yolov5cls readme
* deal with comments
* Add YOLOv5Cls Note
* add yolov5cls test
* add rvm support
* support rvm model
* add rvm demo
* fixed bugs
* add rvm readme
* add TRT support
* add trt support
* add rvm test
* add EXPORT.md
* rename export.md
* rm poros doxyen
* deal with comments
* deal with comments
* add rvm video_mode note
Co-authored-by: Jason <jiangjiajun@baidu.com >
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2022-10-26 14:30:04 +08:00
Jack Zhou
8af35e5114
[Backend] Enable Memory Optim for paddle backend ( #417 )
...
Enable Memory Optim for paddle backend
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-24 19:30:24 +08:00
Wang Xinyu
43d86114d8
[Backend] TRT backend & PP-Infer backend support pinned memory ( #403 )
...
* TRT backend use pinned memory
* refine fd tensor pinned memory logic
* TRT enable pinned memory configurable
* paddle inference support pinned memory
* pinned memory pybindings
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-21 18:51:36 +08:00
huangjianhui
b565c15bf7
[Model] Add tinypose single && pipeline model ( #177 )
...
* Add tinypose model
* Add PPTinypose python API
* Fix picodet preprocess bug && Add Tinypose examples
* Update tinypose example code
* Update ppseg preprocess if condition
* Update ppseg backend support type
* Update permute.h
* Update README.md
* Update code with comments
* Move files dir
* Delete premute.cc
* Add single model pptinypose
* Delete pptinypose old code in ppdet
* Code format
* Add ppdet + pptinypose pipeline model
* Fix bug for posedetpipeline
* Change Frontend to ModelFormat
* Change Frontend to ModelFormat in __init__.py
* Add python posedetpipeline/
* Update pptinypose example dir name
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Create keypointdetection_result.md
* Create README.md
* Create README.md
* Create README.md
* Update README.md
* Update README.md
* Create README.md
* Fix det_keypoint_unite_infer.py bug
* Create README.md
* Update PP-Tinypose by comment
* Update by comment
* Add pipeline directory
* Add pptinypose dir
* Update pptinypose to align accuracy
* Addd warpAffine processor
* Update GetCpuMat to GetOpenCVMat
* Add comment for pptinypose && pipline
* Update docs/main_page.md
* Add README.md for pptinypose
* Add README for det_keypoint_unite
* Remove ENABLE_PIPELINE option
* Remove ENABLE_PIPELINE option
* Change pptinypose default backend
* PP-TinyPose Pipeline support multi PP-Detection models
* Update pp-tinypose comment
* Update by comments
* Add single test example
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-21 09:28:23 +08:00
Jack Zhou
dccb737d8d
[Backend] Add collect shape for pp-trt backend ( #372 )
...
* Add collect_shape attr
* add EnableTunedTensorRtDynamicShape
* Add collect shape python api
* Fix quant model not set trt dynamic shape
* Add shape info print
* Fix shape print
* Use CopyFromCpu instead of ShareExternalData
* Add ENABLE_TRT_BACKEND macro
* Add shared data with
2022-10-20 17:02:56 +08:00
DefTruth
c28f4d6019
[Backend] Add override flag to lite backend ( #402 )
2022-10-20 17:01:56 +08:00
WJJ1995
f5c94e5471
Support Poros Backend ( #188 )
...
* Add poros backend
* Add torch lib
* Add python3 lib
* set c++ 14 for poros
* fixed bugs
* fixed grammer bugs
* fixed grammer bugs
* fixed code bugs
* fixed code bugs
* fixed CreatePorosValue bug
* Add AtType2String for Log
* fixed trt_option
* fixed poros.cmake path
* fixed grammer bug
* fixed grammer bug
* fixed ambiguous reference
* fixed ambiguous reference
* fixed reference error
* fixed include files
* rm ENABLE_TRT_BACKEND in poros
* update CMakeLists.txt
* fixed CMakeLists.txt
* Add libtorch.so in CMakeLists.txt
* Fixed CMakeLists.txt
* Fixed CMakeLists.txt
* Fixed copy bug
* Fixed copy bug
* Fixed copy bug
* Fixed Cmake
* Fixed Cmake
* debug
* debug
* debug
* debug
* debug
* debug
* debug utils
* debug utils
* copy to cpu
* rm log info
* test share mem
* test share mem
* test share mem
* test multi outputs
* test multi outputs
* test multi outputs
* test multi outputs
* test multi outputs
* test multi outputs
* test multi outputs
* time cost
* time cost
* fixed bug
* time collect
* mem copy
* mem copy
* rm time log
* rm share mem
* fixed multi inputs bug
* add set_input_dtypes func
* add SetInputDtypes
* fixed bug
* fixed bug
* fixed prewarm data order
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* fixed bug
* Add compile func
* Add compile func
* Add compile func
* Add is_dynamic option
* Add is_dynamic option
* Add is_dynamic option
* Add is_dynamic option
* rm infer log
* add cuda11.6 poros lib
* fixed bug
* fixed bug
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* fixed multi outputs
* rm logs
* test
* test
* test
* add test log
* add test log
* add test log
* add test log
* support cpu
* support cpu
* support cpu
* support cpu
* support member variable definition
* rm useless log
* fixed name
* resolve conflict
* resolve conflict
* resolve conflict
* fixed cmake
* add GetInputInfos&GetOutputInfos
* add GetInputInfos&GetOutputInfos
* fixed bug
* fixed runtime.py
* add compile func
* add np
* deal with comments
* rm to_inter func
* add property
2022-10-17 15:28:12 +08:00
Jason
24c8fdc27d
Integrate FalconCV to boost image processing ( #332 )
...
* Rename GetCpuMat to GetOpenCVMat
* refine code structure
2022-10-16 14:49:57 +08:00
jiangjiajun
7d9c47753f
Fix dead link in notice
2022-10-16 06:24:47 +00:00
heliqi
831ef42207
fix paddle-mkldnn bug ( #366 )
...
* fix input name bug
* fix paddle-mkldnn bug
2022-10-13 21:37:31 +08:00
Jason
945e197bd1
Fix bug of get input/output information from PaddleBackend ( #339 )
...
* Fix bug of get input/output information from PaddleBackend
* Support Paddle Inference with TensorRT (#340 )
* Fix bug
2022-10-12 11:37:26 +08:00
Jason
548777a283
Add document describes some tricks while using TensorRT backend ( #343 )
...
* add tensorrt tricks doc
* add quick start
2022-10-11 21:19:55 +08:00
yeliang2258
2a68a23baf
Add paddle quantize model support for ORT, TRT and MKLDNN deploy backend ( #257 )
...
* add quantize model support for trt and paddle
* fix bugs
* fix
* update paddle2onnx version
* update version
* add quantize test
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-09 20:00:05 +08:00
ChaoII
116265d239
add override mark ( #312 )
...
Co-authored-by: Jason <jiangjiajun@baidu.com >
2022-10-07 11:38:31 +08:00
Jason
a8e08c9c1a
Merge branch 'develop' into add_float16_dtype
2022-10-05 10:47:52 +08:00
DefTruth
c4288a92e1
[lite] add more options for lite backend ( #313 )
...
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk
* [cmake] fixed patchelf download on mac and android
* [lite] Add threads and power_mode option support
* [pybind] update runtime pybind for lite power mode
* [python] Add set_lite_power_mode api to runtime
* [Lite] add lite enable_fp16 option support
* [lite] add more options for lite backend.
* [cmake] fixed Paddle Lite typo
* [runtime] format LitePowerMode enum comments
* [runtime] format lite option comments
2022-10-04 17:31:48 +08:00
zhoushunjie
faf04b49a6
Add float16 tests
2022-10-02 09:21:00 +00:00
zhoushunjie
ec8b183d9c
remote HOSTDEVICE modifier
2022-10-02 04:28:30 +00:00
Jason
8a1e2c6143
Add some comments for runtime ( #309 )
...
* Add some notes for runtime
* modify note
* modify note
* add some comment
* Add comment for runtime
* Add comment for runtime
* Add comment for runtime
* Add comment for runtime
2022-09-30 14:43:44 +08:00
Jack Zhou
dd365fb721
Add trt max workspace setting ( #308 )
...
* add trt max workspace setting
* fix set trt max workspace
2022-09-30 09:54:34 +08:00
DefTruth
c5f85de356
[lite] Add threads and power_mode option support ( #298 )
...
* [cmake] support Android arm64-v8a & armeabi-v7a native c++ sdk
* [cmake] fixed patchelf download on mac and android
* [lite] Add threads and power_mode option support
* [pybind] update runtime pybind for lite power mode
* [python] Add set_lite_power_mode api to runtime
2022-09-28 18:09:35 +08:00
Jason
360bee46b5
Support build on jetson ( #276 )
...
* support jetson build
* Update allocate.cc
* update setup
* Update CMakeLists.txt
* Update setup.py
2022-09-23 13:11:11 +08:00
DefTruth
5edd9fccd3
[TRT] fix tensorrt serialize file problem ( #274 )
...
fix tensorrt serialize file problem
2022-09-23 00:40:17 +08:00
Jason
addce837bc
Add paddlelite backend support ( #260 )
...
* Add paddlelite backend support
* Update CMakeLists.txt
* Update __init__.py
2022-09-21 13:22:34 +08:00
Jason
5046574b95
Fix bug for part of detection model ( #243 )
...
* fix error for part of detection model
* fix error for part of detection model
* add patch paddle inference
2022-09-16 16:31:34 +08:00
heliqi
0805ead0ed
add 'GetOutputInfos' and 'GetInputInfos' interface ( #232 )
...
add GetOutputInfos GetInputInfos
2022-09-15 13:09:31 +08:00
Jason
68523be411
Modify file structure to separate python and cpp code ( #223 )
...
Modify code structure
2022-09-14 15:44:13 +08:00
Jason
ffbc5cc42d
Move cpp code to directory csrcs ( #42 )
...
* move cpp code to csrcs
* move cpp code to csrcs
2022-07-26 17:59:02 +08:00
Jason
17e4dc6b5e
Support remove multiclass_nms to enable ppyoloe to tensorrt ( #40 )
...
* Add custom operator for onnxruntime ans fix paddle backend
* Polish cmake files and runtime apis
* Remove copy libraries
* fix some issue
* fix bug
* fix bug
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* Support remove multiclass_nms to enable paddledetection run tensorrt
* add common operator multiclassnms
* fix compile problem
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com >
2022-07-26 11:16:01 +08:00
Jason
279c993483
Polish cmake files and runtime apis ( #36 )
...
* Add custom operator for onnxruntime ans fix paddle backend
* Polish cmake files and runtime apis
* Remove copy libraries
* fix some issue
* fix bug
* fix bug
2022-07-25 08:59:53 +08:00
Jason
4b681581b1
Add custom operator for onnxruntime and fix paddle backend ( #35 )
...
Add custom operator for onnxruntime ans fix paddle backend
2022-07-23 22:21:36 +08:00
Jason
51ecb407d4
Add Paddle Inference as backend ( #33 )
...
* Add Paddle Inference as backend
* Update CMakeLists.txt
* Fix default backend option
* Fix some log bug and backend choosing strategy
* Add version of g++
2022-07-22 09:53:46 +08:00
Jason
90061e11f5
Fix outputs order of tensorrt ( #18 )
...
* fix trt output order
* Update trt_backend.cc
2022-07-14 19:19:56 +08:00
DefTruth
a45f189b56
fixed missing trt_backend option & remove un-need data layout check in Cast ( #14 )
...
* update .gitignore
* Added checking for cmake include dir
* fixed missing trt_backend option bug when init from trt
* remove un-need data layout and add pre-check for dtype
* changed RGB2BRG to BGR2RGB in ppcls model
* Update CMakeLists.txt
2022-07-12 15:14:30 +08:00
Jason
2b742d37e0
fix some bugs ( #12 )
2022-07-08 19:36:38 +08:00
jiangjiajun
9d87046d78
first commit
2022-07-05 09:30:15 +00:00
