yeliang2258
cae341e6c5
[Backend] Add fp16 support for ONNXRuntime-GPU ( #1239 )
...
* add fp16 support for ort-gpu
* add enable_ort_fp16 option
* fix
* fix bugs
* use fp16 for test
* update code
* update code
* fix windows bug
2023-03-23 18:23:13 +08:00
Jason
8ec1353cd8
[Doc] Add notes for tensors ( #1533 )
...
Add notes for tensors
2023-03-07 18:46:48 +08:00
DefTruth
e86905ca0d
[Bug Fix] move enable_valid_backend_check option -> runtime_option ( #1450 )
...
* [Model] Add skip valid backend check option -> FastDeployModel
* [Model] Add skip valid backend check option -> FastDeployModel
* [Model] Add skip valid backend check option -> FastDeployModel
* [FastDeploy] move enable_valid_backend_check option -> runtime_option
2023-02-28 09:57:31 +08:00
yunyaoXYY
c38b7d4377
[Backend] Support onnxruntime DirectML inference. ( #1304 )
...
* Fix links in readme
* Fix links in readme
* Update PPOCRv2/v3 examples
* Update auto compression configs
* Add neww quantization support for paddleclas model
* Update quantized Yolov6s model download link
* Improve PPOCR comments
* Add English doc for quantization
* Fix PPOCR rec model bug
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add Ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Support DirectML in onnxruntime
* Support onnxruntime DirectML
* Support onnxruntime DirectML
* Support onnxruntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Remove DirectML vision model example
* Imporve OnnxRuntime DirectML
* Imporve OnnxRuntime DirectML
* fix opencv cmake in Windows
* recheck codestyle
2023-02-17 10:53:51 +08:00
Jason
7a4b2f3177
[Other] Optimize rknpu2 backend ( #1318 )
...
* Optimize rknpu2 backend
* Fix error
* Fix comparision error
2023-02-15 11:11:37 +08:00
Jack Zhou
41c5f2de5a
Merge pull request #1305 from wwbitejotunn/set_stream_infer-shareExData
...
[Backend] Set inference with extra stream and prebind output tensor with share ex data
2023-02-14 17:08:56 +08:00
Zheng-Bicheng
cd5a54cd75
[Bug Fix] Fix the bug reported by RK3568 Runtime under the latest driver ( #1311 )
...
* 更新rkyolo runtime 在rk3568最新驱动下报错的问题
* 删除debug代码
2023-02-14 15:40:15 +08:00
Wang Bojun
a0fc0d5e49
Merge branch 'develop' into set_stream_infer-shareExData
2023-02-14 14:23:33 +08:00
wwbitejotunn
727fdc5863
reuse setStream c++ interface, add set_raw_stream python api
2023-02-14 06:22:47 +00:00
wwbitejotunn
289d353d99
Merge branch 'develop' of https://github.com/paddlepaddle/fastdeploy into set_stream_infer-shareExData
2023-02-13 03:14:18 +00:00
huangjianhui
a30213b504
Merge branch 'develop' into encrypt_model
2023-02-13 10:42:46 +08:00
WJJ1995
59a4ab343f
[Benchmark]Add ResourceUsageMonitor to collect memory info ( #1269 )
...
* add GPL lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* support yolov8
* add pybind for yolov8
* add yolov8 readme
* add cpp benchmark
* add cpu and gpu mem
* public part split
* add runtime mode
* fixed bugs
* add cpu_thread_nums
* deal with comments
* deal with comments
* deal with comments
* rm useless code
* add FASTDEPLOY_DECL
* add FASTDEPLOY_DECL
* fixed for windows
* mv rss to pss
* mv rss to pss
* Update utils.cc
* use thread to collect mem
* Add ResourceUsageMonitor
* rm useless code
* fixed bug
* fixed typo
* update ResourceUsageMonitor
* fixed bug
* fixed bug
* add note for ResourceUsageMonitor
* deal with comments
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-02-10 17:13:22 +08:00
felixhjh
5160771a1c
Update runtime_option.h
2023-02-10 03:24:45 +00:00
wwbitejotunn
59c5fedc36
set stream by raw size_t
2023-02-09 12:53:08 +00:00
Jason
b8afb0d040
[Other] Move comments for deprecated functions ( #1275 )
...
Move comments for deprecated functions
2023-02-09 10:04:18 +08:00
Jason
a4b0565b9a
[Other] Optimize paddle backend ( #1265 )
...
* Optimize paddle backend
* optimize paddle backend
* add version support
2023-02-08 19:12:03 +08:00
Jason
c25aa71fa9
Revert "[Benchmark]Benchmark cpp for YOLOv5" ( #1250 )
...
Revert "[Benchmark]Benchmark cpp for YOLOv5 (#1224 )"
This reverts commit c487359e33
2023-02-07 22:14:48 +08:00
WJJ1995
c487359e33
[Benchmark]Benchmark cpp for YOLOv5 ( #1224 )
...
* add GPL lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* support yolov8
* add pybind for yolov8
* add yolov8 readme
* add cpp benchmark
* add cpu and gpu mem
* public part split
* add runtime mode
* fixed bugs
* add cpu_thread_nums
* deal with comments
* deal with comments
* deal with comments
* rm useless code
* add FASTDEPLOY_DECL
* add FASTDEPLOY_DECL
2023-02-07 21:26:04 +08:00
Jason
713afe7f1c
[Other] Deprecate some option api and parameters ( #1243 )
...
* Optimize Poros backend
* fix error
* Add more pybind
* fix conflicts
* add some deprecate notices
* [Other] Deprecate some apis in RuntimeOption (#1240 )
* Deprecate more options
* modify serving
* Update option.h
* fix tensorrt error
* Update option_pybind.cc
* Update option_pybind.cc
* Fix error in serving
* fix word spell error
2023-02-07 17:57:46 +08:00
Jason
7c9bf11c44
[Other] Optimize Poros backend ( #1232 )
...
* Optimize Poros backend
* fix error
* Add more pybind
* fix conflicts
* add some deprecate notices
2023-02-07 10:18:03 +08:00
DefTruth
f73a538f61
[Backend] support bechmark mode for runtime and backend ( #1201 )
...
* [backend] support bechmark mode for runtime and backend
* [backend] support bechmark mode for runtime and backend
* [pybind11] add benchmark methods pybind
* [pybind11] add benchmark methods pybind
* [Other] Update build scripts
* [Other] Update cmake/summary.cmake
* [Other] update build scripts
* [Other] add ENABLE_BENCHMARK option -> setup.py
* optimize backend time recording
* optimize backend time recording
* optimize trt backend time record
* [backend] optimze backend_time recording for trt
* [benchmark] remove redundant logs
* fixed ov_backend confilct
* [benchmark] fixed paddle_backend conflicts
* [benchmark] fixed paddle_backend conflicts
* [benchmark] fixed paddle_backend conflicts
* [benchmark] remove use_gpu option from ort backend option
* [benchmark] update benchmark_ppdet.py
* [benchmark] update benchmark_ppcls.py
* fixed lite backend conflicts
* [Lite] fixed lite xpu
* add benchmark macro
* add RUNTIME_PROFILE_LOOP macros
* add comments for RUNTIME_PROFILE macros
* add comments for new apis
* add comments for new apis
* update benchmark_ppdet.py
* afixed bugs
* remove unused codes
* optimize RUNTIME_PROFILE_LOOP macros
* optimize RUNTIME_PROFILE_LOOP macros
* add comments for benchmark option and result
* add docs for benchmark namespace
2023-02-06 14:29:35 +08:00
Jason
5033000f41
[Other] Optimize OpenVINO backend ( #1228 )
...
* Optimize OpenVINO backend
* Update ov_backend.cc
2023-02-03 14:18:30 +08:00
Jason
a711f99c69
[Other] Optimize runtime module ( #1211 )
...
* modify ort option
* update code
* Unify backend
2023-02-02 13:10:40 +08:00
Jason
b4e322af63
[Other] Optimize load model from memory function ( #1205 )
...
Optimize option for runtime
2023-02-01 15:50:38 +08:00
huangjianhui
76df90afc3
[Other] FastDeploy TensorRT && ONNX backend support to load model form memory ( #1130 )
...
* Update all backends load model from buffer
* Delete redundant code
* Format code style
* Format code style
* Delete redundant code
* Delete redundant code
* Add some FDASSERTs
* Update load model form memory when cloning engine
* Update clone engine code
* Update set_model_buffer api parameters with char pointer
* Release memory buffer variables after finish init backends
* Fix conflict
* Fix bug
2023-02-01 11:36:09 +08:00
Jason
d00df3d621
[Backend] Remove all lite options in RuntimeOption ( #1109 )
...
* Remove all lite options in RuntimeOption
* Fix code error
* move pybind
* Fix build error
2023-01-13 19:40:34 +08:00
Jason
4aa4ebd7c3
[Other] [Part2] Upgrade runtime module ( #1080 )
...
[Other] Upgrade runtime module
2023-01-09 13:22:51 +08:00
Jason
d7a65e5c70
[Other] Upgrade runtime module ( #1068 )
...
* Upgrade runtime module
* Update option.h
* Fix build error
* Move enumerates
* little modification
* little modification
* little modification:
* Remove some useless flags
2023-01-06 13:44:05 +08:00
