apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-06 17:17:14 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
2,258 Commits 23 Branches 7 Tags
409c5dc05669b51a8fcc392cd161f24956761491
Commit Graph

75 Commits

Author SHA1 Message Date
WJJ1995
409c5dc056 [Backend]Add default value for TRT SetShape() (#1712) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

* add macros

* deal with comments

* deal with comments

* deal with comments

* re-lint

* rm pmap and use mem api

* rm pmap and use mem api

* add mem api

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* deal with comments

* fixed enable_paddle_to_trt

* add log for paddle_trt

* support ppcls benchmark

* use new trt option api

* update benchmark info

* simplify benchmark.cc

* simplify benchmark.cc

* deal with comments

* Add ppseg && ppocr benchmark

* add OCR rec img

* add ocr benchmark

* fixed trt shape

* add trt shape

* resolve conflict

* add ENABLE_BENCHMARK define

* Add ClassifyDiff

* Add Resize for ClassifyResult

* deal with comments

* add convert info script

* resolve conflict

* Add SaveBenchmarkResult func

* fixed bug

* fixed bug

* fixed bug

* add config.txt for option

* fixed bug

* fixed bug

* fixed bug

* add benchmark.sh

* mv thread_nums from 8 to 1

* deal with comments

* deal with comments

* fixed readme

* deal with comments

* add all platform shell

* Update config.arm.txt

* Update config.gpu.txt

* Update config.x86.txt

* fixed printinfo bug

* rm proxy

* add more model support

* all backend config.txt

* deal with comments

* Add MattingDiff compare

* fixed predict bug

* adjust warmup/repeat times

* add e2e/mem configs

* fixed typo

* open collect_mem

* fixed typo

* add trt cache option

* fixed bug

* fixed repeat times

* test for benchmark

* test for det benchmark

* for benchmark

* fixed for x86

* add h2d and d2h config

* renmae txt file

* add dynamic shape for pp_trt

* fixed typo

* Update option.h

* add collect shape

* add default value for SetShape()

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-03-27 14:03:47 +08:00
WJJ1995
5c70db176f [Backend]Add switch_ir_debug for Paddle Backend (#1700) 
* avoid mem copy for cpp benchmark

* set CMAKE_BUILD_TYPE to Release

* Add SegmentationDiff

* change pointer to reference

* fixed bug

* cast uint8 to int32

* Add diff compare for OCR

* Add diff compare for OCR

* rm ppocr pipeline

* Add yolov5 diff compare

* Add yolov5 diff compare

* deal with comments

* deal with comments

* fixed bug

* fixed bug

* fixed thread nums

* Add Failed log

* optimizer x86 pipeline

* Add switch_ir_debug for paddle backend

* fixed for ci
 2023-03-24 17:29:31 +08:00
yeliang2258
cae341e6c5 [Backend] Add fp16 support for ONNXRuntime-GPU (#1239) 
* add fp16 support for ort-gpu

* add enable_ort_fp16 option

* fix

* fix bugs

* use fp16 for test

* update code

* update code

* fix windows bug
 2023-03-23 18:23:13 +08:00
yunyaoXYY
b3e16e9966 ORT support UINT8 and INT8 input and output 2023-03-21 02:23:23 +00:00
Jack Zhou
012c7771c1 [Serving] Add collect shape and fix serving infer (#1658) 
Add collect shape and fix serving infer
 2023-03-20 19:55:30 +08:00
DefTruth
6af3691568 [XPU] Update Lite XPU lib and fixed some bugs (#1647) 2023-03-17 21:39:39 +08:00
Jason
3b1343c726 [Bug] Fix loadding big model loadding problem (#1636) 
Fix loadding big model loadding problem
 2023-03-17 10:25:26 +08:00
Ethan Yang
4f48d84c27 [Backend] optimize cpu usage on openvino backend (#1531) 
* optimiza cpu usage on openvino backend

conver the hint varible to string

* update with affinity mode set-up

---------

Co-authored-by: Ethan <yicheng@intel.com>
Co-authored-by: Jason <jiangjiajun@baidu.com>
 2023-03-17 10:24:25 +08:00
zhoushunjie
b7fb3aff08 remoove fdinfo in BindOutputTensor 2023-03-14 12:53:37 +00:00
Jason
3d36e26b8a [Backend] Remove some deprecated api (#1574) 
Remove some deprecated api
 2023-03-13 13:42:43 +08:00
Jack Zhou
f4736e7931 Merge pull request #1552 from joey12300/fix_delete_pass 
[Backend] Fix delete pass of paddle inference
 2023-03-08 19:57:08 +08:00
Jason
6be2c0367b [Example] Update runtime examples (#1542) 
* Add notes for tensors

* Optimize some apis

* move some warnings
 2023-03-08 16:56:04 +08:00
zhoushunjie
384eca14fd Fix delete pass 2023-03-08 06:32:27 +00:00
Zheng-Bicheng
0687d3b0ad [Model] Add Solov2 For PaddleDetection (#1435) 
* update solov2

* Repair note

* update solov2 postprocess

* update

* update solov2

* update solov2

* fixed bug

* fixed bug

* update solov2

* update solov2

* fix build android bug

* update docs

* update docs

* update docs

* update

* update

* update arch and docs

* update

* update

* update solov2 python

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-03-08 10:01:32 +08:00
Jason
7e4ef4a72c Merge branch 'develop' into rknn_pybind 2023-03-07 18:53:53 +08:00
Jason
8ec1353cd8 [Doc] Add notes for tensors (#1533) 
Add notes for tensors
 2023-03-07 18:46:48 +08:00
Zheng-Bicheng
458523fc29 update pybind 2023-03-07 15:50:49 +08:00
Zheng-Bicheng
610b4c532b [Bug Fix] fixed rknn bugs (#1468) 
fixed rknn bugs
 2023-03-01 10:21:58 +08:00
DefTruth
e86905ca0d [Bug Fix] move enable_valid_backend_check option -> runtime_option (#1450) 
* [Model] Add skip valid backend check option -> FastDeployModel

* [Model] Add skip valid backend check option -> FastDeployModel

* [Model] Add skip valid backend check option -> FastDeployModel

* [FastDeploy] move enable_valid_backend_check option -> runtime_option
 2023-02-28 09:57:31 +08:00
Jack Zhou
524c85745b [Backend] Add fixed size optimization for transformer model (#1430) 
Add enable_fixed_size_opt flag
 2023-02-24 09:45:04 +08:00
WJJ1995
2f8d9c9a57 [Benchmark]Add SegmentationDiff to compare SegmentationResult diff (#1404) 
* avoid mem copy for cpp benchmark

* set CMAKE_BUILD_TYPE to Release

* Add SegmentationDiff

* change pointer to reference

* fixed bug

* cast uint8 to int32
 2023-02-22 14:42:21 +08:00
Jason
dd527388bc Update option.h 2023-02-22 11:28:04 +08:00
Jason
18e33bae5c [Other] Optimize runtime module (#1356) 
* Optimize runtime

* fix error

* [Backend] Add option to print tensorrt conversion log (#1386)

Add option to print tensorrt conversion log

Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com>

---------

Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com>
 2023-02-21 17:01:32 +08:00
WJJ1995
b6658b8094 [Benchmark] Add ppocr && ppseg benchmark (#1344) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

* add macros

* deal with comments

* deal with comments

* deal with comments

* re-lint

* rm pmap and use mem api

* rm pmap and use mem api

* add mem api

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* deal with comments

* fixed enable_paddle_to_trt

* add log for paddle_trt

* support ppcls benchmark

* use new trt option api

* update benchmark info

* simplify benchmark.cc

* simplify benchmark.cc

* deal with comments

* Add ppseg && ppocr benchmark

* add OCR rec img

* add ocr benchmark

* fixed trt shape

* add trt shape

* resolve conflict

* add ENABLE_BENCHMARK define

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-02-20 15:59:27 +08:00
Jason
db471c3466 [Other] Optimize poros backend (#1331) 
* Optimize poros backend

* Fix pybind error

---------

Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com>
 2023-02-17 14:32:35 +08:00
yunyaoXYY
c38b7d4377 [Backend] Support onnxruntime DirectML inference. (#1304) 
* Fix links in readme

* Fix links in readme

* Update PPOCRv2/v3 examples

* Update auto compression configs

* Add neww quantization  support for paddleclas model

* Update quantized Yolov6s model download link

* Improve PPOCR comments

* Add English doc for quantization

* Fix PPOCR rec model bug

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add Ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Support DirectML in onnxruntime

* Support onnxruntime DirectML

* Support onnxruntime DirectML

* Support onnxruntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Remove DirectML vision model example

* Imporve OnnxRuntime DirectML

* Imporve OnnxRuntime DirectML

* fix opencv cmake in Windows

* recheck codestyle
 2023-02-17 10:53:51 +08:00
Jason
7a4b2f3177 [Other] Optimize rknpu2 backend (#1318) 
* Optimize rknpu2 backend

* Fix error

* Fix comparision error
 2023-02-15 11:11:37 +08:00
WJJ1995
c25d1cc1bc [Backend]Fixed enable_paddle_to_trt() bug (#1320) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

* add macros

* deal with comments

* deal with comments

* deal with comments

* re-lint

* rm pmap and use mem api

* rm pmap and use mem api

* add mem api

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* deal with comments

* fixed enable_paddle_to_trt

* add log for paddle_trt

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-02-14 17:51:39 +08:00
Jack Zhou
41c5f2de5a Merge pull request #1305 from wwbitejotunn/set_stream_infer-shareExData 
[Backend] Set inference with extra stream and prebind output tensor with share ex data
 2023-02-14 17:08:56 +08:00
Zheng-Bicheng
cd5a54cd75 [Bug Fix] Fix the bug reported by RK3568 Runtime under the latest driver (#1311) 
* 更新rkyolo runtime 在rk3568最新驱动下报错的问题

* 删除debug代码
 2023-02-14 15:40:15 +08:00
Wang Bojun
a0fc0d5e49 Merge branch 'develop' into set_stream_infer-shareExData 2023-02-14 14:23:33 +08:00
wwbitejotunn
727fdc5863 reuse setStream c++ interface, add set_raw_stream python api 2023-02-14 06:22:47 +00:00
wwbitejotunn
f1ab47a4ef code revine 2023-02-13 18:16:45 +00:00
felixhjh
031d02275b Merge branch 'encrypt_model' of https://github.com/felixhjh/FastDeploy into encrypt_model 2023-02-13 06:32:19 +00:00
felixhjh
45b070e495 Format code 2023-02-13 06:31:04 +00:00
wwbitejotunn
289d353d99 Merge branch 'develop' of https://github.com/paddlepaddle/fastdeploy into set_stream_infer-shareExData 2023-02-13 03:14:18 +00:00
wwbitejotunn
abfa9fd850 prebind output by shareExternalData 2023-02-13 03:11:31 +00:00
huangjianhui
a30213b504 Merge branch 'develop' into encrypt_model 2023-02-13 10:42:46 +08:00
WJJ1995
59a4ab343f [Benchmark]Add ResourceUsageMonitor to collect memory info (#1269) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-02-10 17:13:22 +08:00
huangjianhui
e8b55ab3c6 Merge branch 'develop' into encrypt_model 2023-02-10 13:03:40 +08:00
felixhjh
5160771a1c Update runtime_option.h 2023-02-10 03:24:45 +00:00
felixhjh
942cee83d7 Add decrypt function to load encrypted model 2023-02-09 14:04:10 +00:00
wwbitejotunn
898b063216 get cache dir 2023-02-09 20:56:55 +08:00
wwbitejotunn
c2e5f6317e fix paddle backend 2023-02-09 20:56:55 +08:00
wwbitejotunn
59c5fedc36 set stream by raw size_t 2023-02-09 12:53:08 +00:00
yeliang2258
4742f97819 [Bug Fix] Fix KunlunXin valid_places (#1285) 
fix KunlunXin valid_places
 2023-02-09 20:39:44 +08:00
wwbitejotunn
4b293a89de fix paddle backend 2023-02-09 05:51:30 +00:00
DefTruth
ab5377b5fa [Benchmark] Add PaddleYOLOv8 cpp benchmark example & lite flags option (#1270) 
* [Android] Add PaddleYOLOv8 cpp benchmark example & lite flags option

* [Benchmark] add linux x86_64 gpu benchmark build script
 2023-02-09 11:58:07 +08:00
Jason
b8afb0d040 [Other] Move comments for deprecated functions (#1275) 
Move comments for deprecated functions
 2023-02-09 10:04:18 +08:00
Jason
0fb2f26352 [Other] Optimize sophgo backend (#1273) 
optimize sophgo backend
 2023-02-08 19:50:37 +08:00