Wang Bojun
7b58387d3c
add option for inference precision and cutlass ( #2017 )
...
* add option for inference precision and cutlass
* code clean
* code refine
* code refine
2023-06-09 10:20:48 +08:00
Zheng-Bicheng
8d357814e8
[Backend] Add pybind & PaddleDetection example for TVM ( #1998 )
...
* update
* update
* Update infer_ppyoloe_demo.cc
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-06-04 13:26:47 +08:00
DefTruth
284b1b46c8
[Serving] Support XPU encrypt & auth server ( #2007 )
...
* [patchelf] fix patchelf error for inference xpu
* [serving] add xpu dockerfile and support fd server
* [serving] add xpu dockerfile and support fd server
* [Serving] support XPU + Tritron
* [Serving] support XPU + Tritron
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] add comments for xpu tritron dockerfile
* [Doruntime] fix xpu infer error
* [Doruntime] fix xpu infer error
* [XPU] update xpu dockerfile
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* [XPU] Update XPU L3 Cache setting docs
* [XPU] Add Encryption and AUTH support for XPU Server
* [XPU] Add Encryption and AUTH support for XPU Server
* [Bug Fix] fix paddle reader error
* [Serving] Support XPU encrypt & auth server
* [Serving] Support XPU encrypt & auth server
* [Serving] Support XPU encrypt & auth server
* [Serving] Support XPU encrypt & auth server
* [Triton] switch TAG 22.12 -> TAG 21.10wq
* update xpu auth server script
2023-06-01 21:36:44 +08:00
DefTruth
387c5695b3
[XPU] Update XPU L3 Cache setting docs ( #2001 )
...
* [patchelf] fix patchelf error for inference xpu
* [serving] add xpu dockerfile and support fd server
* [serving] add xpu dockerfile and support fd server
* [Serving] support XPU + Tritron
* [Serving] support XPU + Tritron
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] add comments for xpu tritron dockerfile
* [Doruntime] fix xpu infer error
* [Doruntime] fix xpu infer error
* [XPU] update xpu dockerfile
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* [XPU] Update XPU L3 Cache setting docs
2023-05-30 11:21:04 +08:00
DefTruth
434b48dda5
[Serving] Support FastDeploy XPU Triton Server ( #1994 )
...
* [patchelf] fix patchelf error for inference xpu
* [serving] add xpu dockerfile and support fd server
* [serving] add xpu dockerfile and support fd server
* [Serving] support XPU + Tritron
* [Serving] support XPU + Tritron
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0
* [Dockerfile] add comments for xpu tritron dockerfile
* [Doruntime] fix xpu infer error
* [Doruntime] fix xpu infer error
* [XPU] update xpu dockerfile
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* add xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
* update xpu triton server docs
2023-05-29 14:38:25 +08:00
Zheng-Bicheng
643730bf5f
[Hackathon 181] Add TVM support for FastDeploy on macOS ( #1969 )
...
* update for tvm backend
* update third_party
* update third_party
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-05-25 19:59:02 +08:00
DefTruth
49c033a828
[XPU] Support XPU via Paddle Inference backend ( #1987 )
...
* [backend] Support XPU via Paddle Inference backend
* [backend] Support XPU via Paddle Inference backend
* [backend] Support XPU via Paddle Inference backend
* [XPU] support XPU benchmark via paddle inference
* [XPU] support XPU benchmark via paddle inference
* [benchmark] add xpu paddle h2d config files
2023-05-25 14:13:40 +08:00
heliqi
3e7cb88049
[Serving]support 22.12 ( #1974 )
...
support 22.12
2023-05-22 22:27:13 +08:00
linkk08
c8ff8b63e8
[XPU] Add docs for gm_default_size and fix type ( #1962 )
...
Add description of gm_default_size and change kunlunxin_gm_default_size‘s type to int64_t
2023-05-18 20:30:06 +08:00
DefTruth
1f66d5d9d3
[benchmark] support lite light api & optimize benchmark flags ( #1950 )
...
* [benchmark] support lite light api & optimize benchmark flags
* [backend] remove un-used option member
* [backend] remove un-used option member
2023-05-17 18:51:32 +08:00
linkk08
9ed8d18fcc
[XPU] Add gm_default_size -> Backend::LITE ( #1934 )
...
* add gm_default_size
* add gm_default_size
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-05-16 14:31:22 +08:00
unseenme
3fd21c935e
[Hackthon_4th 244] Added Paddle Lite GPU Backend ( #1907 )
...
* [improved] enum; ConfigureGpu();
* [improved] init()
* [improved] valid place; model dir; is valid;
* [added] WITH_OPENCL in cmake
* [improved] set model; valid place; cmake url; cmake option;
* Update runtime_option.cc
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-05-12 19:44:27 +08:00
seyosum
df8dd3e3ac
【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy ( #1822 )
...
* add horizon backend and PPYOLOE examples
* 更改horizon头文件编码规范
* 更改horizon头文件编码规范
* 更改horizon头文件编码规范
* 增加horizon packages下载及自动安装
* Add UseHorizonNPUBackend Method
* 删除编译FD SDK后多余的头文件,同时更改部分规范
* Update horizon.md
* Update horizon.md
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-05-06 16:10:37 +08:00
linyangshi
9164796645
[Model] Support DINO & DETR and add PaddleDetectionModel class ( #1837 )
...
* 添加paddleclas模型
* 更新README_CN
* 更新README_CN
* 更新README
* update get_model.sh
* update get_models.sh
* update paddleseg models
* update paddle_seg models
* update paddle_seg models
* modified test resources
* update benchmark_gpu_trt.sh
* add paddle detection
* add paddledetection to benchmark
* modified benchmark cmakelists
* update benchmark scripts
* modified benchmark function calling
* modified paddledetection documents
* add PaddleDetectonModel
* reset examples/paddledetection
* resolve conflict
* update pybind
* resolve conflict
* fix bug
* delete debug mode
* update checkarch log
* update trt inputs example
* Update README.md
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-05-05 14:10:33 +08:00
DefTruth
652024d2f6
Revert "Remove Paddle Reader" ( #1860 )
...
Revert "Remove Paddle Reader (#1813 )"
This reverts commit f3d44785c4 .
2023-04-23 23:16:31 +08:00
thunder95
51be3fea78
[Hackthon_4th 177] Support PP-YOLOE-R with BM1684 ( #1809 )
...
* first draft
* add robx iou
* add benchmark for ppyoloe_r
* remove trash code
* fix bugs
* add pybind nms rotated option
* add missing head file
* fix bug
* fix bug2
* fix shape bug
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-04-21 10:48:05 +08:00
Jason
f3d44785c4
Remove Paddle Reader ( #1813 )
...
* Remove Paddle Reader
* support pp-infer c++14
* disable trt cache
---------
Co-authored-by: wang-xinyu <wangxinyu_es@163.com >
2023-04-20 21:12:43 +08:00
DefTruth
326715569d
[Bug Fix] fixed paddle inference headers include policy ( #1827 )
...
* Update __init__.py
* [Python] Update python whl setup.py
* [cmake] support fd & paddle inference custom libs on win/linux
* [cmake] support fd & paddle inference custom libs on win/linux
* [Bug Fix] fixed paddle inference headers include policy
* [Bug Fix] fixed paddle inference headers include policy
* [Bug Fix] fixed paddle inference headers include policy
2023-04-19 14:32:51 +08:00
yeliang2258
a509dd8ec1
[Model] Add Paddle3D smoke model ( #1766 )
...
* add smoke model
* add 3d vis
* update code
* update doc
* mv paddle3d from detection to perception
* update result for velocity
* update code for CI
* add set input data for TRT backend
* add serving support for smoke model
* update code
* update code
* update code
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-04-14 16:30:56 +08:00
Zheng-Bicheng
b30f62af36
[Backend] Refactoring RKNPU2 Backend code ( #1772 )
...
* update rknpu2 runtime
* update rknpu2 runtime
* update rknpu2 runtime
* update for rknpu2 backend
* update for rknpu2 backend
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-04-13 16:37:36 +08:00
Yi-sir
9e20dab0d6
[Example] Merge Download Paddle Model, Paddle->ONNX->MLIR->BModel ( #1643 )
...
* fix infer.py and README
* [Example] Merge Download Paddle Model, Paddle->Onnx->Mlir->Bmodel and
inference into infer.py. Modify README.md
* modify pp_liteseg sophgo infer.py and README.md
* fix PPOCR,PPYOLOE,PICODET,LITESEG sophgo infer.py and README.md
* fix memory overflow problem while inferring with sophgo backend
* fix memory overflow problem while inferring with sophgo backend
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
Co-authored-by: xuyizhou <yizhou.xu@sophgo.com >
2023-03-31 15:08:01 +08:00
DefTruth
7d1b706caf
[Benchmark] Add pure runtime benchmark bin ( #1731 )
...
* [Benchmark] Add pure runtime benchmark bin
* [Benchmark] add cpu/gpu memory collect -> benchmark bin
* [Backend] update trt max_batch_size policy
* [backend] Update trt backend max_batch_size policy
* [Benchmark] Add more model format support -> benchmark bin
2023-03-30 19:03:30 +08:00
WJJ1995
409c5dc056
[Backend]Add default value for TRT SetShape() ( #1712 )
...
* add GPL lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* support yolov8
* add pybind for yolov8
* add yolov8 readme
* add cpp benchmark
* add cpu and gpu mem
* public part split
* add runtime mode
* fixed bugs
* add cpu_thread_nums
* deal with comments
* deal with comments
* deal with comments
* rm useless code
* add FASTDEPLOY_DECL
* add FASTDEPLOY_DECL
* fixed for windows
* mv rss to pss
* mv rss to pss
* Update utils.cc
* use thread to collect mem
* Add ResourceUsageMonitor
* rm useless code
* fixed bug
* fixed typo
* update ResourceUsageMonitor
* fixed bug
* fixed bug
* add note for ResourceUsageMonitor
* deal with comments
* add macros
* deal with comments
* deal with comments
* deal with comments
* re-lint
* rm pmap and use mem api
* rm pmap and use mem api
* add mem api
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* deal with comments
* fixed enable_paddle_to_trt
* add log for paddle_trt
* support ppcls benchmark
* use new trt option api
* update benchmark info
* simplify benchmark.cc
* simplify benchmark.cc
* deal with comments
* Add ppseg && ppocr benchmark
* add OCR rec img
* add ocr benchmark
* fixed trt shape
* add trt shape
* resolve conflict
* add ENABLE_BENCHMARK define
* Add ClassifyDiff
* Add Resize for ClassifyResult
* deal with comments
* add convert info script
* resolve conflict
* Add SaveBenchmarkResult func
* fixed bug
* fixed bug
* fixed bug
* add config.txt for option
* fixed bug
* fixed bug
* fixed bug
* add benchmark.sh
* mv thread_nums from 8 to 1
* deal with comments
* deal with comments
* fixed readme
* deal with comments
* add all platform shell
* Update config.arm.txt
* Update config.gpu.txt
* Update config.x86.txt
* fixed printinfo bug
* rm proxy
* add more model support
* all backend config.txt
* deal with comments
* Add MattingDiff compare
* fixed predict bug
* adjust warmup/repeat times
* add e2e/mem configs
* fixed typo
* open collect_mem
* fixed typo
* add trt cache option
* fixed bug
* fixed repeat times
* test for benchmark
* test for det benchmark
* for benchmark
* fixed for x86
* add h2d and d2h config
* renmae txt file
* add dynamic shape for pp_trt
* fixed typo
* Update option.h
* add collect shape
* add default value for SetShape()
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-03-27 14:03:47 +08:00
WJJ1995
5c70db176f
[Backend]Add switch_ir_debug for Paddle Backend ( #1700 )
...
* avoid mem copy for cpp benchmark
* set CMAKE_BUILD_TYPE to Release
* Add SegmentationDiff
* change pointer to reference
* fixed bug
* cast uint8 to int32
* Add diff compare for OCR
* Add diff compare for OCR
* rm ppocr pipeline
* Add yolov5 diff compare
* Add yolov5 diff compare
* deal with comments
* deal with comments
* fixed bug
* fixed bug
* fixed thread nums
* Add Failed log
* optimizer x86 pipeline
* Add switch_ir_debug for paddle backend
* fixed for ci
2023-03-24 17:29:31 +08:00
yeliang2258
cae341e6c5
[Backend] Add fp16 support for ONNXRuntime-GPU ( #1239 )
...
* add fp16 support for ort-gpu
* add enable_ort_fp16 option
* fix
* fix bugs
* use fp16 for test
* update code
* update code
* fix windows bug
2023-03-23 18:23:13 +08:00
yunyaoXYY
b3e16e9966
ORT support UINT8 and INT8 input and output
2023-03-21 02:23:23 +00:00
Jack Zhou
012c7771c1
[Serving] Add collect shape and fix serving infer ( #1658 )
...
Add collect shape and fix serving infer
2023-03-20 19:55:30 +08:00
DefTruth
6af3691568
[XPU] Update Lite XPU lib and fixed some bugs ( #1647 )
2023-03-17 21:39:39 +08:00
Jason
3b1343c726
[Bug] Fix loadding big model loadding problem ( #1636 )
...
Fix loadding big model loadding problem
2023-03-17 10:25:26 +08:00
Ethan Yang
4f48d84c27
[Backend] optimize cpu usage on openvino backend ( #1531 )
...
* optimiza cpu usage on openvino backend
conver the hint varible to string
* update with affinity mode set-up
---------
Co-authored-by: Ethan <yicheng@intel.com >
Co-authored-by: Jason <jiangjiajun@baidu.com >
2023-03-17 10:24:25 +08:00
zhoushunjie
b7fb3aff08
remoove fdinfo in BindOutputTensor
2023-03-14 12:53:37 +00:00
Jason
3d36e26b8a
[Backend] Remove some deprecated api ( #1574 )
...
Remove some deprecated api
2023-03-13 13:42:43 +08:00
Jack Zhou
f4736e7931
Merge pull request #1552 from joey12300/fix_delete_pass
...
[Backend] Fix delete pass of paddle inference
2023-03-08 19:57:08 +08:00
Jason
6be2c0367b
[Example] Update runtime examples ( #1542 )
...
* Add notes for tensors
* Optimize some apis
* move some warnings
2023-03-08 16:56:04 +08:00
zhoushunjie
384eca14fd
Fix delete pass
2023-03-08 06:32:27 +00:00
Zheng-Bicheng
0687d3b0ad
[Model] Add Solov2 For PaddleDetection ( #1435 )
...
* update solov2
* Repair note
* update solov2 postprocess
* update
* update solov2
* update solov2
* fixed bug
* fixed bug
* update solov2
* update solov2
* fix build android bug
* update docs
* update docs
* update docs
* update
* update
* update arch and docs
* update
* update
* update solov2 python
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-03-08 10:01:32 +08:00
Jason
7e4ef4a72c
Merge branch 'develop' into rknn_pybind
2023-03-07 18:53:53 +08:00
Jason
8ec1353cd8
[Doc] Add notes for tensors ( #1533 )
...
Add notes for tensors
2023-03-07 18:46:48 +08:00
Zheng-Bicheng
458523fc29
update pybind
2023-03-07 15:50:49 +08:00
Zheng-Bicheng
610b4c532b
[Bug Fix] fixed rknn bugs ( #1468 )
...
fixed rknn bugs
2023-03-01 10:21:58 +08:00
DefTruth
e86905ca0d
[Bug Fix] move enable_valid_backend_check option -> runtime_option ( #1450 )
...
* [Model] Add skip valid backend check option -> FastDeployModel
* [Model] Add skip valid backend check option -> FastDeployModel
* [Model] Add skip valid backend check option -> FastDeployModel
* [FastDeploy] move enable_valid_backend_check option -> runtime_option
2023-02-28 09:57:31 +08:00
Jack Zhou
524c85745b
[Backend] Add fixed size optimization for transformer model ( #1430 )
...
Add enable_fixed_size_opt flag
2023-02-24 09:45:04 +08:00
WJJ1995
2f8d9c9a57
[Benchmark]Add SegmentationDiff to compare SegmentationResult diff ( #1404 )
...
* avoid mem copy for cpp benchmark
* set CMAKE_BUILD_TYPE to Release
* Add SegmentationDiff
* change pointer to reference
* fixed bug
* cast uint8 to int32
2023-02-22 14:42:21 +08:00
Jason
dd527388bc
Update option.h
2023-02-22 11:28:04 +08:00
Jason
18e33bae5c
[Other] Optimize runtime module ( #1356 )
...
* Optimize runtime
* fix error
* [Backend] Add option to print tensorrt conversion log (#1386 )
Add option to print tensorrt conversion log
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com >
---------
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com >
2023-02-21 17:01:32 +08:00
WJJ1995
b6658b8094
[Benchmark] Add ppocr && ppseg benchmark ( #1344 )
...
* add GPL lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* support yolov8
* add pybind for yolov8
* add yolov8 readme
* add cpp benchmark
* add cpu and gpu mem
* public part split
* add runtime mode
* fixed bugs
* add cpu_thread_nums
* deal with comments
* deal with comments
* deal with comments
* rm useless code
* add FASTDEPLOY_DECL
* add FASTDEPLOY_DECL
* fixed for windows
* mv rss to pss
* mv rss to pss
* Update utils.cc
* use thread to collect mem
* Add ResourceUsageMonitor
* rm useless code
* fixed bug
* fixed typo
* update ResourceUsageMonitor
* fixed bug
* fixed bug
* add note for ResourceUsageMonitor
* deal with comments
* add macros
* deal with comments
* deal with comments
* deal with comments
* re-lint
* rm pmap and use mem api
* rm pmap and use mem api
* add mem api
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* deal with comments
* fixed enable_paddle_to_trt
* add log for paddle_trt
* support ppcls benchmark
* use new trt option api
* update benchmark info
* simplify benchmark.cc
* simplify benchmark.cc
* deal with comments
* Add ppseg && ppocr benchmark
* add OCR rec img
* add ocr benchmark
* fixed trt shape
* add trt shape
* resolve conflict
* add ENABLE_BENCHMARK define
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-02-20 15:59:27 +08:00
Jason
db471c3466
[Other] Optimize poros backend ( #1331 )
...
* Optimize poros backend
* Fix pybind error
---------
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com >
2023-02-17 14:32:35 +08:00
yunyaoXYY
c38b7d4377
[Backend] Support onnxruntime DirectML inference. ( #1304 )
...
* Fix links in readme
* Fix links in readme
* Update PPOCRv2/v3 examples
* Update auto compression configs
* Add neww quantization support for paddleclas model
* Update quantized Yolov6s model download link
* Improve PPOCR comments
* Add English doc for quantization
* Fix PPOCR rec model bug
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add new paddleseg quantization support
* Add Ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Add ascend model list
* Support DirectML in onnxruntime
* Support onnxruntime DirectML
* Support onnxruntime DirectML
* Support onnxruntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Support OnnxRuntime DirectML
* Remove DirectML vision model example
* Imporve OnnxRuntime DirectML
* Imporve OnnxRuntime DirectML
* fix opencv cmake in Windows
* recheck codestyle
2023-02-17 10:53:51 +08:00
Jason
7a4b2f3177
[Other] Optimize rknpu2 backend ( #1318 )
...
* Optimize rknpu2 backend
* Fix error
* Fix comparision error
2023-02-15 11:11:37 +08:00
WJJ1995
c25d1cc1bc
[Backend]Fixed enable_paddle_to_trt() bug ( #1320 )
...
* add GPL lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* add GPL-3.0 lisence
* support yolov8
* add pybind for yolov8
* add yolov8 readme
* add cpp benchmark
* add cpu and gpu mem
* public part split
* add runtime mode
* fixed bugs
* add cpu_thread_nums
* deal with comments
* deal with comments
* deal with comments
* rm useless code
* add FASTDEPLOY_DECL
* add FASTDEPLOY_DECL
* fixed for windows
* mv rss to pss
* mv rss to pss
* Update utils.cc
* use thread to collect mem
* Add ResourceUsageMonitor
* rm useless code
* fixed bug
* fixed typo
* update ResourceUsageMonitor
* fixed bug
* fixed bug
* add note for ResourceUsageMonitor
* deal with comments
* add macros
* deal with comments
* deal with comments
* deal with comments
* re-lint
* rm pmap and use mem api
* rm pmap and use mem api
* add mem api
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* Add PrintBenchmarkInfo func
* deal with comments
* fixed enable_paddle_to_trt
* add log for paddle_trt
---------
Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com >
2023-02-14 17:51:39 +08:00