apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
2,449 Commits 29 Branches 10 Tags
577b7a7681801099e9eebfdf57f4b4ff5199e950
Commit Graph

172 Commits

Author SHA1 Message Date
Zheng-Bicheng
4e9bcc3718 [Backend] Update RKNN Runtime libs to version "1.5.1b19" (#2156) 
* update rknpu2 runtime libs and rknntoolkit2

* update rknpu2 runtime libs
 2023-08-10 13:27:56 +08:00
DefTruth
12bb44e0de [Bug Fix] fix build xpu encrypt & auth image scripts (#2133) 
* [patchelf] fix patchelf error for inference xpu

* [serving] add xpu dockerfile and support fd server

* [serving] add xpu dockerfile and support fd server

* [Serving] support XPU + Tritron

* [Serving] support XPU + Tritron

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] add comments for xpu tritron dockerfile

* [Doruntime] fix xpu infer error

* [Doruntime] fix xpu infer error

* [XPU] update xpu dockerfile

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* [XPU] Update XPU L3 Cache setting docs

* [XPU] Add Encryption and AUTH support for XPU Server

* [XPU] Add Encryption and AUTH support for XPU Server

* [Bug Fix] fix paddle reader error

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Triton] switch TAG 22.12 -> TAG 21.10wq

* update xpu auth server script

* [Bug Fix] fix build xpu encrypt & auth image scripts
 2023-07-24 21:00:05 +08:00
DefTruth
77ee48f9b8 [Bug Fix] fixed compile error for cpu building (#2105) 
* [cmake] upgrade windows paddle inference -> 2.5.0

* [cmake] upgrade windows paddle inference -> 2.5.0

* fix paddle custom ops bug on windows

* [Backend] refactor paddle custom ops

* [Bug Fix] fixed paddle custom ops windows build error

* [Bug Fix] fixed compile error for cpu building
 2023-07-13 17:02:25 +08:00
DefTruth
2542a75b61 [cmake] upgrade windows paddle inference -> 2.5.0 (#2100) 
* [cmake] upgrade windows paddle inference -> 2.5.0

* [cmake] upgrade windows paddle inference -> 2.5.0

* fix paddle custom ops bug on windows
 2023-07-12 18:39:06 +08:00
yeliang2258
ad1f46f7d9 Add ORT fp16 support in server (#2069) 
* add ort fp16 support in server

* update paddle2onnx url

* update ort fp16 api

* add disable_ort_fp16_op_types in serving
 2023-07-05 17:50:00 +08:00
DefTruth
b2426aefa9 [Backend] add paddle custom ops compatible policy (#2070) 
* Add centerpoint

* fix postprocess op file name

* [Backend] add paddle custom ops compatible policy

* [Backend] add paddle custom ops compatible policy

* [Backend] add paddle custom ops compatible policy

* upgrade linx paddle gpu -> 2.5

* add custom op compatible policy for paddle 2.5

* add custom op compatible policy for paddle 2.5

* add custom op compatible policy for paddle 2.5

* add collect_trt_shape_by_device option for paddle backend

* add collect_trt_shape_by_device option for paddle backend

* add custom op option for python build

* fix python build bugs

* update paddle linux x86 cpu only lib

* update paddle linux gpu lib

* update patchelf cmake

* fix paddle backend option pybind

* update paddle_inference.cmake

* add cuda sm_80 support (A100)

---------

Co-authored-by: zengshao0622 <peter_z96@163.com>
Co-authored-by: qiuyanjun <qiuyanjun@baidu.com>
 2023-06-29 22:32:14 +08:00
DefTruth
dea3795930 [Backend] Add backward compatible for paddle inference 2.4.x (#2062) 
* [Backend] Add backward compatiable for paddle infernence 2.4.x

* [Backend] Add backward compatiable for paddle infernence 2.4.x
 2023-06-25 19:27:36 +08:00
DefTruth
ff631b7838 [Server] Support encrypt & auth for FD Server (#2018) 
* [Server] Support GPU encrypt & auth for FD Triton Server

* fix dockerfile proxy env error

* update build scrpits

* remove some logs

---------

Co-authored-by: root <root@yq02-sys-rpm1206692e6.yq02.baidu.com>
Co-authored-by: qiuyanjun <qiuyanjun@baidu.com>
 2023-06-13 20:44:25 +08:00
Zheng-Bicheng
8d357814e8 [Backend] Add pybind & PaddleDetection example for TVM (#1998) 
* update

* update

* Update infer_ppyoloe_demo.cc

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-06-04 13:26:47 +08:00
DefTruth
284b1b46c8 [Serving] Support XPU encrypt & auth server (#2007) 
* [patchelf] fix patchelf error for inference xpu

* [serving] add xpu dockerfile and support fd server

* [serving] add xpu dockerfile and support fd server

* [Serving] support XPU + Tritron

* [Serving] support XPU + Tritron

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] update xpu tritron docker file -> paddle 0.0.0

* [Dockerfile] add comments for xpu tritron dockerfile

* [Doruntime] fix xpu infer error

* [Doruntime] fix xpu infer error

* [XPU] update xpu dockerfile

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* add xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* update xpu triton server docs

* [XPU] Update XPU L3 Cache setting docs

* [XPU] Add Encryption and AUTH support for XPU Server

* [XPU] Add Encryption and AUTH support for XPU Server

* [Bug Fix] fix paddle reader error

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Serving] Support XPU encrypt & auth server

* [Triton] switch TAG 22.12 -> TAG 21.10wq

* update xpu auth server script
 2023-06-01 21:36:44 +08:00
Zheng-Bicheng
643730bf5f [Hackathon 181] Add TVM support for FastDeploy on macOS (#1969) 
* update for tvm backend

* update third_party

* update third_party

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-25 19:59:02 +08:00
DefTruth
49c033a828 [XPU] Support XPU via Paddle Inference backend (#1987) 
* [backend] Support XPU via Paddle Inference backend

* [backend] Support XPU via Paddle Inference backend

* [backend] Support XPU via Paddle Inference backend

* [XPU] support XPU benchmark via paddle inference

* [XPU] support XPU benchmark via paddle inference

* [benchmark] add xpu paddle h2d config files
 2023-05-25 14:13:40 +08:00
DefTruth
f21f2b3dee [cmake] bump up paddle2onnx -> 1.0.7 (#1966) 
* Update paddle2onnx.cmake

* re-run CIs

* [Bug Fix] fix fd+paddle2.4.2 serving build script error
 2023-05-19 16:38:52 +08:00
DefTruth
b080b90f80 [python] support build paddle2onnx code for python & custom url (#1956) 
Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com>
 2023-05-18 11:40:07 +08:00
DefTruth
33e07410da [cmake] Support custom paddle inference url (#1939) 
* [cmake] Support custom paddle inference url

* [Python] Add custom Paddle Inference URL support for python

* [Docker] Add fd serving Dockerfile for paddle2.4.2

* [Docker] Add fd serving Dockerfile for paddle2.4.2

* [Docker] Add fd serving Dockerfile for paddle2.4.2

* [Docker] Add fd serving Dockerfile for paddle2.4.2

* [Bug Fix] fixed result format string error

* rerunning the re-touch CIs

* rerunning CIs
 2023-05-16 14:30:31 +08:00
unseenme
3fd21c935e [Hackthon_4th 244] Added Paddle Lite GPU Backend (#1907) 
* [improved] enum; ConfigureGpu();

* [improved] init()

* [improved] valid place; model dir; is valid;

* [added] WITH_OPENCL in cmake

* [improved] set model; valid place; cmake url; cmake option;

* Update runtime_option.cc

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-12 19:44:27 +08:00
DefTruth
e75e766b19 [Lite] Bump up Paddle Lite -> latest 0.0.0.ab000121e (#1925) 2023-05-10 14:22:57 +08:00
DefTruth
27a8c897ab [Lite] Support custom Paddle-Lite lib setting (#1906) 2023-05-08 14:05:43 +08:00
DefTruth
77cb9db6da [Model] Support PP-ShiTuV2 models for PaddleClas (#1900) 
* [cmake] add faiss.cmake -> pp-shituv2

* [PP-ShiTuV2] Support PP-ShituV2-Det model

* [PP-ShiTuV2] Support PP-ShiTuV2-Det model

* [PP-ShiTuV2] Add PPShiTuV2Recognizer c++&python support

* [PP-ShiTuV2] Add PPShiTuV2Recognizer c++&python support

* [Bug Fix] fix ppshitu_pybind error

* [benchmark] Add ppshituv2-det c++ benchmark

* [examples] Add PP-ShiTuV2 det & rec examples

* [vision] Update vision classification result

* [Bug Fix] fix trt shapes setting errors
 2023-05-08 14:04:09 +08:00
seyosum
df8dd3e3ac 【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy (#1822) 
* add horizon backend and PPYOLOE examples

* 更改horizon头文件编码规范

* 更改horizon头文件编码规范

* 更改horizon头文件编码规范

* 增加horizon packages下载及自动安装

* Add UseHorizonNPUBackend Method

* 删除编译FD SDK后多余的头文件,同时更改部分规范

* Update horizon.md

* Update horizon.md

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-05-06 16:10:37 +08:00
DefTruth
ef576ce875 [Paddle2ONNX] switch paddle2onnx default policy (#1873) 2023-04-26 10:32:18 +08:00
DefTruth
6e5f5f6802 [cmake] update paddle inference osx lib (#1859) 
* [cmake] update paddle inference osx lib

* Update __init__.py

* Update __init__.py

* Update paddle_inference.cmake

* Update __init__.py
 2023-04-24 14:13:15 +08:00
DefTruth
652024d2f6 Revert "Remove Paddle Reader" (#1860) 
Revert "Remove Paddle Reader (#1813)"

This reverts commit f3d44785c4.
 2023-04-23 23:16:31 +08:00
Jason
f3d44785c4 Remove Paddle Reader (#1813) 
* Remove Paddle Reader

* support pp-infer c++14

* disable trt cache

---------

Co-authored-by: wang-xinyu <wangxinyu_es@163.com>
 2023-04-20 21:12:43 +08:00
DefTruth
326715569d [Bug Fix] fixed paddle inference headers include policy (#1827) 
* Update __init__.py

* [Python] Update python whl setup.py

* [cmake] support fd & paddle inference custom libs on win/linux

* [cmake] support fd & paddle inference custom libs on win/linux

* [Bug Fix] fixed paddle inference headers include policy

* [Bug Fix] fixed paddle inference headers include policy

* [Bug Fix] fixed paddle inference headers include policy
 2023-04-19 14:32:51 +08:00
yeliang2258
a509dd8ec1 [Model] Add Paddle3D smoke model (#1766) 
* add smoke model

* add 3d vis

* update code

* update doc

* mv paddle3d from detection to perception

* update result for velocity

* update code for CI

* add set input data for TRT backend

* add serving support for smoke model

* update code

* update code

* update code

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-14 16:30:56 +08:00
yeliang2258
81fbd54c9d [Other] Add tests for TIMVX (#1605) 
* add tests for timvx

* add mobilenetv1 test

* update code

* fix log info

* update log

* fix test

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-14 10:31:36 +08:00
DefTruth
d1da460230 [Bug Fix] fix fastdeploy & paddle import conflict on windows (#1796) 
Update paddle_inference.cmake
 2023-04-12 09:32:08 +08:00
DefTruth
9ec419129f [XPU] Update Lite XPU lib to fix l3 cache crash for seg models (#1786) 
* Update kunlunxin.cmake

* [XPU] Update Lite XPU lib to fix l3 cache crash
 2023-04-11 18:01:52 +08:00
yeliang2258
e2f5a9ce66 [Model] Add picodet for RV1126 and A311D (#1549) 
* add infer for picodet

* update code

* update lite lib

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-10 22:04:45 +08:00
DefTruth
c90aa7bd6f [Bug Fix] fixed fastdeploy & paddle_inference import conflict (#1780) 
* [cmake] Update paddle_inference.cmake

* Update paddle_inference.cmake

* Update paddle_inference.cmake
 2023-04-08 19:25:44 +08:00
yeliang2258
7b0c55d0c5 [Backend] Add KunlunXin on aarch64 platform (#1701) 
* add xpu in aarch64

* reverse

* fix lib url

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-04-08 14:28:33 +08:00
DefTruth
df11896820 [cmake] Update paddle inference linux -> 2.4 dev8 (#1767) 2023-04-04 22:07:56 +08:00
guxukai
631f94d276 [CVCUDA] Update CV-CUDA to v0.2.1, add vision processor C++ tutorial (#1678) 
* update cvcuda 0.2.0 -> 0.2.1

* add cpp tutorials demo

* fix reviewed problem
 2023-03-24 16:57:35 +08:00
yeliang2258
cae341e6c5 [Backend] Add fp16 support for ONNXRuntime-GPU (#1239) 
* add fp16 support for ort-gpu

* add enable_ort_fp16 option

* fix

* fix bugs

* use fp16 for test

* update code

* update code

* fix windows bug
 2023-03-23 18:23:13 +08:00
DefTruth
2ff7b10adc [cmake] switch paddle 2.4-dev7 -> 2.4-dev6 for windows (#1660) 
* [cmake] switch paddle 2.4-dev7 -> 2.4-dev6 for windows

* [cmake] remove examples from install policy

* Update CMakeLists.txt
 2023-03-21 10:41:34 +08:00
DefTruth
6af3691568 [XPU] Update Lite XPU lib and fixed some bugs (#1647) 2023-03-17 21:39:39 +08:00
WJJ1995
1b37cb0c03 [Benchmark]Add End2End and meminfo configs (#1645) 
* add GPL lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* add GPL-3.0 lisence

* support yolov8

* add pybind for yolov8

* add yolov8 readme

* add cpp benchmark

* add cpu and gpu mem

* public part split

* add runtime mode

* fixed bugs

* add cpu_thread_nums

* deal with comments

* deal with comments

* deal with comments

* rm useless code

* add FASTDEPLOY_DECL

* add FASTDEPLOY_DECL

* fixed for windows

* mv rss to pss

* mv rss to pss

* Update utils.cc

* use thread to collect mem

* Add ResourceUsageMonitor

* rm useless code

* fixed bug

* fixed typo

* update ResourceUsageMonitor

* fixed bug

* fixed bug

* add note for ResourceUsageMonitor

* deal with comments

* add macros

* deal with comments

* deal with comments

* deal with comments

* re-lint

* rm pmap and use mem api

* rm pmap and use mem api

* add mem api

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* Add PrintBenchmarkInfo func

* deal with comments

* fixed enable_paddle_to_trt

* add log for paddle_trt

* support ppcls benchmark

* use new trt option api

* update benchmark info

* simplify benchmark.cc

* simplify benchmark.cc

* deal with comments

* Add ppseg && ppocr benchmark

* add OCR rec img

* add ocr benchmark

* fixed trt shape

* add trt shape

* resolve conflict

* add ENABLE_BENCHMARK define

* Add ClassifyDiff

* Add Resize for ClassifyResult

* deal with comments

* add convert info script

* resolve conflict

* Add SaveBenchmarkResult func

* fixed bug

* fixed bug

* fixed bug

* add config.txt for option

* fixed bug

* fixed bug

* fixed bug

* add benchmark.sh

* mv thread_nums from 8 to 1

* deal with comments

* deal with comments

* fixed readme

* deal with comments

* add all platform shell

* Update config.arm.txt

* Update config.gpu.txt

* Update config.x86.txt

* fixed printinfo bug

* rm proxy

* add more model support

* all backend config.txt

* deal with comments

* Add MattingDiff compare

* fixed predict bug

* adjust warmup/repeat times

* add e2e/mem configs

* fixed typo

---------

Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
 2023-03-17 17:30:47 +08:00
Ethan Yang
4f48d84c27 [Backend] optimize cpu usage on openvino backend (#1531) 
* optimiza cpu usage on openvino backend

conver the hint varible to string

* update with affinity mode set-up

---------

Co-authored-by: Ethan <yicheng@intel.com>
Co-authored-by: Jason <jiangjiajun@baidu.com>
 2023-03-17 10:24:25 +08:00
Jason
6343b0db47 [Build] Support build with source code of Paddle2ONNX (#1559) 
* Add notes for tensors

* Optimize some apis

* move some warnings

* Support build with Paddle2ONNX

* Add protobuf support

* Fix compile on mac

* add clearn package script

* Add paddle2onnx code

* remove submodule

* Add onnx ocde

* remove softlink

* add onnx code

* fix error

* Add cmake file

* fix patchelf

* update paddle2onnx

* Delete .gitmodules

---------

Co-authored-by: PaddleCI <paddle_ci@example.com>
Co-authored-by: pangyoki <pangyoki@126.com>
Co-authored-by: jiangjiajun <jiangjiajun@baidu.lcom>
 2023-03-17 10:03:22 +08:00
zhoushunjie
8489e22b4b update paddle inference to 2.4-dev7 2023-03-14 09:56:16 +00:00
Jack Zhou
f7ef464412 [Other] Use symbolic link pointed to TensorRT library (#1461) 
* Add force rpath

* update to dev-6

* remove force-rpath

* remove nvinfer so

* Add symbol link creation when import fastdeploy first

* Add dst

* update symbol link creation

* Add logging

* logging -> logger

* update tips

* Set logging level

* Add --force-rpath
 2023-03-10 09:09:04 +08:00
Jason
bf2ba72f1b [Compile] Support build FastDeploy with GLIBCXX_ABI=0 (#1528) 
* Support abi=0 build

* trigger ci

* fix cmake
 2023-03-07 09:45:30 +08:00
DefTruth
767647d88e [Android] Add WITH_TENSOR_FUNCS option -> Android (#1512) 
* [Android] Add WITH_TENSOR_FUNCS option -> Android

* [Android] move android cmake codes -> android.cmake

* [Android] move android cmake codes -> android.cmake
 2023-03-06 13:33:43 +08:00
DefTruth
40cec53c27 [XPU] Update Lite XPU lib & add l3 cache benchmark option (#1503) 
* [XPU] Update Lite XPU lib & add l3 cache benchmark option

* Update build_linux_x86_64_cpp_xpu_with_benchmark.sh

* Update kunlunxin.cmake

* Update build_linux_x86_64_cpp_xpu_with_benchmark.sh
 2023-03-06 11:48:15 +08:00
Zheng-Bicheng
8c3ccc2cc2 [Hackathon 182 Model] Update PPOCRV3 For RKNPU2 (#1403) 
* update ppocrv3 for rknpu2

* add config

* add config

* detele unuseful

* update useful results

* Repair note

* Repair note

* fixed bugs

* update
 2023-02-27 15:01:17 +08:00
DefTruth
95f809c1ff [Bug Fix] fix build error while enable flycv on linux x64 (#1365) 
* [Bug Fix] fix build error while enable flycv on linux x64

* [Bug Fix] fix build error while enable flycv on linux x64

* move WITH_FLYCV_STATIC option -> flycv.cmake
 2023-02-20 13:52:31 +08:00
yunyaoXYY
c38b7d4377 [Backend] Support onnxruntime DirectML inference. (#1304) 
* Fix links in readme

* Fix links in readme

* Update PPOCRv2/v3 examples

* Update auto compression configs

* Add neww quantization  support for paddleclas model

* Update quantized Yolov6s model download link

* Improve PPOCR comments

* Add English doc for quantization

* Fix PPOCR rec model bug

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add  new paddleseg quantization support

* Add Ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Add ascend model list

* Support DirectML in onnxruntime

* Support onnxruntime DirectML

* Support onnxruntime DirectML

* Support onnxruntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Support OnnxRuntime DirectML

* Remove DirectML vision model example

* Imporve OnnxRuntime DirectML

* Imporve OnnxRuntime DirectML

* fix opencv cmake in Windows

* recheck codestyle
 2023-02-17 10:53:51 +08:00
DefTruth
ee85a3cade [Benchmark] Add precision evaluation api from benchmark (#1310) 
* [Benchmark] Init benchmark precision api

* [Benchmark] Init benchmark precision api

* [Benchmark] Add benchmark precision api

* [Benchmark] Calculate the statis of diff

* [Benchmark] Calculate the statis of diff

* [Benchmark] Calculate the statis of diff

* [Benchmark] Calculate the statis of diff

* [Benchmark] Calculate the statis of diff

* [Benchmark] Add SplitDataLine utils

* [Benchmark] Add LexSortByXY func

* [Benchmark] Add LexSortByXY func

* [Benchmark] Add LexSortDetectionResultByXY func

* [Benchmark] Add LexSortDetectionResultByXY func

* [Benchmark] Add tensor diff presicion test

* [Benchmark] fixed conflicts

* [Benchmark] fixed calc tensor diff

* fixed build bugs

* fixed ci bugs when WITH_TESTING=ON
 2023-02-16 17:16:14 +08:00
Jack Zhou
e63f5f369e [Backend] Update paddle inference to 2.4-dev5 (#1302) 
update paddle inference to 2.4-dev5
 2023-02-12 15:13:39 +08:00