apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-18 06:31:17 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
Files
43f22f4bf63de74ed26f621ee289d827e4c34ec1
FastDeploy/serving
History
HCQ14 61c2f87e0c [Doc] Update English version of some documents (#1084) 
* Create README_CN.md

* Update README.md

* Update README_CN.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Create README_CN.md

* Update README.md

* Update README.md

* Update README_CN.md

* Create README_CN.md

* Update README.md

* Update README.md

* Update and rename README_en.md to README_CN.md

* Update WebDemo.md

* Update and rename WebDemo_en.md to WebDemo_CN.md

* Update and rename DEVELOPMENT_cn.md to DEVELOPMENT_CN.md

* Update DEVELOPMENT_CN.md

* Update DEVELOPMENT.md

* Update RNN.md

* Update and rename RNN_EN.md to RNN_CN.md

* Update README.md

* Update and rename README_en.md to README_CN.md

* Update README.md

* Update and rename README_en.md to README_CN.md

* Update README.md

* Update README_cn.md

* Rename README_cn.md to README_CN.md

* Update README.md

* Update README_cn.md

* Rename README_cn.md to README_CN.md

* Update export.md

* Update and rename export_EN.md to export_CN.md

* Update README.md

* Update README.md

* Create README_CN.md

* Update README.md

* Update README.md

* Update kunlunxin.md

* Update classification_result.md

* Update classification_result_EN.md

* Rename classification_result_EN.md to classification_result_CN.md

* Update detection_result.md

* Update and rename detection_result_EN.md to detection_result_CN.md

* Update face_alignment_result.md

* Update and rename face_alignment_result_EN.md to face_alignment_result_CN.md

* Update face_detection_result.md

* Update and rename face_detection_result_EN.md to face_detection_result_CN.md

* Update face_recognition_result.md

* Update and rename face_recognition_result_EN.md to face_recognition_result_CN.md

* Update headpose_result.md

* Update and rename headpose_result_EN.md to headpose_result_CN.md

* Update keypointdetection_result.md

* Update and rename keypointdetection_result_EN.md to keypointdetection_result_CN.md

* Update matting_result.md

* Update and rename matting_result_EN.md to matting_result_CN.md

* Update mot_result.md

* Update and rename mot_result_EN.md to mot_result_CN.md

* Update ocr_result.md

* Update and rename ocr_result_EN.md to ocr_result_CN.md

* Update segmentation_result.md

* Update and rename segmentation_result_EN.md to segmentation_result_CN.md

* Update README.md

* Update README.md

* Update quantize.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md
2023-01-06 18:01:34 +08:00
..
docs
[Doc]Revise one wording (#1028)
2023-01-03 10:01:10 +08:00
scripts
Fix serving/scripts build.sh (#1053)
2023-01-04 17:42:51 +08:00
src
[Serving][Backend] Backend support zero_copy_infer and Serving reduce the output memory copy (#703)
2022-11-28 14:07:53 +08:00
CMakeLists.txt
support build cpu images (#341)
2022-10-11 14:17:27 +08:00
Dockerfile
fix serving/Dockerfile (#1082)
2023-01-06 16:38:17 +08:00
Dockerfile_cpu
[Serving]modify docker images name (#992)
2022-12-27 21:29:27 +08:00
Dockerfile_ipu
[Serving]: add ipu support for serving. (#10) (#470)
2022-11-02 09:50:58 +08:00
README_CN.md
fresh doc version release/1.0.2 (#1023)
2023-01-03 09:37:53 +08:00
README.md
[Doc] Update English version of some documents (#1084)
2023-01-06 18:01:34 +08:00

README.md

简体中文 | English

FastDeploy Serving Deployment

Introduction

FastDeploy builds an end-to-end serving deployment based on Triton Inference Server. The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance.

Prepare the environment

Environment requirements

  • Linux
  • If using a GPU image, NVIDIA Driver >= 470 is required (for older Tesla architecture GPUs, such as T4, the NVIDIA Driver can be 418.40+, 440.33+, 450.51+, 460.27+)

Obtain Image

CPU Image

CPU images only support Paddle/ONNX models for serving deployment on CPUs, and supported inference backends include OpenVINO, Paddle Inference, and ONNX Runtime

docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.2-cpu-only-21.10

GPU Image

GPU images support Paddle/ONNX models for serving deployment on GPU and CPU, and supported inference backends including OpenVINO, TensorRT, Paddle Inference, and ONNX Runtime

docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.2-gpu-cuda11.4-trt8.4-21.10

Users can also compile the image by themselves according to their own needs, referring to the following documents:

Other Tutorials

Serving Deployment Demo

Task Model
Classification PaddleClas
Detection PaddleDetection
Detection ultralytics/YOLOv5
NLP PaddleNLP/ERNIE-3.0
NLP PaddleNLP/UIE
Speech PaddleSpeech/PP-TTS
OCR PaddleOCR/PP-OCRv3