afe8444782
* add ocr serving example * 1 1 * Add files via upload * Update README.md * Delete ocr_pipeline.png * Add files via upload * Delete ocr_pipeline.png * Add files via upload * 1 1 * 1 1 * Update README.md * 1 1 * fix codestyle * fix codestyle * Update README_CN.md * Update README_EN.md * Update demo.md * Update demo.md * Add files via upload * Update demo.md * Add files via upload * Delete dynamic_batching.png * Delete instance_group.png * Delete simple_ensemble.png * Add files via upload * Update demo.md * Update demo.md * Update demo.md * Update demo.md * Delete dynamic_batching.png * Delete instance_group.png * Delete simple_ensemble.png * Update demo.md Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: heliqi <1101791222@qq.com>
简体中文 | English
FastDeploy Serving Deployment
Introduction
FastDeploy builds an end-to-end serving deployment based on Triton Inference Server. The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance.
Prepare the environment
Environment requirements
- Linux
- If using a GPU image, NVIDIA Driver >= 470 is required (for older Tesla architecture GPUs, such as T4, the NVIDIA Driver can be 418.40+, 440.33+, 450.51+, 460.27+)
Obtain Image
CPU Image
CPU images only support Paddle/ONNX models for serving deployment on CPUs, and supported inference backends include OpenVINO, Paddle Inference, and ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10
GPU Image
GPU images support Paddle/ONNX models for serving deployment on GPU and CPU, and supported inference backends including OpenVINO, TensorRT, Paddle Inference, and ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
Users can also compile the image by themselves according to their own needs, referring to the following documents:
Other Tutorials
Serving Deployment Demo
| Task | Model |
|---|---|
| Classification | PaddleClas |
| Detection | PaddleDetection |
| Detection | ultralytics/YOLOv5 |
| NLP | PaddleNLP/ERNIE-3.0 |
| NLP | PaddleNLP/UIE |
| Speech | PaddleSpeech/PP-TTS |
| OCR | PaddleOCR/PP-OCRv3 |