apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

[Doc][Serving]modify serving doc (#718)

Browse Source 
modify serving doc
This commit is contained in:
heliqi
2022-11-28 15:14:10 +08:00
committed by GitHub
parent 03947d7b37
commit dd18471b41
10 changed files with 136 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
README_CN.md
View File

@@ -3,7 +3,7 @@
![⚡️FastDeploy](https://user-images.githubusercontent.com/31974251/185771818-5d4423cd-c94c-4a49-9894-bc7a8d1c29d0.png)
</p>
<p align="center">
<a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/FastDeploy?color=ffa"></a>
@@ -56,11 +56,11 @@
- [测试功能] 新增OpenVINO后端Device设置，支持集显/独立显卡的调用；
- **📲 移动端和端侧部署：支持更多模型**
- 新增Android图像分类、目标检测、语义分割、OCR、人脸检测 APK工程及示例.
|<font size=3>图像分类</font>|<font size=3>目标检测</font>|<font size=3>语义分割</font>|<font size=3>文字识别</font>|<font size=3>人脸检测</font>|
|:---:|:---:|:---:|:---:|:---:|
|<font size=2>[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification)</font>|<font size=2>[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/detection)</font>|<font size=2>[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/segmentation)|<font size=2>[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/ocr)</font>|<font size=2>[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/facedet)</font>|
|<font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)</font>|<font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)</font>| <font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)</font>| <font size=2> [扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)</font>|<font size=2> [扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)</font>|
|<font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)</font>|<font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)</font>| <font size=2>[扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)</font>| <font size=2> [扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)</font>|<font size=2> [扫码或点击链接<br>安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)</font>|
| <img src=https://user-images.githubusercontent.com/54695910/203604502-991972a8-5a9c-49cd-9e58-ed8e2a942b9b.png height="90" width="100"> |<img src=https://user-images.githubusercontent.com/54695910/203604475-724be708-27d6-4e56-9c2f-ae2eca24118c.png height="90" width="100">|<img src=https://user-images.githubusercontent.com/54695910/203604459-9a2915bc-91dc-460c-bff6-a0e2584d2eff.png height="90" width="100">|<img src=https://user-images.githubusercontent.com/54695910/203604453-6ce0118e-7b93-4044-8a92-56f2ab65c26a.png height="90" width="100">|<img src=https://user-images.githubusercontent.com/54695910/203604418-7c9703b5-1805-457e-966c-5a6625f212ff.png height="90" width="100">|
- [**more releases information**](./releases)
@@ -76,6 +76,7 @@
- [IPU部署环境编译安装](docs/cn/build_and_install/ipu.md)
- [Jetson部署环境编译安装](docs/cn/build_and_install/jetson.md)
- [Android平台部署环境编译安装](docs/cn/build_and_install/android.md)
- [服务化部署镜像安装](serving/README_CN.md)
- 快速使用
- [Python部署示例](docs/cn/quick_start/models/python.md)
- [C++部署示例](docs/cn/quick_start/models/cpp.md)
@@ -226,29 +227,29 @@ int main(int argc, char* argv[]) {
| 任务场景 | 模型 | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux |
|:----------------------:|:--------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------:|:-------:|:----------:|:-------:|:----------:|:-------:|:-------:|:-----------:|:-------------:|:-------------:|:-------:|
| --- | --- | --- | X86 CPU | NVIDIA GPU | X86 CPU | NVIDIA GPU | X86 CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving |
| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [TorchVison/ResNet](examples/vision/classification/resnet) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [ltralytics/YOLOv5Cls](examples/vision/classification/yolov5cls) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [Megvii-BaseDetection/YOLOX](./examples/vision/detection/yolox) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [WongKinYiu/YOLOv7](./examples/vision/detection/yolov7) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [WongKinYiu/YOLOv7end2end_trt](./examples/vision/detection/yolov7end2end_trt) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
@@ -259,10 +260,10 @@ int main(int argc, char* argv[]) {
| Detection | [WongKinYiu/ScaledYOLOv4](./examples/vision/detection/scaledyolov4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [ppogg/YOLOv5Lite](./examples/vision/detection/yolov5lite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [RangiLyu/NanoDetPlus](./examples/vision/detection/nanodet_plus) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| HeadPose | [omasaht/headpose](examples/vision/headpose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| OCR | [PaddleOCR/PP-OCRv2](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| OCR | [PaddleOCR/PP-OCRv3](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Segmentation | [PaddleSeg/PP-LiteSeg](./examples/vision/segmentation/paddleseg) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
@@ -290,7 +291,7 @@ int main(int argc, char* argv[]) {
| Video Super-Resolution | [PaddleGAN/BasicVSR](./) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Video Super-Resolution | [PaddleGAN/EDVR](./examples/vision/sr/edvr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Video Super-Resolution | [PaddleGAN/PP-MSVSR](./examples/vision/sr/ppmsvsr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| NLP | [PaddleNLP/ERNIE-3.0](./examples/text/ernie-3.0) | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
| Speech | [PaddleSpeech/PP-TTS](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -- | ✅ |
@@ -303,7 +304,7 @@ int main(int argc, char* argv[]) {
<div id="fastdeploy-edge-models"></div>
### 端侧模型支持列表
<details open><summary><b> 端侧模型支持列表（点击可收缩）</b></summary><div>
<div align="center">
@@ -354,7 +355,7 @@ int main(int argc, char* argv[]) {
## 🌐 Web和小程序部署
<div id="fastdeploy-web-models"></div>
<details open><summary><b> Web和小程序部署支持列表（点击可收缩）</b></summary><div>
| 任务场景 | 模型 | [web_demo](examples/application/js/web_demo) |
@@ -368,7 +369,7 @@ int main(int argc, char* argv[]) {
| OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
</div></details>
<div id="fastdeploy-community"></div>
## 社区交流

103
README_EN.md
View File

@@ -40,13 +40,13 @@ Including image classification, object detection, image segmentation, face detec
- **Community**
- **Slack**：Join our [Slack community](https://join.slack.com/t/fastdeployworkspace/shared_invite/zt-1hhvpb279-iw2pNPwrDaMBQ5OQhO3Siw) and chat with other community members about ideas.
- **WeChat**：Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group.
<div align="center">
<img src="https://user-images.githubusercontent.com/54695910/200145290-d5565d18-6707-4a0b-a9af-85fd36d35d13.jpg" width = "100" height = "100" />
</div>
- 🔥 **2022.11.23：Release FastDeploy [release v0.8.0](https://github.com/PaddlePaddle/FastDeploy/tree/release/0.8.0)** <br>
- **🖥️ Server-side and Cloud Deployment: Support more CV models, improve deployment performance**
- Support [PIPNet](./examples/vision/facealign/pipnet), [FaceLandmark1000](./examples/vision/facealign/face_landmark_1000) face alignment models deployment;
- Support [Video Super-Resolution](./examples/vision/sr) series model PP-MSVSR、EDVR、BasicVSR;
@@ -55,11 +55,11 @@ Including image classification, object detection, image segmentation, face detec
- Add Python API to_dlpack interface for FDTensor to support copyless transfer of FDTensor between frameworks.
- **📱 Mobile and Edge Device Deployment: support more CV model**
- Support Android image classification, target detection, semantic segmentation, OCR, face detection APK projects and examples.
|<font size=3>Image Classification</font>|<font size=3>Object Detection</font>|<font size=3>Semantic Segmentation</font>|<font size=3>OCR</font>|<font size=3>Face Detection</font>|
|:---:|:---:|:---:|:---:|:---:|
|<font size=2>[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification)</font>|<font size=2>[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/detection)</font>|<font size=2>[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/segmentation)|<font size=2>[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/ocr)</font>|<font size=2>[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/facedet)</font>|
|<font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)</font>|<font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)</font>| <font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)</font>| <font size=2> [Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)</font>|<font size=2> [Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)</font>|
|<font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)</font>|<font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)</font>| <font size=2>[Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)</font>| <font size=2> [Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)</font>|<font size=2> [Scan&nbsp;the&nbsp;code<br>or&nbsp;click&nbsp;on&nbsp;the&nbsp;link<br>to&nbsp;install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)</font>|
| <img src=https://user-images.githubusercontent.com/54695910/203604502-991972a8-5a9c-49cd-9e58-ed8e2a942b9b.png height="100" width="110"> |<img src=https://user-images.githubusercontent.com/54695910/203604475-724be708-27d6-4e56-9c2f-ae2eca24118c.png height="100" width="110">|<img src=https://user-images.githubusercontent.com/54695910/203604459-9a2915bc-91dc-460c-bff6-a0e2584d2eff.png height="100" width="110">|<img src=https://user-images.githubusercontent.com/54695910/203604453-6ce0118e-7b93-4044-8a92-56f2ab65c26a.png height="100" width="110">|<img src=https://user-images.githubusercontent.com/54695910/203604418-7c9703b5-1805-457e-966c-5a6625f212ff.png height="100" width="110">|
- [**more releases information**](./releases)
@@ -67,60 +67,61 @@ Including image classification, object detection, image segmentation, face detec
## Contents
* <details open><summary><b>📖 Tutorials（click to fold）</b></summary><div>
- Install
- [How to Install FastDeploy Prebuilt Libraries](docs/en/build_and_install/download_prebuilt_libraries.md)
- [How to Build and Install FastDeploy Library on GPU Platform](docs/en/build_and_install/gpu.md)
- [How to Build and Install FastDeploy Library on CPU Platform](docs/en/build_and_install/cpu.md)
- [How to Build and Install FastDeploy Library on IPU Platform](docs/en/build_and_install/ipu.md)
- [How to Build and Install FastDeploy Library on Nvidia Jetson Platform](docs/en/build_and_install/jetson.md)
- [How to Build and Install FastDeploy Library on Android Platform](docs/en/build_and_install/android.md)
- [How to Install FastDeploy Images on Serving Deployment](serving/README_EN.md)
- A Quick Start - Demos
- [Python Deployment Demo](docs/en/quick_start/models/python.md)
- [C++ Deployment Demo](docs/en/quick_start/models/cpp.md)
- [A Quick Start on Runtime Python](docs/en/quick_start/runtime/python.md)
- [A Quick Start on Runtime C++](docs/en/quick_start/runtime/cpp.md)
- API (To be continued)
- [Python API](https://baidu-paddle.github.io/fastdeploy-api/python/html/)
- [C++ API](https://baidu-paddle.github.io/fastdeploy-api/cpp/html/)
- Performance Optimization
- [Quantization Acceleration](docs/en/quantize.md)
- Frequent Q&As
- [1. How to Change Inference Backends](docs/en/faq/how_to_change_backend.md)
- [2. How to Use FastDeploy C++ SDK on Windows Platform](docs/en/faq/use_sdk_on_windows.md)
- [3. How to Use FastDeploy C++ SDK on Android Platform](docs/en/faq/use_sdk_on_android.md)(To be Continued)
- [4. Tricks of TensorRT](docs/en/faq/tensorrt_tricks.md)
- [5. How to Develop a New Model](docs/en/faq/develop_a_new_model.md)(To be Continued)
- More FastDeploy Deployment Module
- [deployment AI Model as a Service](./serving)
- [Benchmark Testing](./benchmark)
</div></details>
* **🖥️ Server-side and Cloud Deployment**
* [A Quick Start for Python SDK](#fastdeploy-quick-start-python)
* [A Quick Start for C++ SDK](#fastdeploy-quick-start-cpp)
* [Supported Server-side and Cloud Model List](#fastdeploy-server-models)
* **📱 Mobile and Edge Device Deployment**
* [Supported Mobile and Edge Model List](#fastdeploy-edge-models)
* **🌐 Browser and Mini Program Deployment**
* [Supported Web and Mini Program Model List](#fastdeploy-web-models)
* [**Community**](#fastdeploy-community)
@@ -252,29 +253,29 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
| Task | Model | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux |
|:----------------------:|:--------------------------------------------------------------------------------------------:|:------------------------------------------------:|:------------------------:|:------------------------:|:------------------------:|:-----------------------:|:---------------------:|:--------------------------:|:---------------------------:|:---------------------------:|:-------:|
| --- | --- | <font size=2> X86 CPU | <font size=2> NVIDIA GPU | <font size=2> Intel CPU | <font size=2> NVIDIA GPU | <font size=2> Intel CPU | <font size=2> Arm CPU | <font size=2> AArch64 CPU | <font size=2> NVIDIA Jetson | <font size=2> Graphcore IPU | Serving |
| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [TorchVison/ResNet](examples/vision/classification/resnet) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [ltralytics/YOLOv5Cls](examples/vision/classification/yolov5cls) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| Detection | [Megvii-BaseDetection/YOLOX](./examples/vision/detection/yolox) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [WongKinYiu/YOLOv7](./examples/vision/detection/yolov7) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [WongKinYiu/YOLOv7end2end_trt](./examples/vision/detection/yolov7end2end_trt) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
@@ -285,10 +286,10 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
| Detection | [WongKinYiu/ScaledYOLOv4](./examples/vision/detection/scaledyolov4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [ppogg/YOLOv5Lite](./examples/vision/detection/yolov5lite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Detection | [RangiLyu/NanoDetPlus](./examples/vision/detection/nanodet_plus) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| HeadPose | [omasaht/headpose](examples/vision/headpose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| OCR | [PaddleOCR/PP-OCRv2](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| OCR | [PaddleOCR/PP-OCRv3](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Segmentation | [PaddleSeg/PP-LiteSeg](./examples/vision/segmentation/paddleseg) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
@@ -316,12 +317,12 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
| Video Super-Resolution | [PaddleGAN/BasicVSR](./) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Video Super-Resolution | [PaddleGAN/EDVR](./examples/vision/sr/edvr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Video Super-Resolution | [PaddleGAN/PP-MSVSR](./examples/vision/sr/ppmsvsr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
| NLP | [PaddleNLP/ERNIE-3.0](./examples/text/ernie-3.0) | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
| Speech | [PaddleSpeech/PP-TTS](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -- | ✅ |
</div></details>
<div id="fastdeploy-edge-doc"></div>
## 📱 Mobile and Edge Device Deployment
@@ -329,7 +330,7 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
<div id="fastdeploy-edge-models"></div>
### Mobile and Edge Model List 🔥🔥🔥🔥
<details open><summary><b> Mobile and Edge Model List（click to fold）</b></summary><div>
<div align="center">
@@ -374,13 +375,13 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
| OCR | [PaddleOCR/PP-OCRv2](examples/vision/ocr/PP-OCRv2) | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
| OCR | [PaddleOCR/PP-OCRv3](examples/vision/ocr/PP-OCRv3) | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
| OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
</div></details>
## 🌐 Browser-based Model List
<div id="fastdeploy-web-models"></div>
<details open><summary><b> Browser-based Model List（click to fold）</b></summary><div>
| Task | Model | [web_demo](examples/application/js/web_demo) |
@@ -394,7 +395,7 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
| OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
</div></details>
## Community
<div id="fastdeploy-community"></div>

18
examples/text/ernie-3.0/serving/README.md
View File

@@ -1,5 +1,9 @@
# ERNIE 3.0 服务化部署示例
在服务化部署前，需确认
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
## 准备模型
下载ERNIE 3.0的新闻分类模型、序列标注模型(如果有已训练好的模型，跳过此步骤):
@@ -43,14 +47,14 @@ models
## 拉取并运行镜像
```bash
# CPU镜像, 仅支持Paddle/ONNX模型在CPU上进行服务化部署，支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.3.0-cpu-only-21.10
# GPU 镜像, 支持Paddle/ONNX模型在GPU/CPU上进行服务化部署，支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.3.0-gpu-cuda11.4-trt8.4-21.10
# x.y.z为镜像版本号，需参照serving文档替换为数字
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
# 运行
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.3.0-cpu-only-21.10 bash
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 bash
```
## 部署模型
@@ -63,7 +67,7 @@ token_cls_rpc_client.py # 序列标注任务发送pipeline预测请求的脚
```
*注意*:启动服务时，Server的每个python后端进程默认申请`64M`内存，默认启动的docker无法启动多个python后端节点。有两个解决方案：
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.3.0-gpu-cuda11.4-trt8.4-21.10 bash`
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash`
- 2.启动服务时设置python后端的`shm-default-byte-size`参数, 设置python后端的默认内存为10M： `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760`
### 分类任务

14
examples/text/uie/serving/README.md
View File

@@ -1,5 +1,9 @@
# UIE 服务化部署示例
在服务化部署前，需确认
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
## 准备模型
下载UIE-Base模型(如果有已训练好的模型，跳过此步骤):
@@ -26,11 +30,11 @@ models
## 拉取并运行镜像
```bash
# CPU镜像, 仅支持Paddle/ONNX模型在CPU上进行服务化部署，支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10
# GPU 镜像, 支持Paddle/ONNX模型在GPU/CPU上进行服务化部署，支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
# x.y.z为镜像版本号，需参照serving文档替换为数字
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /uie_serving 目录
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash

13
examples/vision/classification/paddleclas/serving/README.md
View File

@@ -1,5 +1,10 @@
# PaddleClas 服务化部署示例
在服务化部署前，需确认
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
## 启动服务
```bash
@@ -19,11 +24,11 @@ mv ResNet50_vd_infer/inference_cls.yaml models/preprocess/1/inference_cls.yaml
mv ResNet50_vd_infer/inference.pdmodel models/runtime/1/model.pdmodel
mv ResNet50_vd_infer/inference.pdiparams models/runtime/1/model.pdiparams
# 拉取fastdeploy镜像
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
# GPU镜像
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
@@ -33,7 +38,7 @@ CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --bac
```
>> **注意**:
>> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README.md)
>> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README_CN.md)
>> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改客户端示例中的请求端口号.

4
examples/vision/detection/paddledetection/serving/README.md
View File

@@ -6,7 +6,7 @@ PaddleDetection模型导出和预训练模型下载请看[PaddleDetection模型
在服务化部署前，需确认
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README.md)
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
## 启动服务
@@ -52,7 +52,7 @@ CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models
>> 由于mask_rcnn模型多一个输出，部署mask_rcnn需要将后处理目录(models/postprocess)中的mask_config.pbtxt重命名为config.pbtxt
>> 拉取镜像请看[服务化部署主文档](../../../../../serving/README.md)
>> 拉取镜像请看[服务化部署主文档](../../../../../serving/README_CN.md)
>> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定grpc端口号来启动服务，同时更改客户端示例中的请求端口号.

12
examples/vision/detection/yolov5/serving/README.md
View File

@@ -1,5 +1,10 @@
# YOLOv5 服务化部署示例
在服务化部署前，需确认
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
## 启动服务
```bash
@@ -13,8 +18,11 @@ wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov5s.onnx
# 将模型放入 models/runtime/1目录下, 并重命名为model.onnx
mv yolov5s.onnx models/runtime/1/model.onnx
# 拉取fastdeploy镜像
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字)
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /yolov5_serving 目录
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash

2
serving/Dockerfile
View File

@@ -20,7 +20,7 @@ COPY --from=full /opt/tritonserver/lib /opt/tritonserver/lib
COPY --from=full /opt/tritonserver/include /opt/tritonserver/include
COPY --from=full /opt/tritonserver/backends/python /opt/tritonserver/backends/python
COPY serving/TensorRT-8.4.1.5 /opt/
COPY serving/TensorRT-8.4.1.5 /opt/TensorRT-8.4.1.5
ENV TZ=Asia/Shanghai \
DEBIAN_FRONTEND=noninteractive \

10
serving/README_CN.md
View File

@@ -30,15 +30,17 @@ docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
- [FastDeploy服务化部署镜像编译说明](docs/zh_CN/compile.md)
## 其它文档
- [服务化模型目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型目录)
- [服务化部署配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项)
- [服务化部署示例](docs/zh_CN/demo.md)
- [模型仓库目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型仓库目录)
- [模型配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项)
### 模型示例列表
### 服务化部署示例
| 任务场景 | 模型 |
|---|---|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md)|
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md)|
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md)|

7
serving/README_EN.md
View File

@@ -39,14 +39,15 @@ Users can also compile the image by themselves according to their own needs, ref
- [How to Prepare Serving Model Repository](docs/zh_CN/model_repository.md)
- [Serving Deployment Configuration for Runtime](docs/zh_CN/model_configuration.md)
- [Serving Deployment Demo](docs/zh_CN/demo.md)
- [YOLOV5 - Detection Task](../examples/vision/detection/yolov5/serving/README.md)
### Model List
### Serving Deployment Demo
| Task | Model |
|---|---|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md)|
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md)|
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md)|