From dd18471b4168beadc2f643bc3e684d3433aaf754 Mon Sep 17 00:00:00 2001 From: heliqi <1101791222@qq.com> Date: Mon, 28 Nov 2022 15:14:10 +0800 Subject: [PATCH] [Doc][Serving]modify serving doc (#718) modify serving doc --- README_CN.md | 63 +++++------ README_EN.md | 103 +++++++++--------- examples/text/ernie-3.0/serving/README.md | 18 +-- examples/text/uie/serving/README.md | 14 ++- .../paddleclas/serving/README.md | 13 ++- .../paddledetection/serving/README.md | 4 +- .../vision/detection/yolov5/serving/README.md | 12 +- serving/Dockerfile | 2 +- serving/README_CN.md | 10 +- serving/README_EN.md | 7 +- 10 files changed, 136 insertions(+), 110 deletions(-) diff --git a/README_CN.md b/README_CN.md index 82856f8f9..3f949c27c 100644 --- a/README_CN.md +++ b/README_CN.md @@ -3,7 +3,7 @@ ![⚡️FastDeploy](https://user-images.githubusercontent.com/31974251/185771818-5d4423cd-c94c-4a49-9894-bc7a8d1c29d0.png)

- +

@@ -56,11 +56,11 @@ - [测试功能] 新增OpenVINO后端Device设置，支持集显/独立显卡的调用； - **📲 移动端和端侧部署：支持更多模型** - 新增Android图像分类、目标检测、语义分割、OCR、人脸检测 APK工程及示例. - + |图像分类|目标检测|语义分割|文字识别|人脸检测| |:---:|:---:|:---:|:---:|:---:| |[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification)|[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/detection)|[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/segmentation)|[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/ocr)|[工程代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/facedet)| - |[扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)|[扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)| + |[扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)|[扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)| [扫码或点击链接
安装试用](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)| | ||||| - [**more releases information**](./releases) @@ -76,6 +76,7 @@ - [IPU部署环境编译安装](docs/cn/build_and_install/ipu.md) - [Jetson部署环境编译安装](docs/cn/build_and_install/jetson.md) - [Android平台部署环境编译安装](docs/cn/build_and_install/android.md) + - [服务化部署镜像安装](serving/README_CN.md) - 快速使用 - [Python部署示例](docs/cn/quick_start/models/python.md) - [C++部署示例](docs/cn/quick_start/models/cpp.md) @@ -226,29 +227,29 @@ int main(int argc, char* argv[]) { | 任务场景 | 模型 | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux | |:----------------------:|:--------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------:|:-------:|:----------:|:-------:|:----------:|:-------:|:-------:|:-----------:|:-------------:|:-------------:|:-------:| | --- | --- | --- | X86 CPU | NVIDIA GPU | X86 CPU | NVIDIA GPU | X86 CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving | -| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | +| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Classification | [TorchVison/ResNet](examples/vision/classification/resnet) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Classification | [ltralytics/YOLOv5Cls](examples/vision/classification/yolov5cls) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | Detection | [Megvii-BaseDetection/YOLOX](./examples/vision/detection/yolox) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [WongKinYiu/YOLOv7](./examples/vision/detection/yolov7) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [WongKinYiu/YOLOv7end2end_trt](./examples/vision/detection/yolov7end2end_trt) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | @@ -259,10 +260,10 @@ int main(int argc, char* argv[]) { | Detection | [WongKinYiu/ScaledYOLOv4](./examples/vision/detection/scaledyolov4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [ppogg/YOLOv5Lite](./examples/vision/detection/yolov5lite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [RangiLyu/NanoDetPlus](./examples/vision/detection/nanodet_plus) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | HeadPose | [omasaht/headpose](examples/vision/headpose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | OCR | [PaddleOCR/PP-OCRv2](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | OCR | [PaddleOCR/PP-OCRv3](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Segmentation | [PaddleSeg/PP-LiteSeg](./examples/vision/segmentation/paddleseg) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | @@ -290,7 +291,7 @@ int main(int argc, char* argv[]) { | Video Super-Resolution | [PaddleGAN/BasicVSR](./) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Video Super-Resolution | [PaddleGAN/EDVR](./examples/vision/sr/edvr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Video Super-Resolution | [PaddleGAN/PP-MSVSR](./examples/vision/sr/ppmsvsr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | NLP | [PaddleNLP/ERNIE-3.0](./examples/text/ernie-3.0) | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ | | Speech | [PaddleSpeech/PP-TTS](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -- | ✅ | @@ -303,7 +304,7 @@ int main(int argc, char* argv[]) {

### 端侧模型支持列表 - +
端侧模型支持列表（点击可收缩）
@@ -354,7 +355,7 @@ int main(int argc, char* argv[]) { ## 🌐 Web和小程序部署
- +
Web和小程序部署支持列表（点击可收缩）
| 任务场景 | 模型 | [web_demo](examples/application/js/web_demo) | @@ -368,7 +369,7 @@ int main(int argc, char* argv[]) { | OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
- +
## 社区交流 diff --git a/README_EN.md b/README_EN.md index ad780d66c..fdd18ac07 100644 --- a/README_EN.md +++ b/README_EN.md @@ -40,13 +40,13 @@ Including image classification, object detection, image segmentation, face detec - **Community** - **Slack**：Join our [Slack community](https://join.slack.com/t/fastdeployworkspace/shared_invite/zt-1hhvpb279-iw2pNPwrDaMBQ5OQhO3Siw) and chat with other community members about ideas. - **WeChat**：Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group. - +
- 🔥 **2022.11.23：Release FastDeploy [release v0.8.0](https://github.com/PaddlePaddle/FastDeploy/tree/release/0.8.0)**
- + - **🖥️ Server-side and Cloud Deployment: Support more CV models, improve deployment performance** - Support [PIPNet](./examples/vision/facealign/pipnet), [FaceLandmark1000](./examples/vision/facealign/face_landmark_1000) face alignment models deployment; - Support [Video Super-Resolution](./examples/vision/sr) series model PP-MSVSR、EDVR、BasicVSR; @@ -55,11 +55,11 @@ Including image classification, object detection, image segmentation, face detec - Add Python API to_dlpack interface for FDTensor to support copyless transfer of FDTensor between frameworks. - **📱 Mobile and Edge Device Deployment: support more CV model** - Support Android image classification, target detection, semantic segmentation, OCR, face detection APK projects and examples. - + |Image Classification|Object Detection|Semantic Segmentation|OCR|Face Detection| |:---:|:---:|:---:|:---:|:---:| |[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/classification)|[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/detection)|[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/segmentation)|[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/ocr)|[Project Code](https://github.com/PaddlePaddle/FastDeploy/tree/develop/java/android/app/src/main/java/com/baidu/paddle/fastdeploy/app/examples/facedet)| - |[Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)|[Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)| + |[Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-MobileNetV1.apk)|[Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PicoDet.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-Portrait-HumanSegV2-Lite.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-PP-OCRv2.apk)| [Scan the code
or click on the link
to install](https://bj.bcebos.com/fastdeploy/release/android/FastDeploy-SCRFD.apk)| | ||||| - [**more releases information**](./releases) @@ -67,60 +67,61 @@ Including image classification, object detection, image segmentation, face detec ## Contents *
📖 Tutorials（click to fold）
- + - Install - + - [How to Install FastDeploy Prebuilt Libraries](docs/en/build_and_install/download_prebuilt_libraries.md) - [How to Build and Install FastDeploy Library on GPU Platform](docs/en/build_and_install/gpu.md) - [How to Build and Install FastDeploy Library on CPU Platform](docs/en/build_and_install/cpu.md) - [How to Build and Install FastDeploy Library on IPU Platform](docs/en/build_and_install/ipu.md) - [How to Build and Install FastDeploy Library on Nvidia Jetson Platform](docs/en/build_and_install/jetson.md) - [How to Build and Install FastDeploy Library on Android Platform](docs/en/build_and_install/android.md) - + - [How to Install FastDeploy Images on Serving Deployment](serving/README_EN.md) + - A Quick Start - Demos - + - [Python Deployment Demo](docs/en/quick_start/models/python.md) - [C++ Deployment Demo](docs/en/quick_start/models/cpp.md) - [A Quick Start on Runtime Python](docs/en/quick_start/runtime/python.md) - [A Quick Start on Runtime C++](docs/en/quick_start/runtime/cpp.md) - + - API (To be continued) - + - [Python API](https://baidu-paddle.github.io/fastdeploy-api/python/html/) - [C++ API](https://baidu-paddle.github.io/fastdeploy-api/cpp/html/) - + - Performance Optimization - + - [Quantization Acceleration](docs/en/quantize.md) - + - Frequent Q&As - + - [1. How to Change Inference Backends](docs/en/faq/how_to_change_backend.md) - [2. How to Use FastDeploy C++ SDK on Windows Platform](docs/en/faq/use_sdk_on_windows.md) - [3. How to Use FastDeploy C++ SDK on Android Platform](docs/en/faq/use_sdk_on_android.md)(To be Continued) - [4. Tricks of TensorRT](docs/en/faq/tensorrt_tricks.md) - [5. How to Develop a New Model](docs/en/faq/develop_a_new_model.md)(To be Continued) - + - More FastDeploy Deployment Module - + - [deployment AI Model as a Service](./serving) - + - [Benchmark Testing](./benchmark) - +
* **🖥️ Server-side and Cloud Deployment** - + * [A Quick Start for Python SDK](#fastdeploy-quick-start-python) * [A Quick Start for C++ SDK](#fastdeploy-quick-start-cpp) * [Supported Server-side and Cloud Model List](#fastdeploy-server-models) * **📱 Mobile and Edge Device Deployment** - + * [Supported Mobile and Edge Model List](#fastdeploy-edge-models) * **🌐 Browser and Mini Program Deployment** - + * [Supported Web and Mini Program Model List](#fastdeploy-web-models) * [**Community**](#fastdeploy-community) @@ -252,29 +253,29 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava | Task | Model | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux | |:----------------------:|:--------------------------------------------------------------------------------------------:|:------------------------------------------------:|:------------------------:|:------------------------:|:------------------------:|:-----------------------:|:---------------------:|:--------------------------:|:---------------------------:|:---------------------------:|:-------:| | --- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving | -| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | +| Classification | [PaddleClas/ResNet50](./examples/vision/classification/paddleclas) | [✅](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Classification | [TorchVison/ResNet](examples/vision/classification/resnet) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Classification | [ltralytics/YOLOv5Cls](examples/vision/classification/yolov5cls) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Classification | [PaddleClas/PP-LCNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/PP-LCNetv2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/EfficientNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/GhostNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/MobileNetV3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/ShuffleNetV2](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/SqueeezeNetV1.1](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/Inceptionv3](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Classification | [PaddleClas/PP-HGNet](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Classification | [PaddleClas/SwinTransformer](./examples/vision/classification/paddleclas) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLOE](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PicoDet](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/YOLOX](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/YOLOv3](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLO](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/PP-YOLOv2](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/Faster-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| Detection | [PaddleDetection/Mask-RCNN](./examples/vision/detection/paddledetection) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | Detection | [Megvii-BaseDetection/YOLOX](./examples/vision/detection/yolox) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [WongKinYiu/YOLOv7](./examples/vision/detection/yolov7) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [WongKinYiu/YOLOv7end2end_trt](./examples/vision/detection/yolov7end2end_trt) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | @@ -285,10 +286,10 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava | Detection | [WongKinYiu/ScaledYOLOv4](./examples/vision/detection/scaledyolov4) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [ppogg/YOLOv5Lite](./examples/vision/detection/yolov5lite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Detection | [RangiLyu/NanoDetPlus](./examples/vision/detection/nanodet_plus) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| KeyPoint | [PaddleDetection/TinyPose](./examples/vision/keypointdetection/tiny_pose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | +| KeyPoint | [PaddleDetection/PicoDet + TinyPose](./examples/vision/keypointdetection/det_keypoint_unite) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | HeadPose | [omasaht/headpose](examples/vision/headpose) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Tracking | [PaddleDetection/PP-Tracking](examples/vision/tracking/pptracking) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | OCR | [PaddleOCR/PP-OCRv2](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | OCR | [PaddleOCR/PP-OCRv3](./examples/vision/ocr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Segmentation | [PaddleSeg/PP-LiteSeg](./examples/vision/segmentation/paddleseg) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | @@ -316,12 +317,12 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava | Video Super-Resolution | [PaddleGAN/BasicVSR](./) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Video Super-Resolution | [PaddleGAN/EDVR](./examples/vision/sr/edvr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | | Video Super-Resolution | [PaddleGAN/PP-MSVSR](./examples/vision/sr/ppmsvsr) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | +| Information Extraction | [PaddleNLP/UIE](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | | NLP | [PaddleNLP/ERNIE-3.0](./examples/text/ernie-3.0) | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ | | Speech | [PaddleSpeech/PP-TTS](./examples/text/uie) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -- | ✅ |
- +
## 📱 Mobile and Edge Device Deployment @@ -329,7 +330,7 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava
### Mobile and Edge Model List 🔥🔥🔥🔥 - +
Mobile and Edge Model List（click to fold）
@@ -374,13 +375,13 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava | OCR | [PaddleOCR/PP-OCRv2](examples/vision/ocr/PP-OCRv2) | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- | | OCR | [PaddleOCR/PP-OCRv3](examples/vision/ocr/PP-OCRv3) | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- | | OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- | - +
## 🌐 Browser-based Model List
- +
Browser-based Model List（click to fold）
| Task | Model | [web_demo](examples/application/js/web_demo) | @@ -394,7 +395,7 @@ Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Ava | OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
- + ## Community
diff --git a/examples/text/ernie-3.0/serving/README.md b/examples/text/ernie-3.0/serving/README.md index fcf3b720e..6edc1b790 100644 --- a/examples/text/ernie-3.0/serving/README.md +++ b/examples/text/ernie-3.0/serving/README.md @@ -1,5 +1,9 @@ # ERNIE 3.0 服务化部署示例 +在服务化部署前，需确认 + +- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md) + ## 准备模型 下载ERNIE 3.0的新闻分类模型、序列标注模型(如果有已训练好的模型，跳过此步骤): @@ -43,14 +47,14 @@ models ## 拉取并运行镜像 ```bash -# CPU镜像, 仅支持Paddle/ONNX模型在CPU上进行服务化部署，支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime -docker pull paddlepaddle/fastdeploy:0.3.0-cpu-only-21.10 - -# GPU 镜像, 支持Paddle/ONNX模型在GPU/CPU上进行服务化部署，支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime -docker pull paddlepaddle/fastdeploy:0.3.0-gpu-cuda11.4-trt8.4-21.10 +# x.y.z为镜像版本号，需参照serving文档替换为数字 +# GPU镜像 +docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 +# CPU镜像 +docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10 # 运行 -docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.3.0-cpu-only-21.10 bash +docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 bash ``` ## 部署模型 @@ -63,7 +67,7 @@ token_cls_rpc_client.py # 序列标注任务发送pipeline预测请求的脚 ``` *注意*:启动服务时，Server的每个python后端进程默认申请`64M`内存，默认启动的docker无法启动多个python后端节点。有两个解决方案： -- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.3.0-gpu-cuda11.4-trt8.4-21.10 bash` +- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash` - 2.启动服务时设置python后端的`shm-default-byte-size`参数, 设置python后端的默认内存为10M： `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760` ### 分类任务 diff --git a/examples/text/uie/serving/README.md b/examples/text/uie/serving/README.md index f33a6e41c..c2ade2a54 100644 --- a/examples/text/uie/serving/README.md +++ b/examples/text/uie/serving/README.md @@ -1,5 +1,9 @@ # UIE 服务化部署示例 +在服务化部署前，需确认 + +- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md) + ## 准备模型 下载UIE-Base模型(如果有已训练好的模型，跳过此步骤): @@ -26,11 +30,11 @@ models ## 拉取并运行镜像 ```bash -# CPU镜像, 仅支持Paddle/ONNX模型在CPU上进行服务化部署，支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime -docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 - -# GPU 镜像, 支持Paddle/ONNX模型在GPU/CPU上进行服务化部署，支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime -docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 +# x.y.z为镜像版本号，需参照serving文档替换为数字 +# GPU镜像 +docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 +# CPU镜像 +docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10 # 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /uie_serving 目录 docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash diff --git a/examples/vision/classification/paddleclas/serving/README.md b/examples/vision/classification/paddleclas/serving/README.md index 77d8046be..971e9bbd0 100644 --- a/examples/vision/classification/paddleclas/serving/README.md +++ b/examples/vision/classification/paddleclas/serving/README.md @@ -1,5 +1,10 @@ # PaddleClas 服务化部署示例 +在服务化部署前，需确认 + +- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md) + + ## 启动服务 ```bash @@ -19,11 +24,11 @@ mv ResNet50_vd_infer/inference_cls.yaml models/preprocess/1/inference_cls.yaml mv ResNet50_vd_infer/inference.pdmodel models/runtime/1/model.pdmodel mv ResNet50_vd_infer/inference.pdiparams models/runtime/1/model.pdiparams -# 拉取fastdeploy镜像 +# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字) # GPU镜像 -docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 +docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 # CPU镜像 -docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 +docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10 # 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录 nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash @@ -33,7 +38,7 @@ CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --bac ``` >> **注意**: ->> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README.md) +>> 拉取其他硬件上的镜像请看[服务化部署主文档](../../../../../serving/README_CN.md) >> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改客户端示例中的请求端口号. diff --git a/examples/vision/detection/paddledetection/serving/README.md b/examples/vision/detection/paddledetection/serving/README.md index 5791c8c5a..008fe4f8e 100644 --- a/examples/vision/detection/paddledetection/serving/README.md +++ b/examples/vision/detection/paddledetection/serving/README.md @@ -6,7 +6,7 @@ PaddleDetection模型导出和预训练模型下载请看[PaddleDetection模型 在服务化部署前，需确认 -- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README.md) +- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md) ## 启动服务 @@ -52,7 +52,7 @@ CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models >> 由于mask_rcnn模型多一个输出，部署mask_rcnn需要将后处理目录(models/postprocess)中的mask_config.pbtxt重命名为config.pbtxt ->> 拉取镜像请看[服务化部署主文档](../../../../../serving/README.md) +>> 拉取镜像请看[服务化部署主文档](../../../../../serving/README_CN.md) >> 执行fastdeployserver启动服务出现"Address already in use", 请使用`--grpc-port`指定grpc端口号来启动服务，同时更改客户端示例中的请求端口号. diff --git a/examples/vision/detection/yolov5/serving/README.md b/examples/vision/detection/yolov5/serving/README.md index 4f95df441..3e341ff6d 100644 --- a/examples/vision/detection/yolov5/serving/README.md +++ b/examples/vision/detection/yolov5/serving/README.md @@ -1,5 +1,10 @@ # YOLOv5 服务化部署示例 +在服务化部署前，需确认 + +- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md) + + ## 启动服务 ```bash @@ -13,8 +18,11 @@ wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov5s.onnx # 将模型放入 models/runtime/1目录下, 并重命名为model.onnx mv yolov5s.onnx models/runtime/1/model.onnx -# 拉取fastdeploy镜像 -docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 +# 拉取fastdeploy镜像(x.y.z为镜像版本号，需参照serving文档替换为数字) +# GPU镜像 +docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 +# CPU镜像 +docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10 # 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /yolov5_serving 目录 nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash diff --git a/serving/Dockerfile b/serving/Dockerfile index fc5a45693..139d2cd0e 100644 --- a/serving/Dockerfile +++ b/serving/Dockerfile @@ -20,7 +20,7 @@ COPY --from=full /opt/tritonserver/lib /opt/tritonserver/lib COPY --from=full /opt/tritonserver/include /opt/tritonserver/include COPY --from=full /opt/tritonserver/backends/python /opt/tritonserver/backends/python -COPY serving/TensorRT-8.4.1.5 /opt/ +COPY serving/TensorRT-8.4.1.5 /opt/TensorRT-8.4.1.5 ENV TZ=Asia/Shanghai \ DEBIAN_FRONTEND=noninteractive \ diff --git a/serving/README_CN.md b/serving/README_CN.md index 377698873..1ce2a7a07 100644 --- a/serving/README_CN.md +++ b/serving/README_CN.md @@ -30,15 +30,17 @@ docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 - [FastDeploy服务化部署镜像编译说明](docs/zh_CN/compile.md) ## 其它文档 -- [服务化模型目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型目录) -- [服务化部署配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项) -- [服务化部署示例](docs/zh_CN/demo.md) +- [模型仓库目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型仓库目录) +- [模型配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项) -### 模型示例列表 + +### 服务化部署示例 | 任务场景 | 模型 | |---|---| | Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) | +| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) | | Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) | | NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md)| +| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md)| | Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md)| diff --git a/serving/README_EN.md b/serving/README_EN.md index 30bc405c0..25c421329 100644 --- a/serving/README_EN.md +++ b/serving/README_EN.md @@ -39,14 +39,15 @@ Users can also compile the image by themselves according to their own needs, ref - [How to Prepare Serving Model Repository](docs/zh_CN/model_repository.md) - [Serving Deployment Configuration for Runtime](docs/zh_CN/model_configuration.md) -- [Serving Deployment Demo](docs/zh_CN/demo.md) - - [YOLOV5 - Detection Task](../examples/vision/detection/yolov5/serving/README.md) -### Model List + +### Serving Deployment Demo | Task | Model | |---|---| | Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) | +| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) | | Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) | | NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md)| +| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md)| | Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md)|