FastDeploy/README.md

⚡️FastDeploy是一款简单易用的推理部署工具箱。覆盖业界主流优质预训练模型并提供开箱即用的开发体验，包括图像分类、目标检测、图像分割、人脸检测、人体关键点识别、文字识别等多任务，满足开发者多场景，多硬件、多平台的快速部署需求。

发版历史

• [v0.2.0] 2022.08.18 全面开源服务端部署代码，支持40+视觉模型在CPU/GPU，以及通过TensorRT加速部署

内容目录

• 服务端模型列表
• 服务端快速开始
  • 快速安装
  • Python预测示例
  • C++预测示例
• 轻量化SDK快速实现端侧AI推理部署
  • 边缘侧部署
  • 移动端部署
  • 自定义模型部署
• 社区交流
• Acknowledge
• License

1. 服务端模型列表 🔥🔥🔥

符号说明: (1) ✅: 已经支持; (2) ❔: 计划未来支持; (3) ❌: 暂不支持; (4) contrib: 外部模型

任务场景	模型	API	Linux	Linux	Win	Win	Mac	Mac	Linux	Linux
---	---	---	X86 CPU	NVIDIA GPU	Intel CPU	NVIDIA GPU	Intel CPU	Arm CPU	AArch64 CPU	NVIDIA Jetson
Classification	PaddleClas/ResNet50	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/PPLCNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/PPLCNetv2	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/EfficientNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/GhostNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/MobileNetV1	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/MobileNetV2	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/MobileNetV3	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/ShuffleNetV2	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/SqueeezeNetV1.1	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/Inceptionv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/PPHGNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Classification	PaddleClas/SwinTransformer	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	PaddleDetection/PPYOLOE	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	PaddleDetection/PicoDet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	PaddleDetection/YOLOX	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	PaddleDetection/YOLOv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	PaddleDetection/PPYOLO	Python/C++	✅	✅	✅	✅	✅	❌	❌	❔
Detection	PaddleDetection/PPYOLOv2	Python/C++	✅	✅	✅	✅	✅	❌	❌	❔
Detection	PaddleDetection/FasterRCNN	Python/C++	✅	✅	✅	✅	✅	❌	❌	❔
Detection	Contrib/YOLOX	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/YOLOv7	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/YOLOv6	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/YOLOv5	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/YOLOR	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/ScaledYOLOv4	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/YOLOv5Lite	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Detection	Contrib/NanoDetPlus	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/PPLiteSeg	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/PPHumanSegLite	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/HRNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/PPHumanSegServer	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/Unet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Segmentation	PaddleSeg/Deeplabv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceDetection	Contrib/RetinaFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceDetection	Contrib/UltraFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceDetection	Contrib/YOLOv5Face	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceDetection	Contrib/SCRFD	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceRecognition	Contrib/ArcFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceRecognition	Contrib/CosFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceRecognition	Contrib/PartialFC	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
FaceRecognition	Contrib/VPL	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔
Matting	Contrib/MODNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	❔

2. 服务端快速开始

💡 快速安装 FastDeploy Python/C++ 库

用户根据自己的python版本选择安装对应的wheel包，详细的wheel目录请参考 python安装文档 .

pip install https://bj.bcebos.com/paddlehub/fastdeploy/wheels/fastdeploy_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl

或获取C++预编译库，更多可用的预编译库请参考 C++预编译库下载

wget https://bj.bcebos.com/paddlehub/fastdeploy/cpp/fastdeploy-linux-x64-0.2.0.tgz

准备目标检测模型和测试图片

wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg

2.1 Python预测示例

import cv2
import fastdeploy.vision as vision

model = vision.detection.PPYOLOE("model.pdmodel", "model.pdiparams", "infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)

2.2 C++预测示例

#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("model.pdmodel", "model.pdiparams", "infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(&im, &res)

  auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
}

更多部署案例请参考视觉模型部署示例 .

3. 轻量化SDK快速实现端侧AI推理部署 📱

任务场景	模型	大小(MB)	边缘端	移动端	移动端
---	---	---	Linux	Android	iOS
---	---	---	ARM CPU	ARM CPU	ARM CPU
Classification	PP-LCNet	11.9	✅	✅	✅
Classification	PP-LCNetv2	26.6	✅	✅	✅
Classification	EfficientNet	31.4	✅	✅	✅
Classification	GhostNet	20.8	✅	✅	✅
Classification	MobileNetV1	17	✅	✅	✅
Classification	MobileNetV2	14.2	✅	✅	✅
Classification	MobileNetV3	22	✅	✅	✅
Classification	ShuffleNetV2	9.2	✅	✅	✅
Classification	SqueezeNetV1.1	5	✅	✅	✅
Classification	Inceptionv3	95.5	✅	✅	✅
Classification	PP-HGNet	59	✅	✅	✅
Classification	SwinTransformer_224_win7	352.7	✅	✅	✅
Detection	PP-PicoDet_s_320_coco	4.1	✅	✅	✅
Detection	PP-PicoDet_s_320_lcnet	4.9	✅	✅	✅
Detection	CenterNet	4.8	✅	✅	✅
Detection	YOLOv3_MobileNetV3	94.6	✅	✅	✅
Detection	PP-YOLO_tiny_650e_coco	4.4	✅	✅	✅
Detection	SSD_MobileNetV1_300_120e_voc	23.3	✅	✅	✅
Detection	PP-YOLO_ResNet50vd	188.5	✅	✅	✅
Detection	PP-YOLOv2_ResNet50vd	218.7	✅	✅	✅
Detection	PP-YOLO_crn_l_300e_coco	209.1	✅	✅	✅
Detection	YOLOv5s	29.3	✅	✅	✅
FaceDetection	BlazeFace	1.5	✅	✅	✅
FaceDetection	RetinaFace	1.7	✅	❌	❌
KeypointsDetection	PP-TinyPose	5.5	✅	✅	✅
Segmentation	PP-LiteSeg(STDC1)	32.2	✅	✅	✅
Segmentation	PP-HumanSeg-Lite	0.556	✅	✅	✅
Segmentation	HRNet-w18	38.7	✅	✅	✅
Segmentation	PP-HumanSeg-Server	107.2	✅	✅	✅
Segmentation	Unet	53.7	❌	✅	❌
OCR	PP-OCRv1	2.3+4.4	✅	✅	✅
OCR	PP-OCRv2	2.3+4.4	✅	✅	✅
OCR	PP-OCRv3	2.4+10.6	✅	✅	✅
OCR	PP-OCRv3-tiny	2.4+10.7	✅	✅	✅

3.1 边缘侧部署

• ARM Linux 系统
  • C++ Inference部署（含视频流）
  • C++ 服务化部署
  • Python Inference部署
  • Python 服务化部署

3.2 移动端部署

• iOS 系统部署
• Android 系统部署

3.3 自定义模型部署

• 快速实现个性化模型替换

4. 社区交流

• 加入社区👬： 微信扫描二维码后，填写问卷加入交流群，与开发者共同讨论推理部署痛点问题

5. Acknowledge

本项目中SDK生成和下载使用了EasyEdge中的免费开放能力，再次表示感谢。

6. License

FastDeploy遵循Apache-2.0开源协议。