mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-06 17:17:14 +08:00

Files

yunyaoXYY b0663209f6 Add Examples to deploy quantized models (#342 )

* Add PaddleOCR Support

* Add PaddleOCR Support

* Add PaddleOCRv3 Support

* Add PaddleOCRv3 Support

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Add PaddleOCRv3 Support

* Add PaddleOCRv3 Supports

* Add PaddleOCRv3 Suport

* Fix Rec diff

* Remove useless functions

* Remove useless comments

* Add PaddleOCRv2 Support

* Add PaddleOCRv3 & PaddleOCRv2 Support

* remove useless parameters

* Add utils of sorting det boxes

* Fix code naming convention

* Fix code naming convention

* Fix code naming convention

* Fix bug in the Classify process

* Imporve OCR Readme

* Fix diff in Cls model

* Update Model Download Link in Readme

* Fix diff in PPOCRv2

* Improve OCR readme

* Imporve OCR readme

* Improve OCR readme

* Improve OCR readme

* Imporve OCR readme

* Improve OCR readme

* Fix conflict

* Add readme for OCRResult

* Improve OCR readme

* Add OCRResult readme

* Improve OCR readme

* Improve OCR readme

* Add Model Quantization Demo

* Fix Model Quantization Readme

* Fix Model Quantization Readme

* Add the function to do PTQ quantization

* Improve quant tools readme

* Improve quant tool readme

* Improve quant tool readme

* Add PaddleInference-GPU for OCR Rec model

* Add QAT method to fastdeploy-quantization tool

* Remove examples/slim for now

* Move configs folder

* Add Quantization Support for Classification Model

* Imporve ways of importing preprocess

* Upload YOLO Benchmark on readme

* Upload YOLO Benchmark on readme

* Upload YOLO Benchmark on readme

* Improve Quantization configs and readme

* Add support for multi-inputs model

* Add backends and params file for YOLOv7

* Add quantized model deployment support for YOLO series

* Fix YOLOv5 quantize readme

* Fix YOLO quantize readme

* Fix YOLO quantize readme

* Improve quantize YOLO readme

* Improve quantize YOLO readme

* Improve quantize YOLO readme

* Improve quantize YOLO readme

* Improve quantize YOLO readme

* Fix bug, change Fronted to ModelFormat

* Change Fronted to ModelFormat

* Add examples to deploy quantized paddleclas models

* Fix readme

* Add quantize Readme

* Add quantize Readme

* Add quantize Readme

* Modify readme of quantization tools

* Modify readme of quantization tools

* Improve quantization tools readme

* Improve quantization readme

* Improve PaddleClas quantized model deployment  readme

* Add PPYOLOE-l quantized deployment examples

* Improve quantization tools readme

2022-10-14 13:35:45 +08:00

2.3 KiB

Raw Blame History

PaddleClas 量化模型部署

FastDeploy已支持部署量化模型,并提供一键模型量化的工具. 用户可以使用一键模型量化工具,自行对模型量化后部署, 也可以直接下载FastDeploy提供的量化模型进行部署.

FastDeploy一键模型量化工具

FastDeploy 提供了一键量化工具, 能够简单地通过输入一个配置文件, 对模型进行量化. 详细教程请见: 一键模型量化工具注意: 推理量化后的分类模型仍然需要FP32模型文件夹下的inference_cls.yaml文件, 自行量化的模型文件夹内不包含此yaml文件, 用户从FP32模型文件夹下复制此yaml文件到量化后的模型文件夹内即可。

下载量化完成的PaddleClas模型

用户也可以直接下载下表中的量化模型进行部署.

模型	推理后端	部署硬件	FP32推理时延	INT8推理时延	加速比	FP32 Top1	INT8 Top1	量化方式
ResNet50_vd	ONNX Runtime	CPU	86.87	59 .32	1.46	79.12	78.87	离线量化
ResNet50_vd	TensorRT	GPU	7.85	5.42	1.45	79.12	79.06	离线量化
MobileNetV1_ssld	ONNX Runtime	CPU	40.32	16.87	2.39	77.89	75.09	离线量化
MobileNetV1_ssld	TensorRT	GPU	5.10	3.35	1.52	77.89	76.86	离线量化

上表中的数据, 为模型量化前后，在FastDeploy部署的端到端推理性能.

测试图片为ImageNet-2012验证集中的图片.
推理时延为端到端推理(包含前后处理)的平均时延, 单位是毫秒.
CPU为Intel(R) Xeon(R) Gold 6271C, GPU为Tesla T4, TensorRT版本8.4.15, 所有测试中固定CPU线程数为1.

2.3 KiB Raw Blame History Unescape Escape

PaddleClas 量化模型部署

FastDeploy一键模型量化工具

下载量化完成的PaddleClas模型

详细部署文档

2.3 KiB

Raw Blame History