FastDeploy/tools/common_tools/auto_compression

FastDeploy One-Click Model Auto Compression

FastDeploy, based on PaddleSlim's Auto Compression Toolkit(ACT), provides developers with a one-click model auto compression tool that supports post-training quantization and knowledge distillation training. We take the Yolov5 series as an example to demonstrate how to install and execute FastDeploy's one-click model auto compression.

1.Install

Environment Dependencies

1.Install PaddlePaddle 2.4 version

https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html

2.Install PaddleSlim 2.4 version

pip install paddleslim==2.4.0

Install Fastdeploy Auto Compression Toolkit

FastDeploy One-Click Model Automation compression does not require a separate installation, users only need to properly install the FastDeploy Toolkit

2. How to Use

Demo for One-Click Auto Compression Toolkit

Fastdeploy Auto Compression can include multiple strategies, At present, offline quantization and quantization distillation are mainly used for training. The following will introduce how to use it from two strategies, offline quantization and quantitative distillation.

Offline Quantization

1. Prepare models and Calibration data set

Developers need to prepare the model to be quantized and the Calibration dataset on their own. In this demo, developers can execute the following command to download the yolov5s.onnx model to be quantized and calibration data set.

# Download yolov5.onnx
wget https://paddle-slim-models.bj.bcebos.com/act/yolov5s.onnx

# Download dataset. This Calibration dataset is the first 320 images from COCO val2017
wget https://bj.bcebos.com/paddlehub/fastdeploy/COCO_val_320.tar.gz
tar -xvf COCO_val_320.tar.gz

2. Run fastdeploy compress command to compress the model

The following command is to quantize the yolov5s model, if developers want to quantize other models, replace the config_path with other model configuration files in the configs folder.

fastdeploy compress --config_path=./configs/detection/yolov5s_quant.yaml --method='PTQ' --save_dir='./yolov5s_ptq_model/'

[notice] PTQ is short for post-training quantization

3. Parameters

To complete the quantization, developers only need to provide a customized model config file, specify the quantization method, and the path to save the quantized model.

Parameter	Description
--config_path	Quantization profiles needed for one-click quantization.Configs
--method	Quantization method selection, PTQ for post-training quantization, QAT for quantization distillation training
--save_dir	Output of quantized model paths, which can be deployed directly in FastDeploy

Quantized distillation training

1.Prepare the model to be quantized and the training data set

FastDeploy currently supports quantized distillation training only for images without annotation. It does not support evaluating model accuracy during training. The datasets are images from inference application, and the number of images is determined by the size of the dataset, covering all deployment scenarios as much as possible. In this demo, we prepare the first 320 images from the COCO2017 validation set for users. Note: If users want to obtain a more accurate quantized model through quantized distillation training, feel free to prepare more data and train more rounds.

# Download yolov5.onnx
wget https://paddle-slim-models.bj.bcebos.com/act/yolov5s.onnx

# Download dataset. This Calibration dataset is the first 320 images from COCO val2017
wget https://bj.bcebos.com/paddlehub/fastdeploy/COCO_val_320.tar.gz
tar -xvf COCO_val_320.tar.gz

2.Use fastdeploy compress command to compress models

The following command is to quantize the yolov5s model, if developers want to quantize other models, replace the config_path with other model configuration files in the configs folder.

# Please specify the single card GPU before training, otherwise it may get stuck during the training process.
export CUDA_VISIBLE_DEVICES=0
fastdeploy compress --config_path=./configs/detection/yolov5s_quant.yaml --method='QAT' --save_dir='./yolov5s_qat_model/'

3.Parameters

To complete the quantization, developers only need to provide a customized model config file, specify the quantization method, and the path to save the quantized model.

Parameter	Description
--config_path	Quantization profiles needed for one-click quantization.Configs
--method	Quantization method selection, PTQ for post-training quantization, QAT for quantization distillation training
--save_dir	Output of quantized model paths, which can be deployed directly in FastDeploy

3. FastDeploy One-Click Model Auto Compression Config file examples

FastDeploy currently provides users with compression config files of multiple models, and the corresponding FP32 model, Users can directly download and experience it.

Config file	FP32 model	Note
mobilenetv1_ssld_quant	mobilenetv1_ssld
resnet50_vd_quant	resnet50_vd
efficientnetb0_quant	efficientnetb0
mobilenetv3_large_x1_0_quant	mobilenetv3_large_x1_0
pphgnet_tiny_quant	pphgnet_tiny
pplcnetv2_base_quant	pplcnetv2_base
yolov5s_quant	yolov5s
yolov6s_quant	yolov6s
yolov7_quant	yolov7
ppyoloe_withNMS_quant	ppyoloe_l	Support PPYOLOE's s,m,l,x series models, export the model normally when exporting the model from PaddleDetection, do not remove NMS
ppyoloe_plus_withNMS_quant	ppyoloe_plus_s	Support PPYOLOE+'s s,m,l,x series models, export the model normally when exporting the model from PaddleDetection, do not remove NMS
pp_liteseg_quant	pp_liteseg
deeplabv3_resnet_quant	deeplabv3_resnet101
fcn_hrnet_quant	fcn_hrnet
unet_quant	unet

3. Deploy quantized models on FastDeploy

Once obtained the quantized model, developers can deploy it on FastDeploy. Please refer to the following docs for more details

• YOLOv5 Quantized Model Deployment

• YOLOv6 Quantized Model Deployment

• YOLOv7 Quantized Model Deployment

• PadddleClas Quantized Model Deployment

• PadddleDetection Quantized Model Deployment

• PadddleSegmentation Quantized Model Deployment