mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-12-24 13:28:13 +08:00
c2533f1f75
* Add benchmark for ernie sequence classification * Add pretty print * Update benchmark of ernie * get_table -> get_statistics_table * add comments * Update the output * Add cpu gpu memory statitis * Add gpu utilization sampling Co-authored-by: Jason <jiangjiajun@baidu.com>
26 lines
1.7 KiB
Bash
26 lines
1.7 KiB
Bash
# Download and decompress the ERNIE 3.0 Medium model finetuned on AFQMC
|
|
# wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz
|
|
# tar xvfz ernie-3.0-medium-zh-afqmc.tgz
|
|
|
|
# Download and decompress the quantization model of ERNIE 3.0 Medium model
|
|
# wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc-new-quant.tgz
|
|
# tar xvfz ernie-3.0-medium-zh-afqmc-new-quant.tgz
|
|
|
|
# PP-TRT
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc --backend pp-trt
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc --backend pp-trt --use_fp16 True
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc-new-quant --backend pp-trt
|
|
|
|
# TRT
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc --backend trt
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc --backend trt --use_fp16 True
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --model_dir ernie-3.0-medium-zh-afqmc-new-quant --backend trt --use_fp16 True
|
|
|
|
# CPU PP
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --cpu_num_threads 10 --model_dir ernie-3.0-medium-zh-afqmc --backend pp --device cpu
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --cpu_num_threads 10 --model_dir ernie-3.0-medium-zh-afqmc-new-quant --backend pp --device cpu
|
|
|
|
# CPU ORT
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --cpu_num_threads 10 --model_dir ernie-3.0-medium-zh-afqmc --backend ort --device cpu
|
|
python benchmark_ernie_seq_cls.py --batch_size 40 --cpu_num_threads 10 --model_dir ernie-3.0-medium-zh-afqmc-new-quant --backend ort --device cpu
|