apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
Files
92c2cfa2e7f5d7b87bb070c6ec9e8a59c554f766
FastDeploy/benchmarks/yaml/eb45-21b-a3b-32k-wint8.yaml
Jiang-Jia-Jun 92c2cfa2e7 Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00

7 lines
134 B
YAML
Raw Blame History

max_model_len: 32768
max_num_seqs: 128
kv_cache_ratio: 0.75
tensor_parallel_size: 1
max_num_batched_tokens: 32768
quantization: wint8
Reference in New Issue View Git Blame Copy Permalink