apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[CE]add wint4 ep (#5355)

Browse Source
This commit is contained in:
tianlef
2025-12-03 15:17:47 +08:00
committed by GitHub
parent d5a9b75b4e
commit 04d35ace5e
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
benchmarks/yaml/eb45-32k-wint4-ep4-tp4.yaml Normal file
View File

@@ -0,0 +1,7 @@
num_gpu_blocks_override: 1024
max_model_len: 8192
max_num_seqs: 64
data_parallel_size: 4
tensor_parallel_size: 1
enable_expert_parallel: True
quantization: wint4