FastDeploy/benchmarks/yaml/eb45-32k-wint8-a800-tp8.yaml at 775edcc09ac34a9f64762832cda3b975c6af8d2a - FastDeploy - 子说镜像小站

apps/FastDeploy

mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00

Files

RAM 775edcc09a [Executor] Default use CUDAGraph (#3594 )

* add start intercept

* Adjustment GraphOptConfig

* pre-commit

* default use cudagraph

* set default value

* default use cuda graph

* pre-commit

* fix test case bug

* disable rl

* fix moba attention

* only support gpu

* Temporarily disable PD Disaggregation

* set max_num_seqs of test case as 1

* set max_num_seqs and temperature

* fix max_num_batched_tokens bug

* close cuda graph

* success run wint2

* profile run with max_num_batched_tokens

* 1.add c++ memchecker 2.success run wint2

* updatee a800 yaml

* update docs

* 1. delete check 2. fix plas attn test case

* default use use_unique_memory_pool

* add try-except for warmup

* ban mtp, mm, rl

* fix test case mock

* fix ci bug

* fix form_model_get_output_topp0 bug

* fix ci bug

* refine deepseek ci

* refine code

* Disable PD

* fix sot yaml

2025-10-21 14:25:45 +08:00

7 lines

132 B

YAML

Raw Blame History

 max_model_len: 32768
 max_num_seqs: 96
 gpu_memory_utilization: 0.85
 kv_cache_ratio: 0.71
 tensor_parallel_size: 8
 quantization: wint8