apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Feature] support chunked moe (#4575)

Browse Source 
* [Feature] support chunked moe

* update

* update

* fix and add test

* update

* fix conflict and modity test

* fix fused_moe

* fix fused_moe

* fix docstring

* fix

* fix typo

* fix test

* fix

* fix

* fix test

* fix test
This commit is contained in:
Longzhi Wang
2025-12-01 15:17:18 +08:00
committed by GitHub
parent 6f42c37359
commit add524d80c
10 changed files with 405 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
fastdeploy/config.py
View File

@@ -540,6 +540,11 @@ class ParallelConfig:
self.expert_parallel_size = 1 # EP degree
self.data_parallel_size = 1 # DP degree
self.enable_expert_parallel = False
self.enable_chunked_moe = False
self.chunked_moe_size = 256
self.max_moe_num_chunk = 1
self.moe_num_chunk = 1
self.local_data_parallel_id = 0
# Engine worker queue port
self.engine_worker_queue_port: str = "9923"