apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Feature] support chunked moe (#4575)

Browse Source 
* [Feature] support chunked moe

* update

* update

* fix and add test

* update

* fix conflict and modity test

* fix fused_moe

* fix fused_moe

* fix docstring

* fix

* fix typo

* fix test

* fix

* fix

* fix test

* fix test
This commit is contained in:
Longzhi Wang
2025-12-01 15:17:18 +08:00
committed by GitHub
parent 6f42c37359
commit add524d80c
10 changed files with 405 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
fastdeploy/worker/worker_process.py
View File

@@ -720,6 +720,17 @@ def parse_args():
action="store_true",
help="enable expert parallel",
)
parser.add_argument(
"--enable_chunked_moe",
action="store_true",
help="enable chunked moe",
)
parser.add_argument(
"--chunked_moe_size",
type=int,
default=256,
help="chunk size of moe input",
)
parser.add_argument("--ori_vocab_size", type=int, default=None)
parser.add_argument("--think_end_id", type=int, default=-1)
parser.add_argument("--image_patch_id", type=int, default=-1)