12 Commits

Author	SHA1	Message	Date
chenjian	6854506533	[Bug fix] Fix bug for d blocks not enough (#3479) ... * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Fix bug for memory allocation * Fix bug for D blocks not enough * fix bug when d blocks not enough * fix bug when d blocks not enough * fix cache message recycle step * fix cache message recycle step * Fix step_idx recycle	2025-08-21 11:36:16 +08:00
chenjian	aba94169dc	[Feature] Support batched tokens for EP (#3415) ... * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug	2025-08-18 11:43:36 +08:00
chenjian	7573802a88	[Feature] Support mtp ep in fd (#3340) ... * [Optimize] Add metrics for analysing perf * Fix bug in mtp	2025-08-11 21:49:44 +08:00
chenjian	110f33a530	[Bug fix] Test td cache messager (#3242) ... * support disable cache task in decode node * fix busg * Update engine.py * Update expert_service.py * Update splitwise_connector.py * Optimize log for debug * Optimize log for debug * fix bug --------- Co-authored-by: ltd0924 <ltd0924@sina.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com>	2025-08-06 15:52:45 +08:00
ltd0924	b20ffe3697	[Feature] optimize expert parallel (#3196) ... * optimize * Update expert_service.py * Update worker_process.py * optimize	2025-08-05 17:34:24 +08:00
chenjian	9f9971844f	[Feature] Support ep pd with external module (#3194) ... * Support external module * Support external module * Support external module * Support external module * refactor code to make it more clear * refactor code to make it more clear * refactor code to make it more clear * refactor code to make it more clear * fix according to review * fix according to review * fix according to review * fix according to review * fix according to review * fix according to review * fix bug * fix bug * fix bug * merge --------- Co-authored-by: root <root@tjdm-inf-sci-k8s-hzz2-h12ni8-0202.tjdm.baidu.com>	2025-08-04 20:32:41 +08:00
ltd0924	d17886de19	[Feature] support ep in mixed mode (#3001) ... * [LLM] support ep * Update worker_process.py * Update expert_service.py * Update worker_process.py * format files	2025-07-30 20:43:39 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
ltd0924	9c25dcca0b	[LLM] Update Multinode Deployment (#2830) ... * [LLM] fix multinode bugs * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] fix ci bugs * Update fastdeploy/engine/args_utils.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [LLM] update random port * [LLM] update random port * [LLM] fix ci bugs * fix ci bugs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-07-16 23:42:54 +08:00
Longzhi Wang	57b086dc6b	[Bug fix] Add the missing pod_ip param to the launch_cache_manager function. (#2742) ... * [Bug fix] fix the missing position args in expert_service.py * update	2025-07-08 14:52:13 +08:00
ltd0924	68b4755587	[LLM] support multi node deploy (#2708) ... * [LLM] support multi node deploy * Update engine.py * fix bugs * fix * [LLM] support multi node deploy * [LLM] support multi node deploy --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-06 10:33:51 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00