14 Commits

Author	SHA1	Message	Date
chenjian	aba94169dc	[Feature] Support batched tokens for EP (#3415) ... * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug * Support batched tokens for EP and fix bug	2025-08-18 11:43:36 +08:00
chenjian	110f33a530	[Bug fix] Test td cache messager (#3242) ... * support disable cache task in decode node * fix busg * Update engine.py * Update expert_service.py * Update splitwise_connector.py * Optimize log for debug * Optimize log for debug * fix bug --------- Co-authored-by: ltd0924 <ltd0924@sina.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com>	2025-08-06 15:52:45 +08:00
chenjian	9f9971844f	[Feature] Support ep pd with external module (#3194) ... * Support external module * Support external module * Support external module * Support external module * refactor code to make it more clear * refactor code to make it more clear * refactor code to make it more clear * refactor code to make it more clear * fix according to review * fix according to review * fix according to review * fix according to review * fix according to review * fix according to review * fix bug * fix bug * fix bug * merge --------- Co-authored-by: root <root@tjdm-inf-sci-k8s-hzz2-h12ni8-0202.tjdm.baidu.com>	2025-08-04 20:32:41 +08:00
gaoziyuan	4021d66ea5	【Feature】add fd plugins && rm model_classes (#3123) ... * add fd plugins && rm model_classed * fix reviews * add docs * fix * fix unitest ci	2025-08-03 19:53:20 -07:00
Yzc216	980126b83a	[Feature] multi source download (#3005) ... * multi-source download * multi-source download * huggingface download revision * requirement * style * add revision arg * test * pre-commit * Change default download * change requirements.txt * modify English Documentation * documentation	2025-07-24 17:42:09 +08:00
Yzc216	e14587a954	[Feature] multi-source download (#2986) ... * multi-source download * multi-source download * huggingface download revision * requirement * style * add revision arg * test * pre-commit	2025-07-24 14:26:37 +08:00
chenjian	85a78d695d	[Feature] Support block scheduler v1 for FD (#2928) ... * Support FD block scheduler v1 * Support FD block scheduler v1 * Support FD block scheduler v1 * Fix according to copilot review * Fix according to review * Remove is_dummy * Fix bug when real_bsz=1 * Fix infer first token cost time --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-23 20:31:31 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
sg263	e679567d59	[Trace]fix opentelemetry can not work in uvicorn (#2906) ... * add opentelemetry * add opentelemetry * add opentelemetry on dequeue * add opentelemetry on dequeue * add opentelemetry on dequeue * fix annotation * fix annotation when add opentelemetry * fix opentelemetry-instrumentation-fastapi * fix pentelemetry-bootstrap * fix opentelemetry can not work in uvicorn * move conf to env --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-17 23:16:45 +08:00
ltd0924	d245d1ca6c	[LLM] support send batch data and aggregate data (#2860) ... * [LLM] support send batch data and aggregate data * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] update	2025-07-16 23:42:20 +08:00
Sunny-bot1	240d6236bc	[Fix]fix top_k_top_p sampling (#2801) ... * fix topk-topp * update * add base_non_truncated	2025-07-10 22:35:10 +08:00
chen	888780ffde	[Feature] block_wise_fp8 support triton_moe_backend (#2767)	2025-07-09 19:22:47 +08:00
Jiang-Jia-Jun	05c670e593	[Sync] Update to latest code (#2679) ... * [Sync] Update to latest code * Add new code files * Add new code files * update code * Try to fix build.sh * Try to fix build.sh * Update code * Update requirements.txt * Update code --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-07-03 15:43:53 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00