16 Commits

Author	SHA1	Message	Date
Zero Rains	0fb37ab7e4	update flake8 version to support pre-commit in python3.12 (#3000) ... * update flake8 version to support pre-commit in python3.12 * polish code	2025-07-24 01:43:31 -07:00
ltd0924	f935d6f862	[BugFix] fix multinode deployment (#2977)	2025-07-24 15:04:04 +08:00
Yzc216	e14587a954	[Feature] multi-source download (#2986) ... * multi-source download * multi-source download * huggingface download revision * requirement * style * add revision arg * test * pre-commit	2025-07-24 14:26:37 +08:00
chenjian	85a78d695d	[Feature] Support block scheduler v1 for FD (#2928) ... * Support FD block scheduler v1 * Support FD block scheduler v1 * Support FD block scheduler v1 * Fix according to copilot review * Fix according to review * Remove is_dummy * Fix bug when real_bsz=1 * Fix infer first token cost time --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-23 20:31:31 +08:00
Zero Rains	89a485b69f	[Feature] Support using prefix-caching + cudagraph for inference (#2924) ... * fix the bug in cudagraph+prefix-caching but still have some bug with profile Change-Id: Ibf2ba3f2e3b08641d03f4b1391d7c862c3efa397 * add the signal to make sure cache manager launched * fix judge condition * reomove useless control * update control stream * update * fix xpu * change the do_profile flag * update * add new threads to init cache_manager --------- Co-authored-by: RAM <gstian5555@outlook.com>	2025-07-22 00:59:45 -07:00
zhink	0262ef7eb3	custom all reduce support cuda graph (#2938) ... * Support enabling cuda graph and custom all reduce at the same time, and fix the overwritten custom all reduce flag * rename communication_op to communication	2025-07-21 22:52:03 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
ltd0924	9c25dcca0b	[LLM] Update Multinode Deployment (#2830) ... * [LLM] fix multinode bugs * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] fix ci bugs * Update fastdeploy/engine/args_utils.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [LLM] update random port * [LLM] update random port * [LLM] fix ci bugs * fix ci bugs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-07-16 23:42:54 +08:00
Yuanle Liu	dda4a9f848	rl update (#2861)	2025-07-16 00:33:10 -07:00
RAM	0fad10b35a	[Executor] CUDA Graph support padding batch (#2844) ... * cuda graph support padding batch * Integrate the startup parameters for the graph optimization backend and provide support for user - defined capture sizes. * Do not insert max_num_seqs when the user specifies a capture list * Support set graph optimization config from YAML file * update cuda graph ci * fix ci bug * fix ci bug	2025-07-15 19:49:01 -07:00
chen	d33105baeb	[Feature] Online Chat API Support Return logprobs (#2777) ... * online chat support logprobs * check xpu * check vl_gpu_model_runner and xpu_model_runner * get_worker() check platform	2025-07-10 16:33:40 +08:00
zhink	b89180f1cd	[Feature] support custom all-reduce (#2758) ... * [Feature] support custom all-reduce * add vllm adapted	2025-07-09 16:00:27 +08:00
ltd0924	68b4755587	[LLM] support multi node deploy (#2708) ... * [LLM] support multi node deploy * Update engine.py * fix bugs * fix * [LLM] support multi node deploy * [LLM] support multi node deploy --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-06 10:33:51 +08:00
Jiang-Jia-Jun	05c670e593	[Sync] Update to latest code (#2679) ... * [Sync] Update to latest code * Add new code files * Add new code files * update code * Try to fix build.sh * Try to fix build.sh * Update code * Update requirements.txt * Update code --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-07-03 15:43:53 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00
jiangjiajun	684703fd72	[LLM] First commit the llm deployment code	2025-06-09 19:20:15 +08:00