12 Commits

Author	SHA1	Message	Date
Zero Rains	89a485b69f	[Feature] Support using prefix-caching + cudagraph for inference (#2924) ... * fix the bug in cudagraph+prefix-caching but still have some bug with profile Change-Id: Ibf2ba3f2e3b08641d03f4b1391d7c862c3efa397 * add the signal to make sure cache manager launched * fix judge condition * reomove useless control * update control stream * update * fix xpu * change the do_profile flag * update * add new threads to init cache_manager --------- Co-authored-by: RAM <gstian5555@outlook.com>	2025-07-22 00:59:45 -07:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
YuanRisheng	101ad33332	[BugFix] Fix Configs (#2849) ... * fix config * fix config	2025-07-15 19:50:36 -07:00
RAM	0fad10b35a	[Executor] CUDA Graph support padding batch (#2844) ... * cuda graph support padding batch * Integrate the startup parameters for the graph optimization backend and provide support for user - defined capture sizes. * Do not insert max_num_seqs when the user specifies a capture list * Support set graph optimization config from YAML file * update cuda graph ci * fix ci bug * fix ci bug	2025-07-15 19:49:01 -07:00
YuanRisheng	4c7b8bc458	Simplify the Config code (#2770) ... * simplify the code * fix vl * delete config * fix * perfect code * fix ci * fix xpu * fix xpu * fix server * resolve conflict * fix mtp * resolve conflict * fix xpu * fix xpu * fix vl * fix log * fix qwen moe * fix qwen moe * fix qwen moe	2025-07-14 19:50:05 +08:00
Sunny-bot1	f6ad26fc08	fix topp default value (#2814)	2025-07-11 17:10:21 +08:00
Sunny-bot1	240d6236bc	[Fix]fix top_k_top_p sampling (#2801) ... * fix topk-topp * update * add base_non_truncated	2025-07-10 22:35:10 +08:00
littledgg	59071268b6	[Executor] Move forward_meta.py to fastdeploy/model_executor (#2774) ... * Use PEP 563 in attention.py and fix conflict * merge commit * Change what was left out last time	2025-07-10 20:36:51 +08:00
chen	d33105baeb	[Feature] Online Chat API Support Return logprobs (#2777) ... * online chat support logprobs * check xpu * check vl_gpu_model_runner and xpu_model_runner * get_worker() check platform	2025-07-10 16:33:40 +08:00
Sunny-bot1	e45050cae3	[Feature] support top_k_top_p sampling (#2753) ... * support top_k_top_p sampling * fix * add api param * add api para * fix * fix * fix * fix * fix * fix * fix	2025-07-09 20:58:58 -07:00
RAM	03a74995b8	Clear dead code And supplementary notes (#2757) ... * 1.supplementary notes 2.delete dead code * fix bug of forward meta * Global modification of forward meta * fix vl model_runner bug	2025-07-09 16:17:34 +08:00
EnflameGCU	d0f4d6ba3a	[GCU] Support gcu platform (#2702) ... baseline: e7fa57ebae Co-authored-by: yongqiangma <xing.wo@163.com>	2025-07-08 13:00:52 +08:00