1031 Commits

Author	SHA1	Message	Date
freeliuzc	bbae094cb9	[Optimization] Reduce memory allocate for cudaGraph (#4838) ... * optimize memory allocate * rename env	2025-11-06 13:32:47 +08:00
kxz2002	e0d98d00bc	[Cherry-Pick] Modify follow-up push parameters and Modify the verification method for thinking length (#4086) (#4826) ... * resolve #4086 conflict * fix unit test	2025-11-05 21:36:28 +08:00
xiaoxiaohehe001	ee37882a26	[NewFeature] support eplb noaux (#4725) ... * support eplb noaux * support eplb noaux * add eplb noaux test	2025-11-05 20:59:12 +08:00
Yuan Xiaolan	1e88754133	support set dy-C8 from args (#4475)	2025-11-04 17:01:35 +08:00
lizhenyun01	d65e00f8fb	[Feature] support get_task with tensor (#4751) ... * [Feature] support get_task with tensor * set FD_ENABLE_E2W_TENSOR_CONVERT default 0	2025-11-04 11:00:13 +08:00
kevin	9835697163	[Feature] Check bos url (#4677) ... * add bos url check * update code * update code	2025-10-31 15:26:30 +08:00
RAM	7847b44172	[Graph Optimization] Refactor default capture list (#4631) ... * refactore default capture list & refine code * fix bug * fix ci bug * Fix test case	2025-10-31 14:18:27 +08:00
yangjianfengo1	eef85e4ff0	[BugFix] update eb5 video chunk (#4705) ... * 修复视频和图片分chunk * 修复视频和图片分chunk * 修复视频和图片分chunk	2025-10-31 14:08:23 +08:00
kevin	c785c2dab1	[Scheduler] update v1 prefill batch (#4563) ... * update v1 scheduler * update code * update code * update code * update code * remove mm prefill batch * update code * fix bug * update code	2025-10-31 14:03:18 +08:00
Sunny-bot1	3f15e6fa15	load cache scale (#4623)	2025-10-31 11:57:57 +08:00
Jiang-Jia-Jun	71135d58a0	Change log level from info to debug for response	2025-10-30 14:02:50 +08:00
GoldPancake	05c1167c74	fix mtp logprob bugs (#4663)	2025-10-30 13:47:23 +08:00
Dangweichong	40cfed5bc9	[Feature] support eb5 video chunk (#4671)	2025-10-30 11:01:32 +08:00
gongshaotian	1081cad4a0	set default value as false	2025-10-29 15:51:50 +08:00
gongshaotian	fa85956c6f	add draft model using cudagraph switch	2025-10-29 15:51:50 +08:00
lizhenyun01	006c7e5a0d	[Feature] Support attention dp balance for mixed deployment (#4594) ... * [Feature] Support attention dp balance for mixed deployment * add abstractmethod * add config to rl * fix argument style	2025-10-29 15:23:51 +08:00
freeliuzc	c9be8762b6	[MTP]Merge support attn (#4591) ... * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import	2025-10-27 21:13:08 +08:00
GoldPancake	2cf0b0b715	[Feature] support mtp distribution equivalence verification (#4566) ... * support mtp distribution equivalence verification * fix bugs * add unit test	2025-10-27 11:01:28 +08:00
RAM	1a21e6c529	support mtp draft model with ep (#4581)	2025-10-27 09:34:54 +08:00
kevin	dd7fe27152	add hasher and ImagePosition	2025-10-23 15:20:21 +08:00
guozhuangzhuang	1531004085	fix image token output (#4487) ... * fix * fix * fix * add test case * add test case * add test case	2025-10-22 14:59:05 +08:00
李泳桦	ec499a0104	[Cherry-pick] fix requests & block metrics (#4500) ... * [fix] fix requests & block metrics * [chore] rename variables	2025-10-21 10:43:33 +08:00
ltd0924	3cd9d3060a	[Fearture] Support mm model close prefix cache (#4502) ... * support mm prefix cache close * add * fix * fix * fix --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-10-21 09:56:47 +08:00
GoldPancake	9c7187998c	[Feature] support mtp logprob (#4457) ... * support logprob in mtp * remove debug code * fix * feat: add draft_logprobs for Speculative Decode MTP * Revert "feat: add draft_logprobs for Speculative Decode MTP" This reverts commit d5a3c5c933. * fix * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix some bugs * fix codestyle * fix bugs * fix bugs * fix bugs * fix bus * fix bugs * fix unitest --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com> Co-authored-by: sunlei18 <sunlei18@sunlei18deMacBook-Pro.local>	2025-10-20 10:18:00 +08:00
RAM	920df5be5a	[Graph Optimization][Speculative Decoding] Fix the bug of CUDAGraph + MTP + EP (#4430) ... * Fix MTP dummy run bug * Target Model and Draft Model using the same flag * aovid moe bug in cudagraph padding * In mtp replace use_cudagraph as step_use_cudagraph	2025-10-17 14:22:05 +08:00
guozhuangzhuang	cfd93c0966	fix: image token output (#4399) ... * fix: image token output * fix: code style * fix: CompletionOutput.decode_type	2025-10-16 14:51:32 +08:00
Yuanle Liu	83f97d1196	support speculate_limit_thinking_content_length_v2 (#4428) ... * support speculate_limit_thinking_content_length_v2 * fix * fix import	2025-10-16 13:23:16 +08:00
gaoziyuan	74ae214f1a	fix ep perf (#4381)	2025-10-15 18:38:20 +08:00
freeliuzc	c3499875bd	[MTP]support mtp chunk_prefill_v1 (#4365) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output * fix mtp chunkprefill output, fix unit test * fix save_output --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-15 15:33:59 +08:00
ltd0924	dd425b89ed	[BugFix] fix cache port and zmq close bugs (#4371) ... * Update common_engine.py * Update zmq_client.py * Update expert_service.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-15 10:29:30 +08:00
yangjianfengo1	bc7193f21d	增加4合一视频选择 (#4372)	2025-10-15 09:58:24 +08:00
ltd0924	fa9a3eef4f	Update token_processor.py (#4395)	2025-10-15 09:43:28 +08:00
Jundong Liu	0b7a5778ab	[Executor]CUDAGraph support Speculate Decode (#4258) ... * [Executor]CUDAGraph support Speculate Decode * fix problem * solve problem * fix * fast compile * CUDAGraph + mtp support eb5(only target model) * Revert "fast compile" This reverts commit 3cfe8373ed. * fix precommit * solve comment * fix comment about #pragram unroll --------- Co-authored-by: gongshaotian <gstain5555@outlook.com> Co-authored-by: gongshaotian <gstian5555@outlook.com>	2025-10-13 15:21:41 +08:00
Zero Rains	07db281647	[Cherry-Pick][BugFix]fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD (#4252) ... * fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD * support dp	2025-10-13 10:18:53 +08:00
freeliuzc	8d629568f2	[MTP]fix speculate-decoding in dpep mode (#4351)	2025-10-11 17:16:57 +08:00
gaoziyuan	0c4c28d799	Update rollout_model.py (#4349)	2025-10-11 11:30:05 +08:00
ltd0924	28aa18bfc1	[Feature] support prefix cache + dp (#4356) ... * fix * fix * fix * [Feature] support clear data * update * fix * fix * fix * fix * [BugFix] fix clear data * Update api_server.py * Update api_server.py * [Feature] support fd decode response * Update engine.py * Update envs.py * Update expert_service.py * Update common_engine.py * [Feature] support prefix cache + ep * fix * add * Update expert_service.py * Update common_engine.py * Update common_engine.py * Update common_engine.py * Update common_engine.py * Update common_engine.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-10-11 10:02:08 +08:00
Yuanle Liu	3c9eedd562	Simplify CUDAGraph creation logic (#4298) ... * Simplify CUDAGraph creation logic Refactor CUDAGraph initialization to always use unique memory pool if configured. * Conditionally import CUDAGraph based on CUDA compilation	2025-10-10 10:46:16 +08:00
ltd0924	c35a21a99a	[Feature] support fd return decode response (#4300) ... * fix * fix * fix * [Feature] support clear data * update * fix * fix * fix * fix * [BugFix] fix clear data * Update api_server.py * Update api_server.py * [Feature] support fd decode response * Update engine.py * Update envs.py * Update expert_service.py * Update common_engine.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-09-28 16:11:50 +08:00
freeliuzc	c8985727a6	support mtp in hybird-dp-tp mode (#4299)	2025-09-28 15:58:45 +08:00
GoldPancake	076c30cb0f	fix top_p_candidates and support separate setting of sampling params for mtp (#4189) ... * fix top_p_candidates * For separate setting params for mtp * delete print * fix	2025-09-28 11:41:20 +08:00
ltd0924	f8c6a354a1	[BUGFIX] clear request (#4286) ... * fix * fix * fix * [Feature] support clear data * update * fix * fix * fix * fix * [BugFix] fix clear data * Update api_server.py * Update api_server.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-27 14:08:48 +08:00
freeliuzc	b176cba474	support mtp in ep64 (#4280)	2025-09-26 15:38:03 +08:00
Yuanle Liu	dcf633c4d9	delete default value reasoning_max_tokens (#4250) ... * delete default value reasoning_max_tokens * Adjust max_tokens and reasoning_max_tokens logic	2025-09-26 10:42:27 +08:00
Zhong Hui	213f15ef55	fix ernie vl distributed attr. (#4259)	2025-09-25 20:06:29 +08:00
lizhenyun01	bab779011c	[CudaGraph] support cudagraph use shared pool (#4199) ... * support cudagraph use shared pool * add envs * change CUDAGRAPH_POOL_ID to int * change CUDAGRAPH_POOL_ID to use_memory_pool * unify use_unique_memory_pool * fix use_unique_memory_pool	2025-09-24 21:32:04 +08:00
freeliuzc	e2b68b33c9	fix mtp in rl (#4234)	2025-09-24 16:59:24 +08:00
ltd0924	1aab1c8d06	[BugFix] fix clear data (#4227) ... * [Feature] support adapter * fix * fix * fix * fix * fix * fix * [BugFix] fix clear data * Update api_server.py	2025-09-24 11:23:44 +08:00
freeliuzc	94b6e7a341	[MTP][RL]support rl reshard wenxin-tools-145 (#4173) ... * support mtp reshard in rl mode * fix function	2025-09-23 20:40:26 +08:00
Yuanle Liu	389c5dd3a2	Each module should have its own plugins_loaded (#4149)	2025-09-23 15:44:46 +08:00