164 Commits

Author	SHA1	Message	Date
Yuan Xiaolan	1e88754133	support set dy-C8 from args (#4475)	2025-11-04 17:01:35 +08:00
kevin	c785c2dab1	[Scheduler] update v1 prefill batch (#4563) ... * update v1 scheduler * update code * update code * update code * update code * remove mm prefill batch * update code * fix bug * update code	2025-10-31 14:03:18 +08:00
gongshaotian	1081cad4a0	set default value as false	2025-10-29 15:51:50 +08:00
lizhenyun01	006c7e5a0d	[Feature] Support attention dp balance for mixed deployment (#4594) ... * [Feature] Support attention dp balance for mixed deployment * add abstractmethod * add config to rl * fix argument style	2025-10-29 15:23:51 +08:00
freeliuzc	c9be8762b6	[MTP]Merge support attn (#4591) ... * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import	2025-10-27 21:13:08 +08:00
GoldPancake	9c7187998c	[Feature] support mtp logprob (#4457) ... * support logprob in mtp * remove debug code * fix * feat: add draft_logprobs for Speculative Decode MTP * Revert "feat: add draft_logprobs for Speculative Decode MTP" This reverts commit d5a3c5c933. * fix * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix some bugs * fix codestyle * fix bugs * fix bugs * fix bugs * fix bus * fix bugs * fix unitest --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com> Co-authored-by: sunlei18 <sunlei18@sunlei18deMacBook-Pro.local>	2025-10-20 10:18:00 +08:00
RAM	920df5be5a	[Graph Optimization][Speculative Decoding] Fix the bug of CUDAGraph + MTP + EP (#4430) ... * Fix MTP dummy run bug * Target Model and Draft Model using the same flag * aovid moe bug in cudagraph padding * In mtp replace use_cudagraph as step_use_cudagraph	2025-10-17 14:22:05 +08:00
freeliuzc	c3499875bd	[MTP]support mtp chunk_prefill_v1 (#4365) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output * fix mtp chunkprefill output, fix unit test * fix save_output --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-15 15:33:59 +08:00
Jundong Liu	0b7a5778ab	[Executor]CUDAGraph support Speculate Decode (#4258) ... * [Executor]CUDAGraph support Speculate Decode * fix problem * solve problem * fix * fast compile * CUDAGraph + mtp support eb5(only target model) * Revert "fast compile" This reverts commit 3cfe8373ed. * fix precommit * solve comment * fix comment about #pragram unroll --------- Co-authored-by: gongshaotian <gstain5555@outlook.com> Co-authored-by: gongshaotian <gstian5555@outlook.com>	2025-10-13 15:21:41 +08:00
Zero Rains	07db281647	[Cherry-Pick][BugFix]fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD (#4252) ... * fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD * support dp	2025-10-13 10:18:53 +08:00
ltd0924	f8c6a354a1	[BUGFIX] clear request (#4286) ... * fix * fix * fix * [Feature] support clear data * update * fix * fix * fix * fix * [BugFix] fix clear data * Update api_server.py * Update api_server.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-27 14:08:48 +08:00
lzy	be98f6e950	supports internode_ll_two_stage (#4143) ... * supports internode_ll_two_stage * supports internode_ll_two_stage * supports internode_ll_two_stage * supports internode_ll_two_stage	2025-09-22 14:55:06 +08:00
ltd0924	f75697c2d1	[Feature] support clear data (#4185) ... * fix * fix * fix * [Feature] support clear data * update * fix * fix * fix * fix	2025-09-21 20:41:27 +08:00
Yuan Xiaolan	25aa2d94aa	cp dynamic Cfp8 (#4120) ... * supports dynamic Cfp8 * add unittest * fix dynamic Cfp8 computing error * fix Cfp8 for RL load --------- Co-authored-by: carryyu <569782149@qq.com>	2025-09-17 11:55:47 +08:00
Yuanle Liu	2883746132	fix model_weights_signal (#4092) ... * fix model_weights_signal	2025-09-13 11:55:25 +08:00
gaoziyuan	10768a4d79	[NewFeture]add ep rollout model init and update/clear ep buffer (#3927) ... * add ep rollout model init && add deep update/clear * fix test	2025-09-12 14:15:13 +08:00
RAM	63d24b2210	[Executor] Adjust signal sending order in RL training (#3773) (#4066) ... * Adjust processing order * fix bug * fix update_parameters bug * refine code	2025-09-11 15:41:32 +08:00
freeliuzc	2f473ba966	[Feature][MTP]Support MTP for rl-model (#4009) ... * qk norm for speculate decode C16 * support mtp in v1_scheduler mode * support mtp rope_3d * support mtp features * add unit test && del some log --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com> Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com>	2025-09-10 13:34:37 +08:00
Zero Rains	d43549953c	[Cherry-Pick][Bug Fix]fix the bug for real size 0 in cudagraph (#3888) ... * fix the bug for real size 0 in cudagraph * fix cache_messager --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:06:10 +08:00
chenjian	8d77c1cb51	[Optimize] optimize prefix cache in release22 (#3889) ... * optimize prefix cache in release22 * optimize prefix cache in release22 * fix worker * fix * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-06 09:52:01 +08:00
chenjian	41cd3e24c9	[Feature] Enable prefix caching as default (#3816) ... * [Feature] Enable prefix caching as default * [Feature] Enable prefix caching as default * Set prefix caching as default * skip dynamic load * fix kill bug * fix kill bug * fix kill bug * fix ci * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-06 09:51:34 +08:00
yinwei	77c1bd0813	[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER (#3900) ... * fix bug * fix bug * update * udpate * update	2025-09-05 19:17:25 +08:00
chenjian	8567ada09e	[Fix] disable scheduler v1 in guided decoding (#3877) ... * disable scheduler v1 in guided decoding * disable scheduler v1 in guided decoding	2025-09-04 20:54:55 +08:00
lizhenyun01	d40d3a5a4f	fix DP&&TP (#3872)	2025-09-04 14:38:26 +08:00
chenjian	a0c03510c0	[Bug fix] Fix prompt token ids dtype in v1 (#3861)	2025-09-04 11:02:37 +08:00
chenjian	fb1e0d6a87	[Feature] Set scheduler v1 as default (#3812) ... * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default	2025-09-04 11:02:10 +08:00
ltd0924	0f42771a84	[Feature] support model weight update in ep (#3802) ... * Update config.py * Update ep.py * Update fused_moe_backend_base.py * Update dynamic_weight_manager.py * Update worker_process.py * fix ci	2025-09-02 20:52:47 +08:00
yangjianfengo1	3754a9906d	[Feature] block sparse attention (#3668) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config * 修复显存 * add test server * add test server * fix mlp 最后一层使用full attn	2025-08-29 19:46:30 +08:00
zhouchong	ccd52b5596	[Model]support qwen2_5_vl (#3557) ... * adapt qwen_2_5_vl model * adapt qwen_2_5_vl VIT model * adapt qwen2_5_vl images_embeds * adapt qwen2_5_vl 3D rope * adapt qwen2_5_vl 3D rope v2 * adapt qwen2_5_vl processor * adapt qwen2_5_vl bypass resampler_model * adapt qwen2_5_vl 绕过部分ernie逻辑 * adapt qwen2_5_vl 绕过部分ernie逻辑 v2 * adapt qwen2_5_vl 权重加载与命名修改 * adapt qwen2_5_vl 非必须think_end_id * adapt qwen2_5_vl 区分多种模型的extract_vision_features * fix:adapt qwen2_5_vl model * adapt qwen2_5_vl norm * adapt qwen2_5_vl processor 更新 * adapt qwen2_5_vl image and video success * adapt qwen2_5_vl 部分整理代码 * adapt qwen2_5_vl 支持多卡 * adapt qwen2_5_vl on latest develop * adapt qwen2_5_vl RL * adapt qwen2_5_vl 整理代码 * support noex rope3d * adapt qwen2_5_vl add init.py * adapt qwen2_5_vl add init.py v2 * adapt qwen2_5_vl remove space * adapt qwen2_5_vl remove space v2 * adapt qwen2_5_vl pre-commit * adapt qwen2_5_vl update * adapt qwen2_5_vl pre-commit v2 * adapt qwen2_5_vl modify comments * adapt qwen2_5_vl fix indentation * adapt qwen2_5_vl fix indentation v2 --------- Co-authored-by: wangyafeng <wangyafeng@baidu.com> Co-authored-by: xiaoxiaohehe001 <49090790+xiaoxiaohehe001@users.noreply.github.com> Co-authored-by: CSWYF3634076 <58356743+CSWYF3634076@users.noreply.github.com>	2025-08-29 18:28:39 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
Ryan	45f81b34f0	add dtype int32 (#3692)	2025-08-29 14:56:35 +08:00
lifulll	72094d4d82	enable dcu ci (#3402)	2025-08-29 10:23:08 +08:00
Yuanle Liu	4957908275	add input_processor plugin (#3657) ... * add input_processor plugin * update * update * update * update * update * update * update * update * update * update * update	2025-08-28 22:53:57 +08:00
ming1753	02b3644903	[Bug Fix] VL Support w4a8/w4afp8 (#3686)	2025-08-28 21:38:35 +08:00
gaoziyuan	fc635acc47	[BugFix]fix dp&ep&tp and muti node infer (#3629) ... * rm log * fix bug * fix bug * fix dp&ep&tp and muti node infer * fix --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-08-28 19:09:10 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
lizexu123	b28a0343a6	fix ENABLE_V1_KVCACHE_SCHEDULER (#3625) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-27 21:21:29 +08:00
Jiang-Jia-Jun	c694fa2879	Revert "[Feature] block sparse attention (#3209)" (#3647) ... This reverts commit 646a0c2fd8.	2025-08-27 17:35:04 +08:00
李泳桦	b2afdf4fc6	[fix] qwen output inconsistency when top_p=0 (#3634) ... * [fix] qwen output inconsistency when top_p=0 * [fix] remove decode pre_id code	2025-08-27 17:16:23 +08:00
chen	ce9c0917c5	[Precision] Support lm_head layer running in float32 (#3597) ... * support lm_head fp32 bf16 fp16 * support lm_head fp32 bf16 fp16 * add doc and check code * lm_head_fp32 specify lm_head as fp32 * code check * check doc	2025-08-27 11:34:53 +08:00
yangjianfengo1	646a0c2fd8	[Feature] block sparse attention (#3209) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config	2025-08-26 07:16:04 -07:00
gaoziyuan	82e64b13e1	[NewFeature]Support dp multi api server && Fix some bug in mixed ep && merge develop (#3598) ... * [Feature] update ep * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix queue ports idx * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * Update engine.py * fix ci * fix some bug in mixed ep * add server fix and op fix * rm some log * fix code style * ltd fix * fix * fix * fix some bug * fix bug * fix bug * fix style * Update config.py * Update splitwise_connector.py * Update cache_messager.py * Update __init__.py * merge and fix * Update engine.py * Update common_engine.py * Update run_ci_xpu.sh * Update ernie_processor.py * Update ernie_processor.py --------- Co-authored-by: ltd0924 <ltd0924@sina.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com>	2025-08-26 19:59:02 +08:00
Yuanle Liu	cbce94a00e	rename ernie_xxx to ernie4_5_xxx (#3621) ... * rename ernie_xxx to ernie4_5_xxx * ci fix	2025-08-26 19:29:27 +08:00
bukejiyu	3200a80de3	[v1 loader]support fp8 (#3593) ... * support fp8 * update ci	2025-08-26 02:42:46 -07:00
lzy	d339df2e90	Supports DP+TP+EP hybrid parallel deployment strategy (#3489) ... * Support DP+TP+EP hybrid parallel deployment strategy * Support DP+TP+EP hybrid parallel deployment strategy * fix conflict * add moe_tp_ep function split_allgather_out * del tp_group in moe_cutlass_backend * for ci * fix parallel_config for ci * del log	2025-08-26 00:04:01 -07:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
RAM	2fa173e327	[Executor] CUDAGraph support RL training (#3265) ... * add clear graph opt backend * cuda graph support rl * add branch * 1.fix dynamic_weight_manager bug 2.add clear api for CasualLM * open test case * fix typo * update mkdocs.yaml * [Docs]Update mkdocs.yml * update test case * use unittest in graph test case	2025-08-25 20:59:30 +08:00
Kane2011	2ae7ab28d2	[MetaxGPU] adapt to the latest fastdeploy on metax gpu (#3492)	2025-08-25 17:44:20 +08:00
chen	9cab3f47ff	[Feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing (#3552) ... * [feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing * infer engine support temp_scaled_logprobs and top_p_normalized_logprobs * delete some code * code check * code check and add doc * fix tokenizer.decoder(-1), return 'Invalid Token' * add ci for temp_scaled and top_p logprobs * check test * check seq len time shape * logprob clip inf --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com>	2025-08-25 14:11:49 +08:00
zhink	df7c31012b	Modified to support custom all reduce by default (#3538)	2025-08-22 16:59:05 +08:00