118 Commits

Author	SHA1	Message	Date
xiaoxiaohehe001	ee37882a26	[NewFeature] support eplb noaux (#4725) ... * support eplb noaux * support eplb noaux * add eplb noaux test	2025-11-05 20:59:12 +08:00
RAM	741a012d15	[Graph Optimization] cherry-pick other spec padding kernel (#4776) ... * cherry-pick orther spec kernel * refine code	2025-11-04 11:03:51 +08:00
freeliuzc	c9be8762b6	[MTP]Merge support attn (#4591) ... * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import	2025-10-27 21:13:08 +08:00
GoldPancake	2cf0b0b715	[Feature] support mtp distribution equivalence verification (#4566) ... * support mtp distribution equivalence verification * fix bugs * add unit test	2025-10-27 11:01:28 +08:00
Sunny-bot1	08853711b4	support static C8 (#4565)	2025-10-23 22:09:07 +08:00
RAM	07182103d3	Support CUDAGraph Padding + MTP (#4546)	2025-10-22 20:09:50 -07:00
GoldPancake	9c7187998c	[Feature] support mtp logprob (#4457) ... * support logprob in mtp * remove debug code * fix * feat: add draft_logprobs for Speculative Decode MTP * Revert "feat: add draft_logprobs for Speculative Decode MTP" This reverts commit d5a3c5c933. * fix * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix some bugs * fix codestyle * fix bugs * fix bugs * fix bugs * fix bus * fix bugs * fix unitest --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com> Co-authored-by: sunlei18 <sunlei18@sunlei18deMacBook-Pro.local>	2025-10-20 10:18:00 +08:00
RAM	920df5be5a	[Graph Optimization][Speculative Decoding] Fix the bug of CUDAGraph + MTP + EP (#4430) ... * Fix MTP dummy run bug * Target Model and Draft Model using the same flag * aovid moe bug in cudagraph padding * In mtp replace use_cudagraph as step_use_cudagraph	2025-10-17 14:22:05 +08:00
Yuanle Liu	83f97d1196	support speculate_limit_thinking_content_length_v2 (#4428) ... * support speculate_limit_thinking_content_length_v2 * fix * fix import	2025-10-16 13:23:16 +08:00
YUNSHEN XIE	0982dfb705	Cherry-Pick PR4033 (#4082) ... * [setup optimize]Support git submodule (#4033) * support git submodule * update setup * fix ci network * fix clone * revert clone linux * delete args * fix ci * update * fix --------- Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>	2025-10-15 22:09:02 +08:00
freeliuzc	c3499875bd	[MTP]support mtp chunk_prefill_v1 (#4365) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output * fix mtp chunkprefill output, fix unit test * fix save_output --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-15 15:33:59 +08:00
freeliuzc	bf12bee887	[MTP][Cfp8]supports spec dynamic cfp8 (#4290) (#4392) ... * supports spec dynamic cfp8 * supports spec dynamic cfp8 --------- Co-authored-by: lzy <569782149@qq.com>	2025-10-14 19:36:41 +08:00
Jundong Liu	0b7a5778ab	[Executor]CUDAGraph support Speculate Decode (#4258) ... * [Executor]CUDAGraph support Speculate Decode * fix problem * solve problem * fix * fast compile * CUDAGraph + mtp support eb5(only target model) * Revert "fast compile" This reverts commit 3cfe8373ed. * fix precommit * solve comment * fix comment about #pragram unroll --------- Co-authored-by: gongshaotian <gstain5555@outlook.com> Co-authored-by: gongshaotian <gstian5555@outlook.com>	2025-10-13 15:21:41 +08:00
freeliuzc	5035dd82ed	[MTP]support more branchs in topp kernel (#4353)	2025-10-11 11:27:35 +08:00
Nyakku Shigure	fd5fd0bdd7	Remove redundant inplace outputs for append_attention (#4341)	2025-10-10 10:45:26 +08:00
GoldPancake	076c30cb0f	fix top_p_candidates and support separate setting of sampling params for mtp (#4189) ... * fix top_p_candidates * For separate setting params for mtp * delete print * fix	2025-09-28 11:41:20 +08:00
Sunny-bot1	8a506500f3	[BugFix] Fix EP MoE prefill function (#4101)	2025-09-24 00:31:41 -07:00
Yuan Xiaolan	1e86418c4a	optimize dy_cfp8's performance (#4145) ... Co-authored-by: carryyu <569782149@qq.com>	2025-09-19 09:35:28 +08:00
Yuan Xiaolan	25aa2d94aa	cp dynamic Cfp8 (#4120) ... * supports dynamic Cfp8 * add unittest * fix dynamic Cfp8 computing error * fix Cfp8 for RL load --------- Co-authored-by: carryyu <569782149@qq.com>	2025-09-17 11:55:47 +08:00
gaoziyuan	b6caf6e622	suppoort expert num 3 pre rank (#4133)	2025-09-16 17:34:44 +08:00
chen	2485333f71	ep support logprob (#4089)	2025-09-12 21:11:16 +08:00
freeliuzc	2f473ba966	[Feature][MTP]Support MTP for rl-model (#4009) ... * qk norm for speculate decode C16 * support mtp in v1_scheduler mode * support mtp rope_3d * support mtp features * add unit test && del some log --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com> Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com>	2025-09-10 13:34:37 +08:00
yangjianfengo1	9213a58a06	【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (#3771) (#3835) ... * fix w4afp8 * 增加集中式配置 * codestyle * fix fa3 append attn	2025-09-03 19:36:45 +08:00
plusNew001	87ef0f5d30	[XPU] Update XPU stable xvllm and xtdk version for 2.2 & Change CI Case (#3855) ... * Update no_proxy environment variable in CI workflow * Install lsof and kill api_server processes Install lsof tool and kill processes using it. * Update dependency versions for stable release * Update CI script to use stable dependencies	2025-09-03 19:33:06 +08:00
lizhenyun01	bed09ae8f8	fix mask_offset in append_attn (#3745) ... * fix mask_offset in append_attn * fix test	2025-08-31 15:03:16 +08:00
Sunny-bot1	fe5d09f9ee	[FIX]Fix Machete compile via ENABLE_MACHETE (#3727) ... * add ENABLE_MACHETE * fix * revert * update * pre_commit * fix * fix --------- Co-authored-by: Ayakouji <yuhongh@qq.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: aquagull <hongyuh@qq.com>	2025-08-30 17:50:17 +08:00
yangjianfengo1	3754a9906d	[Feature] block sparse attention (#3668) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config * 修复显存 * add test server * add test server * fix mlp 最后一层使用full attn	2025-08-29 19:46:30 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
Ryan	45f81b34f0	add dtype int32 (#3692)	2025-08-29 14:56:35 +08:00
co63oc	b6edd15d55	fix scaled_gemm_f8_i4_f16_weight_quantize input (#3685)	2025-08-29 11:04:04 +08:00
lifulll	72094d4d82	enable dcu ci (#3402)	2025-08-29 10:23:08 +08:00
Liumengyuan	e93d4cfcdd	Add with_output version AppendAttention (#3302) ... * get use_output from fd_config * add clear TODO description * add mask_offset para to align with develop * fix bug * fix use_output logic * fix sot bug	2025-08-28 17:10:18 +08:00
yangjianfengo1	e81046fdad	【New Feature】集中式支持w4afp8 (#3644) ... * 支持tp w4afp8 * code style	2025-08-28 10:53:24 +08:00
周周周	76513f6416	Support 45t fp8 8 GPU (#3659)	2025-08-28 10:52:53 +08:00
Sunny-bot1	479c8b85d3	[Optimize]support machete weight only gemm (#3561) ... * support machete weight only gemm * add generate * update * fix * change file location * add sm_version limit * fix * fix * fix ci * fix coverage * fix xpu	2025-08-28 09:49:58 +08:00
Liumengyuan	11803e0907	fix undefined cuPointerGetAttribute symbol error (#3628)	2025-08-27 20:24:59 +08:00
Jiang-Jia-Jun	c694fa2879	Revert "[Feature] block sparse attention (#3209)" (#3647) ... This reverts commit 646a0c2fd8.	2025-08-27 17:35:04 +08:00
xiaoxiaohehe001	ad319a87cc	support fa3 rope3d (#3622)	2025-08-27 11:31:29 +08:00
yangjianfengo1	646a0c2fd8	[Feature] block sparse attention (#3209) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config	2025-08-26 07:16:04 -07:00
gaoziyuan	82e64b13e1	[NewFeature]Support dp multi api server && Fix some bug in mixed ep && merge develop (#3598) ... * [Feature] update ep * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix queue ports idx * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * Update engine.py * fix ci * fix some bug in mixed ep * add server fix and op fix * rm some log * fix code style * ltd fix * fix * fix * fix some bug * fix bug * fix bug * fix style * Update config.py * Update splitwise_connector.py * Update cache_messager.py * Update __init__.py * merge and fix * Update engine.py * Update common_engine.py * Update run_ci_xpu.sh * Update ernie_processor.py * Update ernie_processor.py --------- Co-authored-by: ltd0924 <ltd0924@sina.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com>	2025-08-26 19:59:02 +08:00
lzy	d339df2e90	Supports DP+TP+EP hybrid parallel deployment strategy (#3489) ... * Support DP+TP+EP hybrid parallel deployment strategy * Support DP+TP+EP hybrid parallel deployment strategy * fix conflict * add moe_tp_ep function split_allgather_out * del tp_group in moe_cutlass_backend * for ci * fix parallel_config for ci * del log	2025-08-26 00:04:01 -07:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
xiaoxiaohehe001	70c75798a7	[NewFeatures] support noex rope3d (#3542) ... * [NewFeatures] support noex rope3d * [NewFeatures] support noex rope3d encoder	2025-08-26 11:44:57 +08:00
Kane2011	2ae7ab28d2	[MetaxGPU] adapt to the latest fastdeploy on metax gpu (#3492)	2025-08-25 17:44:20 +08:00
Yuan Xiaolan	9205c88da1	support w4afp8 EP inference (#3044)	2025-08-25 11:27:45 +08:00
lengxia	137e539456	[Feature][XPU] add custom kernels for mtp (#3537)	2025-08-25 10:14:17 +08:00
freeliuzc	76759108c9	[Feature][SpeculativeDecoding]Support tree-attention (#3514) ... * support tree-attention * fix merge bug * fix unit-test api * fix merge bug	2025-08-22 13:36:41 +08:00
yangjianfengo1	e5aa7087db	【bug fix】修复w4a8编译慢 (#3510) ... * 修复w4a8编译 * code style * 修复tma copy	2025-08-21 18:50:14 +08:00
Ryan	bcdfc1d6b9	Add custom op declaration for all_reduce (#3473) ... * add custom op declaration * roll back try except	2025-08-20 20:29:58 +08:00
yangjianfengo1	b047681c5d	【New Feature】支持Fp8 group Gemm 24稀疏 (#3463) ... * 支持24稀疏 * code style * 增加stmatrix 宏定义判断 * code style	2025-08-19 02:54:47 -07:00