151 Commits

Author	SHA1	Message	Date
Sunny-bot1	a751d977bc	[Optimization] Fuse get_max_len and get_kv_max_len (#4369) ... * opt split_q_block * fuse max_lens and max kv_len	2025-10-13 20:35:00 +08:00
freeliuzc	365601ea5a	[MTP]support more branchs in topp kernel (#4352)	2025-10-11 11:33:52 +08:00
AIbin	f7eaca3971	【Bug Fix】mla enables tensorcore by default (#4354) ... * mla tensor-core kernel is enabled by default	2025-10-10 20:45:16 +08:00
lzy	3aa04fbf21	[MTP][Cfp8]supports spec dynamic cfp8 (#4290) ... * supports spec dynamic cfp8 * supports spec dynamic cfp8 --------- Co-authored-by: freeliuzc <lzc842650834@gmail.com>	2025-10-10 16:08:10 +08:00
yinwei	20c7b741f4	[XPU] Support W4A8C8-TP4-300B Model (#4068) ... * support w4a8 * delete ep block attn * delete moe_topk_select * update note * update * delte useless info * update * add some note * fix some format * update scale info * add ans baseline --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-10 15:41:32 +08:00
Zhenghai Zhang	c46d5e48f8	【Hackathon 9th No.86】autogen MultiQueryAppendC8Attention template_instantiation -part (#4330) ... * split MultiQueryAppendC8Attention template_instantiation * update setup_ops.py * fix ci * fix bug	2025-10-10 15:07:48 +08:00
AIbin	c4ebaf8a07	【Inference Optimize】MLA Tensor-Core is enabled by default (#4335)	2025-10-10 10:54:56 +08:00
Nyakku Shigure	5f80862578	Remove redundant inplace outputs for append_attention (#4340)	2025-10-10 10:21:27 +08:00
RAM	aa27b03bc0	[Executor]CUDAGraph support Speculate Decode (#3769) ... * success run ngram * Revert "[Code Simplification] remove cum_offsets (#3410)" This reverts commit 32b39620bc. * success run ngram5 tp4 42bs * success run ngram5 tp4 42bs * mtp draft commit * add decorator for target model * enable draft model in cudagraph v0.5 * revert revrt cum_offset * enable target model in cudagraph v0.9 And clean debug code * Revert "success run ngram" This reverts commit 8351e83993. * add reverted code * enable target model in cudagraph v0.9 * solve comment * fix bid < 0 * Enable Target Model Padding And Draft Model in cudagraph * solve problem * delete rebuild padding debug note * fast compile * Add capture list for mtp * success run 256 tp1 mtp * Enable Lite TP2 Bsz256 * realy enable tp2 bsz 256 * fix problem * Solve problem for Draft model in cudagraph * Solve comment * replace emptytensor as zeros * Solve comments * Revert "fast compile" This reverts commit 834639a7ff. * fix bug * fix merge bug * fix typo * fix bug --------- Co-authored-by: lizexu <2694294196@qq.com> Co-authored-by: littledgg <1658565283@qq.com> Co-authored-by: zeroRains <linjunlu@zerorains.top> Co-authored-by: gongshaotian <gstain5555@outlook.com>	2025-10-09 21:18:29 +08:00
fangfangssj	af3872215e	[BugFix]remove redundant includes (#4312) ... Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-09-30 17:54:19 +08:00
chen	81959c7d88	[NewFeature]custom_allreduce support cudagraph recapture (#4305) ... * custom_allreduce support cudagraph recapture * add shut_down/restart default group	2025-09-29 15:56:54 +08:00
xiaozude	7c919070f7	[Metax] support cutlass moe & optimize flash attention (#4208)	2025-09-29 11:22:43 +08:00
李泳桦	6265f4385f	[feat] support prefix cache clearing when /clear_load_weight is called (#4008) ... * [feat] support clearing prefix cache (cherry-picked from release/2.1) * [fix] fix ipc suffix, use port instead * [fix] fix prefix caching not enabled * [fix] fix key/value_cache_scales indent * [fix] fix ep group all-reduce * [fix] fix clear/update lock not working when workers > 1 * [chore] add preemption triggered info log * [fix] fix code style * [fix] fix max_num_seqs config * [fix] do not force enable_prefix_caching=False in dynamic loading * [fix] fix ci * Revert "[fix] fix ci" This reverts commit 0bc6d55cc8. * [fix] initialize available_gpu_block_num with max_gpu_block_num * [fix] fix config splitwise_role * [fix] fix clearing caches synchronization and add more logs * [chore] print cache_ready_signal in log * [fix] fix scheduler_config.splitwise_role * [fix] fix cache_messager cache_ready_signal create=True * [fix] stop cache messager from launching in mixed deployment	2025-09-28 19:42:53 +08:00
Zhenghai Zhang	aa045aa84f	fix typos (#4274)	2025-09-27 09:25:43 +08:00
Lucas	87179cb744	[XPU] support XPU VL model inference (#4030) ... * [XPU] support XPU VL model inference * fix image op import and device check * rebase develop * fix perf	2025-09-25 14:34:15 +08:00
chen	7c1fd19f0f	[OPs] MoE support wfp8afp8(channelwise) and improve per_token_quant_fp8 (#4238)	2025-09-24 16:39:51 +08:00
Yohanna	44010cee13	FIX] Fix CUDA error(700): 'cudaErrorIllegalAddress' in CascadeAppendWriteCacheKVQKV cache_kernel(). Continue when batch_id_per_token[token_idx] is default value -1. (#4218)	2025-09-24 14:08:49 +08:00
fmiao2372	f1b5392e20	[Intel HPU] Support intel hpu platform (#4161) ... * [Intel HPU] Support intel hpu platform * fix some issues * apply precommit and move AttentionBackend_HPU * fix format issue * correct ops import * fix ci issue * update code in layers * fix code style issue * remove dense tp moe ep mode * fix enc_dec_block_num * fix rebase issue * rename hpu to gaudi in readme * rename ForwardMeta_HPU to HPUForwardMeta	2025-09-24 12:27:50 +08:00
yyssys	d6e59447f5	[XPU] Enable XPU V1 mode based on environment variable (#4213) ... * Enable XPU V1 mode based on environment variable * add default param to xft_moe_fc_block_eb for latest xvllm compatibility; update run_ci_xpu to use latest xvllm	2025-09-24 10:29:48 +08:00
chen	1a6283424e	Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)	2025-09-23 18:41:33 +08:00
yzwu	504461b6b5	[Iluvatar GPU] Optimize attention performance and fix moe load ckpt error (#3651)	2025-09-22 21:13:59 +08:00
chenjian	918ccdb123	[Feature] Support pd ep deployment with yiyan adapter (#4029) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * Support pd ep deployment with yiyan adapter * Support pd ep deployment with yiyan adapter * refactor cache messager * support scheduler v1 in PD * suppport pd v1 + chunk prefill * suppport pd v1 + chunk prefill * add eplb * support eplb * support eplb * support eplb * support v1 * fix * fix * fix bug * remove eplb support * support prefix cache in P * fix bug * fix bug * support one stop in V1 * fix bug * fix ci * fix ci * fix * fix * fix * fix * fix --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-22 16:41:38 +08:00
co63oc	c4830ef24c	fix typos (#4176) ... * fix typos * fix	2025-09-22 14:27:17 +08:00
chen	66a98b44ed	ep support logprob (#4089) (#4151)	2025-09-19 14:07:31 +08:00
gaoziyuan	896e3bb606	[NewFeture]add ep rollout model init and update/clear ep buffer (#4039) ... * fix gid * merge * fix test * fix bug * fix * fix ci	2025-09-17 20:24:53 +08:00
Yuan Xiaolan	de8638b1e9	fix dynamic Cfp8 computing error (#4119) ... Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-16 20:21:49 +08:00
Sunny-bot1	442543cd6b	fix ep wint8 (#4102)	2025-09-16 11:05:33 +08:00
co63oc	17a27170bc	fix typos (#4093)	2025-09-15 18:33:30 +08:00
zhupengyang	9409665713	[xpu] support ep (#4067)	2025-09-15 13:53:11 +08:00
Sunny-bot1	b1a5b756a3	[Optimize] Support WINT8 and group scale for Machete (#3905)	2025-09-15 12:01:34 +08:00
co63oc	8466219ec8	fix typos (#3840) ... * fix typos * ci --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-12 11:04:38 +08:00
YuanRisheng	d2d04c2d5e	[setup optimize]Support git submodule (#4033) ... * support git submodule * update setup * fix ci network * fix clone * revert clone linux * delete args * fix ci * update	2025-09-11 17:41:16 +08:00
AIbin	a7392a0ff9	【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886) ... * support MLA chunk_size auto search & cuda_graph	2025-09-11 10:46:09 +08:00
chen	637d96c6ae	[Feature] Support zai-org/GLM-4.5-Air BF16 model (#3928) ... * support glm45_air	2025-09-10 19:36:10 +08:00
freeliuzc	7ee100903f	support rope_3d in spec mode (#4034)	2025-09-10 03:15:05 -07:00
Ayakouji	453487d5b0	[Feat] ernie4_5_vl_moe support CudaGraph (#3226) ... * delete dynamic control flow for decode * coda-style * fix scatter/gather typos and use input stream instead default stream * support 0-Size Tensor * update runner and model * using static mem address as input * fix mem leak * refine code * update mm_buffer * fix typo * fix buffersize * fix unk token * refine code * refine * support other arch * open cudagraph in vlci * fix * update * update * update * fix cmd * update --------- Co-authored-by: aquagull <hongyuh@qq.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-10 13:11:57 +08:00
zhupengyang	9d0074a91a	[xpu] add ep custom ops (#3911)	2025-09-10 12:22:50 +08:00
周周周	dbab579299	clean code (#4020)	2025-09-10 10:56:15 +08:00
lzy	f12159b630	del batch id per token (#3963) ... * Update decoder_write_cache_with_rope_kernel.cu del batch_id_per_token * Update decoder_write_cache_with_rope_impl.cuh * Update test_append_attention.py * Update test_append_attention.py	2025-09-08 21:58:34 +08:00
co63oc	aadd6a94d8	fix typos (#3951)	2025-09-08 15:22:41 +08:00
co63oc	2033450391	rename ep_moe_prefill_func ep_moe_expert_dispatch (#3938)	2025-09-08 15:19:28 +08:00
Jundong Liu	3d0aaa5923	[Excutor] Experiment Feature-Support Prefill in cudagraph (#3459) ... * Support prefill in Cudagraph * Refactor GetBlockShapeAndSplitKVBlock Kernel V2 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.1 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.2 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.3 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.4 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.5 * Solve problem about encoder_num_blocks_x_cpu * Add early-exit mechanism for attention kernel * fix test case about append-attention * Update testcode, Add annotations to related tensors * move get_input_length_list * solve test_code * Add annotations about early-exit for attention kernel * Add annotations about early-exit for attention kernel2 * solve comment * solve mtp --------- Co-authored-by: RAM <gstian5555@outlook.com>	2025-09-08 13:12:24 +08:00
lzy	af49b81ffd	supports dynamic Cfp8 (#3767) ... * supports dynamic Cfp8 * add unittest	2025-09-07 20:41:29 -07:00
co63oc	30a1c1783f	rename eagle_get_base_model_hidden_states.cu (#3753)	2025-09-07 10:24:58 +08:00
周周周	f6f726c773	clean code in sttantion (#3917)	2025-09-05 20:49:01 +08:00
Yuan Xiaolan	2cf55168ca	load hadamard_block_size from config (#3797)	2025-09-05 17:07:58 +08:00
freeliuzc	88d44a2c93	support mtp in v1_scheduler mode (#3695)	2025-09-04 17:39:59 +08:00
xiaoxiaohehe001	f265a26f8b	support mtp rope_3d (#3791) ... * support mtp rope_3d * Update speculate_write_cache_with_rope_kernel.cu	2025-09-04 17:18:05 +08:00
plusNew001	3790505319	[XPU] Update XPU stable xvllm and xtdk version for 2.2 (#3853) ... * Add debug environment variable exports Added debug environment variable exports for CLANG_PATH and XVLLM_PATH. * Lock paddlepaddle-xpu version in CI script Temporarily lock paddlepaddle-xpu version due to framework update issues. * Update no_proxy environment variable in CI workflow * Install lsof tool in run_ci_xpu.sh * Update dependency versions for stable release * Update paddlepaddle-xpu installation command	2025-09-03 23:21:00 +08:00
lizexu123	4c998c3636	[Code Simplification] delete cum_offsets_out (#3815) ... * fix * fix	2025-09-03 16:15:33 +08:00