164 Commits

Author	SHA1	Message	Date
ming1753	cba185f1fe	[Feature] Optim PaddleOCR-VL (#4873) ... * [Feature] Optim PaddleOCR-VL * fix bug	2025-11-07 14:56:44 +08:00
YuBaoku	819b2dbbae	Revert "【New Feature】W4afp8 supports per group quantization (#4272)" (#4854) ... This reverts commit 93fcf7e4ec.	2025-11-06 17:48:28 +08:00
yangjianfengo1	93fcf7e4ec	【New Feature】W4afp8 supports per group quantization (#4272) ... * w4afp8 支持per group * code style * 精度完成 * revert append attn utils * ffn1 动态量化 * ffn2 支持动态量化 * code style * code style * 修改单测 * 修改单测 * fix bug * Implement conditional parameter creation for layers Add parameter creation for up_gate_proj_in_scale when ep_size > 1. * code style * fix conflict * code style * code style * 修复w4aint8 精度 * fix ci --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com>	2025-11-05 21:00:23 +08:00
周周周	937eb3c6ed	[get_padding_offset.] clean get_padding_offset.cu (#4777) ... [get_padding_offset.] clean get_padding_offset.cu (#4777)	2025-11-05 10:47:40 +08:00
freeliuzc	11398790d3	[Speculative Decoding][MTP]Support attn mask offset (#4641) ... * [MTP]Merge support attn (#4591) * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import * support attn_mask_offset in mtp mode * add update_attn_mask op * fix unit test && fix code-style	2025-11-03 10:08:01 +08:00
freeliuzc	f44f4bafd1	support mtp in splitewise and scheduler_v1 mode (#4743)	2025-11-03 10:07:15 +08:00
周周周	6e01be28e0	format code (#4720)	2025-11-01 19:13:50 +08:00
Yuanle Liu	b301bd6c31	[BugFix] fix thinking bug (#4710) ... * fix thinking bug * fix ut * update * fix	2025-10-31 22:00:31 +08:00
周周周	10358bf1a0	fix noaux (#4731)	2025-10-31 21:25:11 +08:00
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
周周周	0089287534	[noauxtc_kernel] remove useless code (#4643) ... * remove num_tokens * remove num_tokens * false * final commit	2025-10-30 18:59:04 +08:00
Zhenghai Zhang	1712e1351b	【Hackathon 9th No.86】autogen MoeFastHardamardImplWrapper template_instantiation (#4592) ... * autogen MoeFastHardamardImplWrapper template_instantiation * fix codestyle * fix codestyle * add impl cu files	2025-10-30 10:28:36 +08:00
Ryan	e25c067f70	[OP] Add InferShape&InferDtype for per_token_quant_padding (#4667) ... * add InferShape&InferDtype for per_token_quant_padding * fix codestyle	2025-10-30 10:28:26 +08:00
Ryan	07956a87b3	[Graph Optimization] Fix IR graph dependency error exposed after enabling SOT by updating the return value of TextImageGatherScatter (#4610) ... * fix TextImageGatherScatter in sot * fix codestyle	2025-10-28 18:31:23 +08:00
RAM	86d5006a57	[Graph Optimization][Speculative Decoding] Update yaml and fix typo (#4612)	2025-10-28 11:43:26 +08:00
Lucas	5c6105f4a2	[XPU] bind some OPs for VL model with pybind (#4522)	2025-10-27 10:50:08 +08:00
xiaozude	f7069b8057	[Metax] adapt DeepSeek (#4498)	2025-10-24 10:14:53 +08:00
Sunny-bot1	8718fa34b2	support static C8 (#4568)	2025-10-23 22:01:03 +08:00
RAM	9dc5c3e370	[Graph Optimization] Support CUDAGraph Padding + MTP (#4545) ... * Support CUDAGraph Padding + MTP * support orther write cache kernel	2025-10-23 20:57:26 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
Yuanle Liu	3b58310c26	enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525) ... * fix custom_ops * paddleformers>=0.3.1	2025-10-21 22:06:06 +08:00
RAM	775edcc09a	[Executor] Default use CUDAGraph (#3594) ... * add start intercept * Adjustment GraphOptConfig * pre-commit * default use cudagraph * set default value * default use cuda graph * pre-commit * fix test case bug * disable rl * fix moba attention * only support gpu * Temporarily disable PD Disaggregation * set max_num_seqs of test case as 1 * set max_num_seqs and temperature * fix max_num_batched_tokens bug * close cuda graph * success run wint2 * profile run with max_num_batched_tokens * 1.add c++ memchecker 2.success run wint2 * updatee a800 yaml * update docs * 1. delete check 2. fix plas attn test case * default use use_unique_memory_pool * add try-except for warmup * ban mtp, mm, rl * fix test case mock * fix ci bug * fix form_model_get_output_topp0 bug * fix ci bug * refine deepseek ci * refine code * Disable PD * fix sot yaml	2025-10-21 14:25:45 +08:00
Yuanle Liu	cef3164c3b	Optimizing the performance of think length limit using custom operators (#4279) ... * delete impl * delete min_length&max_length * support limit thinking content strategy * fix * fix * fix * update * fix set_value_by_flags_and_idx * fix * fix * fix * fix * update * fix * fix * fix typo * fix ci * fix * fix * support mtp * fix * fix * update * update	2025-10-20 21:09:13 +08:00
GoldPancake	47595a2480	[Feature] support mtp logprob (#4464) ... * support mtp logprob * fix unitest	2025-10-20 15:18:12 +08:00
Haonan Luo	1b9f351d21	Support GPT-OSS-BF16 (#4240) ... * [Feature] AppendAtten support sinks & HEAD_DIM=64 * fix bug * fix bug * fix bug * fix bug * [Feature] support gpt-oss * fix bug * add mask * support-gpt-oss * support-gpt-oss * fix long seq * support wint8 * support wint8 * support wint8 * update test * change sliding windows init pos --------- Co-authored-by: ming1753 <ideaminghp@163.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com>	2025-10-20 14:44:58 +08:00
周周周	817210e47f	[ATTN]delete code and add ffn and moe layer level test (#4440) ... * delete code * delete code * delete code * commit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit	2025-10-19 16:23:11 +08:00
yzwu	4b661512ca	[Iluvatar GPU] Adapt VL model (#4313)	2025-10-17 16:13:38 +08:00
Sunny-bot1	930f7b781c	[Optimization] Put get_block_shape_and_split_kv_block in cuda graph for append attention backend (#4443) ... * get block in cuda graph * fix sot	2025-10-17 10:59:56 +08:00
Ryan	49cea8fb1c	[SOT][Cudagraph] Remove BreakGraph of #3302 && update CustomOp (#3694) ... * rm inplace info && to(gpu) * update append_attention * unpin paddle version * add full_cuda_graph=False * add blank line --------- Co-authored-by: SigureMo <sigure.qaq@gmail.com>	2025-10-17 10:57:55 +08:00
Zhenghai Zhang	6adfbe07ad	【Hackathon 9th No.86】autogen MultiQueryDecoderAttention template_instantiation -part (#4383) ... * split MultiQueryDecoderAttention template_instantiation * update comment * CI	2025-10-16 17:08:19 +08:00
freeliuzc	582aebd48b	[MTP]support mtp chunk_prefill_v1 (#4366) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output, fix unit test * fix unit test * fix save_output	2025-10-15 13:21:32 +08:00
Sunny-bot1	6d0cc0dd9c	[Optimization] Optimize split_q_block kernel (#4367)	2025-10-15 11:28:00 +08:00
Sunny-bot1	a751d977bc	[Optimization] Fuse get_max_len and get_kv_max_len (#4369) ... * opt split_q_block * fuse max_lens and max kv_len	2025-10-13 20:35:00 +08:00
freeliuzc	365601ea5a	[MTP]support more branchs in topp kernel (#4352)	2025-10-11 11:33:52 +08:00
AIbin	f7eaca3971	【Bug Fix】mla enables tensorcore by default (#4354) ... * mla tensor-core kernel is enabled by default	2025-10-10 20:45:16 +08:00
lzy	3aa04fbf21	[MTP][Cfp8]supports spec dynamic cfp8 (#4290) ... * supports spec dynamic cfp8 * supports spec dynamic cfp8 --------- Co-authored-by: freeliuzc <lzc842650834@gmail.com>	2025-10-10 16:08:10 +08:00
Zhenghai Zhang	c46d5e48f8	【Hackathon 9th No.86】autogen MultiQueryAppendC8Attention template_instantiation -part (#4330) ... * split MultiQueryAppendC8Attention template_instantiation * update setup_ops.py * fix ci * fix bug	2025-10-10 15:07:48 +08:00
AIbin	c4ebaf8a07	【Inference Optimize】MLA Tensor-Core is enabled by default (#4335)	2025-10-10 10:54:56 +08:00
Nyakku Shigure	5f80862578	Remove redundant inplace outputs for append_attention (#4340)	2025-10-10 10:21:27 +08:00
RAM	aa27b03bc0	[Executor]CUDAGraph support Speculate Decode (#3769) ... * success run ngram * Revert "[Code Simplification] remove cum_offsets (#3410)" This reverts commit 32b39620bc. * success run ngram5 tp4 42bs * success run ngram5 tp4 42bs * mtp draft commit * add decorator for target model * enable draft model in cudagraph v0.5 * revert revrt cum_offset * enable target model in cudagraph v0.9 And clean debug code * Revert "success run ngram" This reverts commit 8351e83993. * add reverted code * enable target model in cudagraph v0.9 * solve comment * fix bid < 0 * Enable Target Model Padding And Draft Model in cudagraph * solve problem * delete rebuild padding debug note * fast compile * Add capture list for mtp * success run 256 tp1 mtp * Enable Lite TP2 Bsz256 * realy enable tp2 bsz 256 * fix problem * Solve problem for Draft model in cudagraph * Solve comment * replace emptytensor as zeros * Solve comments * Revert "fast compile" This reverts commit 834639a7ff. * fix bug * fix merge bug * fix typo * fix bug --------- Co-authored-by: lizexu <2694294196@qq.com> Co-authored-by: littledgg <1658565283@qq.com> Co-authored-by: zeroRains <linjunlu@zerorains.top> Co-authored-by: gongshaotian <gstain5555@outlook.com>	2025-10-09 21:18:29 +08:00
fangfangssj	af3872215e	[BugFix]remove redundant includes (#4312) ... Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-09-30 17:54:19 +08:00
chen	81959c7d88	[NewFeature]custom_allreduce support cudagraph recapture (#4305) ... * custom_allreduce support cudagraph recapture * add shut_down/restart default group	2025-09-29 15:56:54 +08:00
xiaozude	7c919070f7	[Metax] support cutlass moe & optimize flash attention (#4208)	2025-09-29 11:22:43 +08:00
李泳桦	6265f4385f	[feat] support prefix cache clearing when /clear_load_weight is called (#4008) ... * [feat] support clearing prefix cache (cherry-picked from release/2.1) * [fix] fix ipc suffix, use port instead * [fix] fix prefix caching not enabled * [fix] fix key/value_cache_scales indent * [fix] fix ep group all-reduce * [fix] fix clear/update lock not working when workers > 1 * [chore] add preemption triggered info log * [fix] fix code style * [fix] fix max_num_seqs config * [fix] do not force enable_prefix_caching=False in dynamic loading * [fix] fix ci * Revert "[fix] fix ci" This reverts commit 0bc6d55cc8. * [fix] initialize available_gpu_block_num with max_gpu_block_num * [fix] fix config splitwise_role * [fix] fix clearing caches synchronization and add more logs * [chore] print cache_ready_signal in log * [fix] fix scheduler_config.splitwise_role * [fix] fix cache_messager cache_ready_signal create=True * [fix] stop cache messager from launching in mixed deployment	2025-09-28 19:42:53 +08:00
Zhenghai Zhang	aa045aa84f	fix typos (#4274)	2025-09-27 09:25:43 +08:00
chen	7c1fd19f0f	[OPs] MoE support wfp8afp8(channelwise) and improve per_token_quant_fp8 (#4238)	2025-09-24 16:39:51 +08:00
Yohanna	44010cee13	FIX] Fix CUDA error(700): 'cudaErrorIllegalAddress' in CascadeAppendWriteCacheKVQKV cache_kernel(). Continue when batch_id_per_token[token_idx] is default value -1. (#4218)	2025-09-24 14:08:49 +08:00
chen	1a6283424e	Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)	2025-09-23 18:41:33 +08:00
yzwu	504461b6b5	[Iluvatar GPU] Optimize attention performance and fix moe load ckpt error (#3651)	2025-09-22 21:13:59 +08:00
chenjian	918ccdb123	[Feature] Support pd ep deployment with yiyan adapter (#4029) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * Support pd ep deployment with yiyan adapter * Support pd ep deployment with yiyan adapter * refactor cache messager * support scheduler v1 in PD * suppport pd v1 + chunk prefill * suppport pd v1 + chunk prefill * add eplb * support eplb * support eplb * support eplb * support v1 * fix * fix * fix bug * remove eplb support * support prefix cache in P * fix bug * fix bug * support one stop in V1 * fix bug * fix ci * fix ci * fix * fix * fix * fix * fix --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-22 16:41:38 +08:00