331 Commits

Author	SHA1	Message	Date
Neil Zhu	4403a21d4b	[Metax] refactor cutlass moe and optimize flash attention (#5361) ... * [Metax] refactor moe and flash attention backend --------- Co-authored-by: zhangchenyi_dl <16219492+zhangchenyidl@user.noreply.gitee.com>	2025-12-10 17:15:17 +08:00
周周周	83a9ef51d7	[Others] add assert and only count the actual load in cuda_graph (#5445)	2025-12-10 11:22:54 +08:00
freeliuzc	53460935ec	fix attention bug in spec decoding (#5460)	2025-12-10 10:56:37 +08:00
Haonan Luo	e397c4fba6	[Others] remove add_bias option (#5425)	2025-12-09 17:39:35 +08:00
周周周	31410415db	FA3 support qwen3 (#5441)	2025-12-09 16:16:16 +08:00
K11OntheBoat	8d99bac532	Remove CUDA ERROR 9 of inputs of get_padding_offset kernel (#5440) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-12-09 14:17:30 +08:00
chen	76649b45c1	[Optimization] compulte real max_logprobs in batch (#5430)	2025-12-09 14:15:05 +08:00
xiaozude	c06a6234b9	[Metax] optimize mla attention (#5258)	2025-12-09 11:18:19 +08:00
Sunny-bot1	364197c4b5	support w4afp8 mtp (#5429)	2025-12-08 20:24:00 +08:00
周周周	2aea8a3a60	[Others] Remove useless code (#5404)	2025-12-08 13:59:46 +08:00
bukejiyu	c3a8a16f4c	fix deepseek (#5410)	2025-12-06 00:45:48 +08:00
bukejiyu	f6eb4dcc40	bf16 deepseek (#5379)	2025-12-05 22:23:30 +08:00
RAM	b2908b8e82	[New][RL] Support Rollout Routing Replay (#5405) ... * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064f3d. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-12-05 22:06:26 +08:00
Jiang-Jia-Jun	c45e064f3d	Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402) ... This reverts commit 96d2d4877b.	2025-12-05 20:19:39 +08:00
RAM	96d2d4877b	[RL] Support Rollout Routing Replay (#5321) ... * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-12-05 20:01:33 +08:00
zccjjj	5b900667e3	[XPU] support ep4tp1+v1 loader (#5398)	2025-12-05 18:51:15 +08:00
周周周	c83dc58105	[Feature] support Two batch overlap, mainly used in Prefill (#5078)	2025-12-05 14:58:50 +08:00
lizhenyun01	d436640735	[BugFix] Fix flash_attn_backend	2025-12-05 14:33:38 +08:00
zccjjj	e927c65742	[XPU] [Optimization] [EP] EP communication optimization. (#5145)	2025-12-05 10:03:45 +08:00
bukejiyu	620d1da1c9	deepseek torch (#5373)	2025-12-04 23:26:53 +08:00
Longzhi Wang	5cd17fd662	[Models] Add forward_meta to moe models' forward function (#5138) ... * [Models] Add forward_meta to moe models' forward function * fix missing param * fix * fix * fix forward_meta * fix test and remove chunked MoE releated in config * fix test * fix * fix	2025-12-04 13:26:58 +08:00
fmiao2372	209006e6a6	[Intel HPU] fix memory fragmentation issue due to warmup process and fix moe all_reduce issue (#5357)	2025-12-04 11:29:41 +08:00
Yuanle Liu	be0c960260	[BugFix] dynamic cache kv block_wise_fp8 not need create layer.cache_k_scale (#5362)	2025-12-03 05:32:59 -08:00
lzy	690bcb8e50	[Optimization] 1.fix tp+ep moe_forward; 2.set max_prefill_batch=env.MAX_PREFILL_NUM (#5315)	2025-12-03 13:33:15 +08:00
Sunny-bot1	3629db4129	[Quantization] Support w4afp8 MoE dynamic quantization (#5282) ... * support dynamic activation quant for w4afp8 * support dynamic w4afp8 * add test * fix * fix --------- Co-authored-by: zhoutianzi666 <17801055074@163.com>	2025-12-02 18:56:16 +08:00
周周周	fb7f951612	[UNITEST] add test (#5305)	2025-12-02 17:59:01 +08:00
lizexu123	c563eca791	[Feature] support reward model (#5301) ... * Your commit message here * add test * update develop * support reward * support enable_chunk_prefill * support bingfa * support convert is reward * update test * delete print * fix enable_thinking * add document * fix place * fix test * fix * support enable_prefix_caching * add no-enable_prefix-caching test * fix * support enable_prefix_caching * delete print * fix document * fix * fix test * fix document and delete chinese * udpate * enable_thinking * fix test	2025-12-02 14:55:31 +08:00
K11OntheBoat	2e1680838f	[PD Disaggregation] Support PD deployment of DeepSeekv3. (#5251) ... * Support deepseekv3 cache transfer for PD deploy * clean some log info --------- Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-12-02 14:11:50 +08:00
chen	aa35ce449d	[Optimization] EP empty_input_forward Remove Communication (#5254)	2025-12-01 21:10:40 +08:00
Longzhi Wang	add524d80c	[Feature] support chunked moe (#4575) ... * [Feature] support chunked moe * update * update * fix and add test * update * fix conflict and modity test * fix fused_moe * fix fused_moe * fix docstring * fix * fix typo * fix test * fix * fix * fix test * fix test	2025-12-01 15:17:18 +08:00
Jundong Liu	6f42c37359	[Deterministic] Move paddle version batch invariant pkg to Fastdeploy (#4763) ... * Move batch invariant pkg to Fastdeploy * fix problem and pre-commit * move test * Change testcase to FD style * Add testcase for log_softmax * Add testcase for mean * Add testcase for addmm * fix pre-commit * API check v0.9 * move to layers and add comment about log_softmax * Update fastdeploy/model_executor/layers/batch_invariant_ops/batch_invariant_ops.py 存在于原版代码注释中的版本控制遗留的内容，确实应该去除 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/batch_invariant/test_batch_invariance_op_mean.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/batch_invariant/test_batch_invariance_op_logsoftmax.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/layers/batch_invariant_ops/batch_invariant_ops.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * change comment after copilot fix * fix bug about addmm * avoid global effect by enable_torch_proxy --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-01 11:25:48 +08:00
ming1753	70ec1e17c1	[Features] add audio request & fix embedding bug (#5201) ... * [Features] add audio request & fix embedding bug * fix bug	2025-12-01 11:12:17 +08:00
cmcamdy	9f4977eb74	[xpu] support mtp for xpu(mix) (#5274) ... * [XPU] support kernel for mtp(base) * [XPU] support kernel for mtp(base) * format * format * format * fix gather next token * fix step && add test * fix * mv pre/post process * add adjust batch / gather next token for mtp * fix code style * fix mtp kenrel name * fix mtp kernel test * mv xpu pre/post process * mv xpu pre/post process * [xpu] support mtp * fix code style	2025-12-01 11:03:14 +08:00
fmiao2372	2c7683d551	[Intel HPU] change MoE weights and scales from list to tensor and add… (#5289) ... * [Intel HPU] change MoE weights and scales from list to tensor and add q/k rms norm * update doc * move HPU_CHUNK_SIZE into envs	2025-11-28 19:17:05 +08:00
bukejiyu	1539fd6056	[BugFix]Set default OMP_NUM_THREADS=3 and fix extra GPU memory usage in DeepSeek (#5219) ... * fix bug * update * update * update * fix copy * update	2025-11-28 14:22:04 +08:00
lizhenyun01	aba4fc657f	[Feature] support flash_mask_attention backend (#5134) ... * [Feature] suppert flash_mask_attention backend * fix unittest * clean code	2025-11-28 10:12:16 +08:00
chen	35f85baf09	[BugFix]fix v1 loader lm head fp32 (#5270)	2025-11-27 20:12:56 +08:00
Yuanle Liu	cb56d46694	[Optimization] Refine row parallel bias and nranks and moe all_reduce (#5247) ... * rename nranks to tp_size and fix bias in v1 loader * fix * update	2025-11-26 05:09:09 -08:00
chen	209970836e	[BugFix] BF16 MoE Cutlass Backend Support EP (#5242)	2025-11-26 19:16:22 +08:00
xiaozude	d5bd64336a	[Metax] support ENABLE_V1_KVCACHE_SCHEDULER (#5163)	2025-11-24 19:19:49 +08:00
xiaoxiaohehe001	e150a418d4	support moe offline quant (#5142)	2025-11-24 18:59:18 +08:00
xiaoxiaohehe001	95f3c8c641	[Fix] Fix eplb bug and support fp8 load weight (#5178) ... * fix eplb part2 * fix eplb part2 * fix eplb part2	2025-11-24 15:31:37 +08:00
xiaoxiaohehe001	6471dade4a	[Fix] Fix noaux ep test (#5161) ... * support noaux eplb * noaux_eplb * noaux_eplb * noaux_eplb * noaux_eplb	2025-11-21 16:36:41 +08:00
bukejiyu	34f59d9800	[RL]Fix missing is_distributed attribute (#5150) ... * fix * update	2025-11-21 14:14:25 +08:00
xiaoxiaohehe001	6ca2651995	[Feature] Support noaux for eplb (#5143) ... * support noaux eplb * noaux_eplb * noaux_eplb * noaux_eplb	2025-11-21 14:10:32 +08:00
ddchenhao66	e70e2279ce	[PD Disaggregation][XPU] Add XPU support for PD disaggregation (#5113) ... * [XPU] xpu support PD disaggregation * [XPU] fix the issue of cache KV transfer process startup failure on non-zero XPU cards * [XPU] xpu support PD disaggregation in v1 scheduler --------- Co-authored-by: ddchenhao66 <dhaochen163.com>	2025-11-21 14:09:01 +08:00
Yonghua Li	43097a512a	[BugFix] [PD Disaggregation] fix v1 scheduler prefill node profile run & ipc transfer protocol (#5132) ... * [fix] fix v1 scheduler profile run for append attention in prefill node * [fix] skip send_signal if kv signal not inited for gpu and xpu * [fix] extend fix to flash_attn & mla_attn * [fix] fix v1 pd run in ipc transfer protocol * [ci] add test for v1 pd profile run using ipc transfer protocol * [style] fix code style check * [style] fix code style again * [fix] fix profile run * [update] remove --num-gpu-blocks-override in example script * [chore] rename forward_meta is_profiling to is_dummy_or_profile_run	2025-11-20 21:39:22 +08:00
Ryan	0857099191	mv import (#5146)	2025-11-20 19:25:56 +08:00
周周周	385fe6dade	[Others] clean code (#5133)	2025-11-20 18:44:08 +08:00
周周周	6fa34102e8	[Others]get_block_shape_and_split_kv_block clean code (#5123)	2025-11-20 16:40:04 +08:00