482 Commits

Author	SHA1	Message	Date
Sunny-bot1	40f3897a4e	support w4afp8 moe offline permute & load (#5613)	2025-12-22 15:12:57 +08:00
bukejiyu	4aa2c6871b	[RL]Support loading weights via the load_weights function for RL (#5549) ... * RL support load_weights * fix	2025-12-18 02:27:05 -08:00
yzwu	ac013803f3	[Iluvatar] Support V1_KVCACHE_SCHEDULER and paddleocr-vl rope mode (#5555)	2025-12-18 02:14:25 -08:00
Longzhi Wang	d8587e987e	[Model] tp+ep support v1_loader (#5465) ... * [Model] tp+ep support v1_loader * fix * fix mtp_linear * fix mtp_linear * fix * fix * fix v0 loader * fix * Add get_tensor for ep * fix linear weight_loader * fix typo * fix	2025-12-18 14:31:54 +08:00
zhupengyang	8735cb5045	[XPU] refactor moe ffn (#5501) ... - remove BKCL_DISPATCH_ALL_GATHER - support sparse mode - support moe quant_method	2025-12-18 14:14:05 +08:00
fmiao2372	404cf0ece4	[Intel HPU] enable tensor_wise_fp8 (#5324) ... * [Intel HPU] enable tensor_wise_fp8 * update code based on comments * fix code style issue * fix bug about RP 5138 * mv kv_cache modifications to HPU backend * fix FP8 Precision Issues * fix FP8 Precision Issues * Add quantization UT --------- Co-authored-by: yanfeich <yanfei.cheng@intel.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-17 16:45:03 +08:00
freeliuzc	15f5112ecb	[Speculative Decoding]Support different inferseed in speculate decoding (#5568) ... * fix mtp entropy drop in RL * optimize usage and fix unit test * optimize padding_sampling_params speed(vectorized) --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-17 16:14:29 +08:00
Yuanle Liu	867803ae10	[BugFix] fix speculate_limit_thinking_content_length (#5590) ... * fix speculate_limit_thinking_content_length * update	2025-12-16 04:31:45 -08:00
RAM	6fc5eccf83	[RL] R3 Support RDMA Store (#5467) ... * [RL] R3 support rdma store * refine notes * refine code * disable prefix cache * support preempted task and put cpu tensor	2025-12-16 16:50:13 +08:00
Yuanle Liu	b8e4828373	[BugFix] fix dynamic c8 in v1 loader (#5562)	2025-12-15 04:07:54 -08:00
zhang-chenyi	77f8ba06e7	[Metax] fix release2.4 and support cudagraph (#5547) ... Co-authored-by: xiaozude <xiaozude@outlook.com>	2025-12-15 14:23:33 +08:00
Ryan	d01cb274d6	[Graph Optimization][CI] Add ERNIE45T 21B sot test (#5538)	2025-12-13 00:43:15 +08:00
Lucas	888c4b992d	[XPU] refactor of block_attn param 'pos_emb_type' (#5511)	2025-12-12 14:30:09 +08:00
Ryan	4eb55332f6	[Models] Add forward_meta to VocabParallelEmbedding of all models (#5524)	2025-12-12 14:11:31 +08:00
kevin	954a145d57	[Optimization] support mm prefill batch (#5313) ... * support mm prefill batch * update code * update code * update code * update code * fix encoder cache bug * update code * update code * fix bug * fix paddle ocr bug * fix xpu bug * update code	2025-12-11 22:21:14 +08:00
bukejiyu	4066dfb4a6	RL fix (#5503)	2025-12-11 19:25:27 +08:00
Ryan	e58fed3665	[Graph Optimization][BugFix][CI] Fix 0size bug && add unitest (#5495)	2025-12-11 16:25:26 +08:00
周周周	ff353b922f	[Others] update tbo related code (#5485)	2025-12-11 12:34:46 +08:00
Neil Zhu	4403a21d4b	[Metax] refactor cutlass moe and optimize flash attention (#5361) ... * [Metax] refactor moe and flash attention backend --------- Co-authored-by: zhangchenyi_dl <16219492+zhangchenyidl@user.noreply.gitee.com>	2025-12-10 17:15:17 +08:00
周周周	83a9ef51d7	[Others] add assert and only count the actual load in cuda_graph (#5445)	2025-12-10 11:22:54 +08:00
freeliuzc	53460935ec	fix attention bug in spec decoding (#5460)	2025-12-10 10:56:37 +08:00
lizexu123	95eab9f9ee	[Feature] support stop_token_ids (#5399) ... * support stop_token_ids * fix * delete chinese * support both * delete print	2025-12-09 17:49:12 +08:00
Haonan Luo	e397c4fba6	[Others] remove add_bias option (#5425)	2025-12-09 17:39:35 +08:00
周周周	31410415db	FA3 support qwen3 (#5441)	2025-12-09 16:16:16 +08:00
Nyakku Shigure	e1c4a12e34	[Graph Optimization][CINN] Use CINN in PaddleOCR-VL ViT part (#5223) ... --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-12-09 14:37:00 +08:00
K11OntheBoat	8d99bac532	Remove CUDA ERROR 9 of inputs of get_padding_offset kernel (#5440) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-12-09 14:17:30 +08:00
chen	76649b45c1	[Optimization] compulte real max_logprobs in batch (#5430)	2025-12-09 14:15:05 +08:00
xiaozude	c06a6234b9	[Metax] optimize mla attention (#5258)	2025-12-09 11:18:19 +08:00
Sunny-bot1	364197c4b5	support w4afp8 mtp (#5429)	2025-12-08 20:24:00 +08:00
周周周	2aea8a3a60	[Others] Remove useless code (#5404)	2025-12-08 13:59:46 +08:00
Nyakku Shigure	0c66163dfd	[Loader][BugFix] Fix some parameters place on CPU in PaddleOCR-VL (#5413) ... * [BugFix] Fix some parameter place on CPU in PaddleOCR-VL * clean log * fix codestyle	2025-12-08 10:01:00 +08:00
bukejiyu	c3a8a16f4c	fix deepseek (#5410)	2025-12-06 00:45:48 +08:00
bukejiyu	f6eb4dcc40	bf16 deepseek (#5379)	2025-12-05 22:23:30 +08:00
RAM	b2908b8e82	[New][RL] Support Rollout Routing Replay (#5405) ... * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064f3d. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-12-05 22:06:26 +08:00
Jiang-Jia-Jun	c45e064f3d	Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402) ... This reverts commit 96d2d4877b.	2025-12-05 20:19:39 +08:00
RAM	96d2d4877b	[RL] Support Rollout Routing Replay (#5321) ... * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-12-05 20:01:33 +08:00
zccjjj	5b900667e3	[XPU] support ep4tp1+v1 loader (#5398)	2025-12-05 18:51:15 +08:00
周周周	c83dc58105	[Feature] support Two batch overlap, mainly used in Prefill (#5078)	2025-12-05 14:58:50 +08:00
lizhenyun01	d436640735	[BugFix] Fix flash_attn_backend	2025-12-05 14:33:38 +08:00
zccjjj	e927c65742	[XPU] [Optimization] [EP] EP communication optimization. (#5145)	2025-12-05 10:03:45 +08:00
bukejiyu	620d1da1c9	deepseek torch (#5373)	2025-12-04 23:26:53 +08:00
Yuanle Liu	41c63f6056	remove fastsafetensors (#5371)	2025-12-04 19:22:04 +08:00
Longzhi Wang	5cd17fd662	[Models] Add forward_meta to moe models' forward function (#5138) ... * [Models] Add forward_meta to moe models' forward function * fix missing param * fix * fix * fix forward_meta * fix test and remove chunked MoE releated in config * fix test * fix * fix	2025-12-04 13:26:58 +08:00
fmiao2372	209006e6a6	[Intel HPU] fix memory fragmentation issue due to warmup process and fix moe all_reduce issue (#5357)	2025-12-04 11:29:41 +08:00
Yuanle Liu	be0c960260	[BugFix] dynamic cache kv block_wise_fp8 not need create layer.cache_k_scale (#5362)	2025-12-03 05:32:59 -08:00
Daci	83dbc4e5dd	[Feature] Guided Decoding add LLguidance backend (#5124) ... * llguidance * add requirements_guided_decoding.txt and doc * fix test_guidance_*.py * fix test_guidance_*.py && mv * fix llguidance choice * test_guidance_* * rm lazy loader --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-03 20:23:57 +08:00
lzy	690bcb8e50	[Optimization] 1.fix tp+ep moe_forward; 2.set max_prefill_batch=env.MAX_PREFILL_NUM (#5315)	2025-12-03 13:33:15 +08:00
Sunny-bot1	3629db4129	[Quantization] Support w4afp8 MoE dynamic quantization (#5282) ... * support dynamic activation quant for w4afp8 * support dynamic w4afp8 * add test * fix * fix --------- Co-authored-by: zhoutianzi666 <17801055074@163.com>	2025-12-02 18:56:16 +08:00
周周周	fb7f951612	[UNITEST] add test (#5305)	2025-12-02 17:59:01 +08:00
qw86972190	6048ea37bd	[XPU]add enable_logprob (#5279) ... * [XPU]Update document * [XPU]Update documentation * [XPU]add enable_logprob * Fix code style issues * “doc” * “docs” * “doc” * Fix code style via pre-commit --------- Co-authored-by: root <root@gajl-bbc-onlinec-com-1498354.gajl.baidu.com>	2025-12-02 15:32:28 +08:00