85 Commits

Author	SHA1	Message	Date
chen	3161014e49	[BugFix]fix v1 loader moe bf16, and supoort dynamic_load_weight create quant param (#4229) ... * fix v1 loader moe bf16, and supoort dynamic_load_weight create quant param * include_stop_str_in_output=False not return eos text	2025-09-24 14:12:05 +08:00
fmiao2372	f1b5392e20	[Intel HPU] Support intel hpu platform (#4161) ... * [Intel HPU] Support intel hpu platform * fix some issues * apply precommit and move AttentionBackend_HPU * fix format issue * correct ops import * fix ci issue * update code in layers * fix code style issue * remove dense tp moe ep mode * fix enc_dec_block_num * fix rebase issue * rename hpu to gaudi in readme * rename ForwardMeta_HPU to HPUForwardMeta	2025-09-24 12:27:50 +08:00
chen	1a6283424e	Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)	2025-09-23 18:41:33 +08:00
yzwu	504461b6b5	[Iluvatar GPU] Optimize attention performance and fix moe load ckpt error (#3651)	2025-09-22 21:13:59 +08:00
chenjian	918ccdb123	[Feature] Support pd ep deployment with yiyan adapter (#4029) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * Support pd ep deployment with yiyan adapter * Support pd ep deployment with yiyan adapter * refactor cache messager * support scheduler v1 in PD * suppport pd v1 + chunk prefill * suppport pd v1 + chunk prefill * add eplb * support eplb * support eplb * support eplb * support v1 * fix * fix * fix bug * remove eplb support * support prefix cache in P * fix bug * fix bug * support one stop in V1 * fix bug * fix ci * fix ci * fix * fix * fix * fix * fix --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-22 16:41:38 +08:00
YuanRisheng	24180fba0a	[FDConfig]Remove splitwise_role and engine_worker_queue_port in FDConfig (#4147) ... * remove splitwise_role and engine_worker_queue_port * fix xpu * fix xpu * fix xpu * fix unittest * resolve conflct	2025-09-19 17:01:52 +08:00
gaoziyuan	896e3bb606	[NewFeture]add ep rollout model init and update/clear ep buffer (#4039) ... * fix gid * merge * fix test * fix bug * fix * fix ci	2025-09-17 20:24:53 +08:00
zhupengyang	9409665713	[xpu] support ep (#4067)	2025-09-15 13:53:11 +08:00
bukejiyu	29ed617f0f	[v1 loader]qwen Offline fp8 (#4036) ... * support offline fp8 * update ut * update ut * update ut * fix * update * update	2025-09-15 13:44:11 +08:00
chen	4859f40b20	[Feature] GLM-45-AIR Support Mix Quantization(Dense wfp8afp8 and wint8 triton_moe_backend) (#4051)	2025-09-11 20:08:09 +08:00
Yuanle Liu	c3b2a60fb8	[BugFix] Fix the abnormal memory usage caused by shape errors in the triton moe backend (#4026) ... * fix device_id to in * fix triton_moe bug	2025-09-09 20:05:54 -07:00
bukejiyu	e52ce1c4b1	cache feature (#3857)	2025-09-07 18:52:46 +08:00
Yuan Xiaolan	2cf55168ca	load hadamard_block_size from config (#3797)	2025-09-05 17:07:58 +08:00
YuanRisheng	0a1ce612c2	V1 loader support ep (#3801)	2025-09-03 16:05:41 +08:00
bukejiyu	b6a4115369	[v1loader]Reduce EB300B model loading time (#3700) ... * speed up eb45 * update	2025-09-02 19:13:57 +08:00
Yuanle Liu	306c024ff3	[BugFix] fix error of import paddle.base.core.Config (#3761) ... * 延迟 import Config * support chunked_prefill * support chunked_prefill	2025-09-02 17:23:27 +08:00
ltd0924	905d89e42f	[Feature] support model weight update in ep (#3765) ... * support model weight update in ep * support model weight update in ep * support model weight update in ep * support model weight update in ep * Update fused_moe_backend_base.py * Update worker_process.py * Update worker_process.py * Update dynamic_weight_manager.py	2025-09-02 17:16:03 +08:00
co63oc	d6369b4d51	fix typos (#3684)	2025-09-01 17:50:17 +08:00
lizexu123	455205f991	[Features] support hugging face qwen3 moe (#3649) ... * split ut * qwen3-30B-A3B * fix * add test * add test_torch_model.py * fix test_torch_model.py * delete print * fix moe * delete init.py * fix * fix --------- Co-authored-by: bukejiyu <395822456@qq.com> Co-authored-by: bukejiyu <52310069+bukejiyu@users.noreply.github.com>	2025-08-30 15:26:05 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
xiaoxiaohehe001	1bf4fc7f36	support w4afp8 eplb (#3680)	2025-08-29 14:43:06 +08:00
Yuanle Liu	4957908275	add input_processor plugin (#3657) ... * add input_processor plugin * update * update * update * update * update * update * update * update * update * update * update	2025-08-28 22:53:57 +08:00
gaoziyuan	fc635acc47	[BugFix]fix dp&ep&tp and muti node infer (#3629) ... * rm log * fix bug * fix bug * fix dp&ep&tp and muti node infer * fix --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-08-28 19:09:10 +08:00
Liumengyuan	2a73a6df03	fix_fp8_deepgemm_moe_tp_bug (#3658)	2025-08-28 17:19:02 +08:00
Yuan Xiaolan	d37331fc71	fix w4afp8_gemm_scale_permute import error on A100 (#3611)	2025-08-28 11:42:23 +08:00
yangjianfengo1	e81046fdad	【New Feature】集中式支持w4afp8 (#3644) ... * 支持tp w4afp8 * code style	2025-08-28 10:53:24 +08:00
周周周	76513f6416	Support 45t fp8 8 GPU (#3659)	2025-08-28 10:52:53 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
lzy	1265f6c192	deepgemm don't support tp+ep (for ci) (#3638) ... * deepgemm don't support tp+ep (for ci) * deepgemm don't support tp+ep (for ci)	2025-08-27 16:39:19 +08:00
gaoziyuan	82e64b13e1	[NewFeature]Support dp multi api server && Fix some bug in mixed ep && merge develop (#3598) ... * [Feature] update ep * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix queue ports idx * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * fix ci * Update engine.py * fix ci * fix some bug in mixed ep * add server fix and op fix * rm some log * fix code style * ltd fix * fix * fix * fix some bug * fix bug * fix bug * fix style * Update config.py * Update splitwise_connector.py * Update cache_messager.py * Update __init__.py * merge and fix * Update engine.py * Update common_engine.py * Update run_ci_xpu.sh * Update ernie_processor.py * Update ernie_processor.py --------- Co-authored-by: ltd0924 <ltd0924@sina.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com>	2025-08-26 19:59:02 +08:00
bukejiyu	3200a80de3	[v1 loader]support fp8 (#3593) ... * support fp8 * update ci	2025-08-26 02:42:46 -07:00
xiaoxiaohehe001	9afa236e39	[NewFeatures] support eplb (#3547) ... * [NewFeatures] support eplb * fix eplb	2025-08-26 16:19:30 +08:00
lzy	d339df2e90	Supports DP+TP+EP hybrid parallel deployment strategy (#3489) ... * Support DP+TP+EP hybrid parallel deployment strategy * Support DP+TP+EP hybrid parallel deployment strategy * fix conflict * add moe_tp_ep function split_allgather_out * del tp_group in moe_cutlass_backend * for ci * fix parallel_config for ci * del log	2025-08-26 00:04:01 -07:00
lizexu123	c43a4bec00	[Features] support hugging face qwen3 dense and qwen2 model (#3574) ... * support qwen2 and qwen3 hugging face * fix moe * defualt_v1 loader * hugging_face_format deprecated * modify hugging_face_foramt to model_format * model_format auto * fix environemt * fix bug * fix qwen3-0.6 bug * model_format is str * fix	2025-08-26 10:54:53 +08:00
Yuan Xiaolan	9205c88da1	support w4afp8 EP inference (#3044)	2025-08-25 11:27:45 +08:00
bukejiyu	77514e3e1e	[V1 Loader] support weight_only (#3413) ... * support wint4/wint8 * delete smoe case * update ci * print log	2025-08-23 13:13:41 +08:00
YuanRisheng	85fbf5455a	[V1 Loader]Ernie VL support loader v1 (#3494) ... * ernie vl support new loader * add unittest * fix test	2025-08-22 11:16:57 +08:00
Zero Rains	30b3f2dc07	[BugFix][V1 Loader] fix the bug in creat weight for block_wise_fp8 (#3486)	2025-08-20 05:52:54 -07:00
Zero Rains	fef447e350	[V1 Loader] Support MOE parameters create and load for DeepGemm and marlin backend (#3447) ... * support deepgemm backend * support marlin backend * remove print * fix process_prequanted_weights	2025-08-19 14:15:53 +08:00
YuanRisheng	09c979f3dd	[V1 Loader] Support Ernie text（moe and dense） (#3110) ... * new loader support 0.3B * fix weight * support parallel load * support parallel load * fix slice * support moe * delete code * perfect code * perfect code	2025-08-14 20:25:28 +08:00
Kane2011	b4fef2cf29	[MetaxGPU] Support FastDeploy on metax gpu (#3241) ... * [MetaxGPU] Support FastDeploy on metax gpu * Update metax_worker.py 1. change worker log; 2. remove custom allreduce, adapt it later; 3. remove cuda graph; * Update __init__.py 1. remove metax's key work comment * Update __init__.py 1. remove metax's key word comment; 2. add fused_moe_kernel_paddle import --------- Co-authored-by: yongqiangma <xing.wo@163.com>	2025-08-13 11:11:54 +08:00
Zero Rains	42af0b4b64	[V1 Loader] Support DeepSeekV3(bf16) (#3294) ... * Support new loader for DeepSeekV3(bf16) * update paddle version * remove useless attr	2025-08-11 13:39:28 +08:00
gaoziyuan	a799d14df1	[Bugfix] Fix model accuracy in some ops (#3231) ... * fix noaux_tc op * fix * update * fix qk norm * fix linear for prequant loader * test * fix * fix * rm some print * fix noaux_tc op * test * Fix the confused enable_early_stop when only set early_stop_config (#3214) * fix the confused early_stop_config when only set early_stop_config * pre-commit * write a general method * Add ci case for min token and max token (#3229) Co-authored-by: xujing43 <xujing43@baidu.com> * add some evil cases (#3240) * add repitation early stop cases * add repitation early stop cases * add bad cases * add bad cases * add evil cases * qwen3_moe (#3084) * [Feature] support seed parameter (#3161) * support seed * fix * add SamplingMetadata seed test * The next_tokens values are inconsistent! * add air and rejection seed test * fix * add SamplingParams seed test * fix seed=0 * Default to defualt * fix * fix args_utils * fix review * fix review * fix * fix * add xpu,gcu,iluvatar support seed * fix * 【Fix Bug】 修复 fa3 支持集中式bug (#3235) * fix fa3 集中式bug * 增加qknorm参数 * fix qk norm * fix * update * fix linear for prequant loader * fix * fix * rm some print * fix * fix moe init weight&scale * fix moe init weight&scale --------- Co-authored-by: bukejiyu <395822456@qq.com> Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com> Co-authored-by: Zero Rains <linjunlu@zerorains.top> Co-authored-by: xjkmfa <108254620+xjkmfa@users.noreply.github.com> Co-authored-by: xujing43 <xujing43@baidu.com> Co-authored-by: Divano <dddivano@outlook.com> Co-authored-by: bukejiyu <52310069+bukejiyu@users.noreply.github.com> Co-authored-by: lizexu123 <39205361+lizexu123@users.noreply.github.com> Co-authored-by: yangjianfengo1 <125249383+yangjianfengo1@users.noreply.github.com> Co-authored-by: qingqing01 <dangqingqing@baidu.com>	2025-08-08 17:30:37 +08:00
Zero Rains	ce1f353c70	Move create_parameters to __init__ in FuseMOE for CultassBackend and TritonBackend (#3148) ... * w4a8 bug * fix w4a8 bug * remove code * modify the triton backend * fix ep * fix the bug with tensor_wise_fp8 in triton backend * fix the RL * fix bug by merge * fix the bug in w4a8 * fix the tensor_wise_fp8 bug * fix RL	2025-08-08 15:55:47 +08:00
bukejiyu	20839abccf	qwen3_moe (#3084)	2025-08-06 14:45:27 +08:00
Yuan Xiaolan	af543b7f0f	revise get_moe_scores (#3164)	2025-08-05 16:43:07 +08:00
RichardWooSJTU	f5c64a074c	[EP] Refactor DeepEP Engine Organization for Mixed Mode & Buffer Management Optimization (#3182) ... * Add support for mixed-ep across multi nodes * code refine --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com>	2025-08-05 15:40:11 +08:00
Yuan Xiaolan	1f8289e106	fix expertwise_scale (#3181)	2025-08-04 20:06:15 +08:00
Yuan Xiaolan	5f56d289a7	fix is_permuted (#3098) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-31 19:58:05 +08:00
Yuan Xiaolan	35935da9e5	support W4A8 EPLB (#3075)	2025-07-30 14:34:12 +08:00