251 Commits

Author	SHA1	Message	Date
chenjian	0d7f29841d	[Cherry-pick] Fix bug in develop for eb5 (#4845) ... * fix bug for PD EP * fix * optimize perf for engine worker queue * fix bug * fix internode ll two stage * fix for ci	2025-11-06 14:01:42 +08:00
K11OntheBoat	62dfad4a5f	[PD Disaggregation] Support Qwen3-MoE use PD + EP inference. (#4691) ... support Qwen-MoE PD/EP	2025-11-06 10:32:15 +08:00
yangjianfengo1	93fcf7e4ec	【New Feature】W4afp8 supports per group quantization (#4272) ... * w4afp8 支持per group * code style * 精度完成 * revert append attn utils * ffn1 动态量化 * ffn2 支持动态量化 * code style * code style * 修改单测 * 修改单测 * fix bug * Implement conditional parameter creation for layers Add parameter creation for up_gate_proj_in_scale when ep_size > 1. * code style * fix conflict * code style * code style * 修复w4aint8 精度 * fix ci --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com>	2025-11-05 21:00:23 +08:00
yinwei	ea1dd0e735	[XPU]Support V1 loader in weight_only Model (#4808) ... * support v1 loader in wint8 * code style * update --------- Co-authored-by: root <root@gajl-bbc-onlinec-com-1498356.gajl.baidu.com>	2025-11-05 17:09:11 +08:00
zhupengyang	2fd254e5b7	support ep+tp at op layer (#4688)	2025-11-05 11:15:57 +08:00
chen	1c3ca48128	[Feature][Executor] GPU Model Runner Supports prompt_logprobs and max_logprobs (#4769)	2025-11-05 10:43:25 +08:00
xiaozude	74722308f2	[Metax] adapt cutlass moe and fix mla attention (#4602) ... Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-11-05 10:03:49 +08:00
lzy	3e9dda39ab	supports pd partn (#4615) ... * supports pd partn * fix codestype	2025-11-04 16:36:35 +08:00
lzy	af7e0f27f3	supports internode_ll_two_stage (#4162) ... * supports internode_ll_two_stage * supports internode_ll_two_stage * supports internode_ll_two_stage * supports internode_ll_two_stage * supports D internode_ll_two_stage * fix codestype * fix xpu internode_ll_two_stage * fix xpu internode_ll_two_stage	2025-11-04 16:35:40 +08:00
freeliuzc	855a2a609a	fix attn_params (#4787)	2025-11-04 13:01:38 +08:00
Yuan Xiaolan	8690cf8569	fix Cfp8 for RL load (#4144)	2025-11-03 17:51:51 +08:00
yinwei	377f3bf5f2	[XPU] add v1 support for bf16 (#4744) ... * support v1 loader * update code style * update code	2025-11-03 14:13:17 +08:00
freeliuzc	11398790d3	[Speculative Decoding][MTP]Support attn mask offset (#4641) ... * [MTP]Merge support attn (#4591) * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import * support attn_mask_offset in mtp mode * add update_attn_mask op * fix unit test && fix code-style	2025-11-03 10:08:01 +08:00
lizexu123	4ac6de9a3c	[Feature] support pooling model runner (#4590) ... * support qwen3-embedding * support qwen3-embedding-0.6b * fix * fix bug * fix test_return_token_ids.py and update enable_thinking * fix mtp dummy_run * merge develop * fix np.float32 * delete FD_DISABLE_CHUNKED_PREFILL and FD_USE_GET_SAVE_OUTPUT_V1 * delete and build_stream_transfer_data * fix test_update_v1: * fix * fix * update dummy_run post_process * delete test_update_v1 * fix * fix dummy_run * fix model_path * fix model_path * fix dummy_run	2025-10-31 22:32:05 +08:00
AIbin	316f784016	fix wint2 config (#4721)	2025-10-31 15:44:14 +08:00
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
周周周	0089287534	[noauxtc_kernel] remove useless code (#4643) ... * remove num_tokens * remove num_tokens * false * final commit	2025-10-30 18:59:04 +08:00
GoldPancake	fddda50cb9	Add ut for speculative sampler (#4650)	2025-10-30 10:37:49 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
Lucas	0a0c74e717	[XPU] Support PaddleOCR-VL model for XPU (#4529) ... * [XPU] support PaddleOCR-VL in XPU * [XPU] fix PaddleOCR-VL pos_emb_type	2025-10-28 20:35:04 +08:00
freeliuzc	c63361fd1d	[Speculative Decoding][MTP]Support mtp in epdptp mode (#4614) ... * support mtp many features * support mtp reshard in rl mode * fix function * support mtp ep * support mtp in hybird-dp-tp mode * default open scheduler_v1 in mtp	2025-10-28 16:02:47 +08:00
周周周	3729e910a6	remove dev sync in prefill (#4598)	2025-10-27 19:54:43 +08:00
RAM	25a983ba9c	1.fix the bug of draft model with ep 2.fix sampler bug (#4589)	2025-10-27 17:47:34 +08:00
chen	5c63a089f6	[Feature] Support logprobs_mode (#4567)	2025-10-27 14:27:48 +08:00
Lucas	5c6105f4a2	[XPU] bind some OPs for VL model with pybind (#4522)	2025-10-27 10:50:08 +08:00
ming1753	e4e3cede7f	[Feature] Support Paddle-OCR (#4396) ... * init * update code * fix code style & disable thinking * adapt for common_engine.update_mm_requests_chunk_size * use 3d rope * use flash_attn_unpadded * opt siglip * update to be compatible with the latest codebase * fix typo * optim OCR performance * fix bug * fix bug * fix bug * fix bug * normlize name * modify xpu rope * revert logger * fix bug * fix bug * fix bug * support default_v1 * optim performance * fix bug --------- Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com>	2025-10-24 23:34:30 +08:00
yyssys	822dea8d5f	[XPU]Moe uses a new operator (#4585) ... * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response	2025-10-24 23:01:46 +08:00
xiaozude	f7069b8057	[Metax] adapt DeepSeek (#4498)	2025-10-24 10:14:53 +08:00
zhupengyang	3a43dbf82d	[XPU] merge apply_tp, ops support token_num = 0 (#4507)	2025-10-23 19:09:58 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
gaoziyuan	d85ef5352a	【BugFix】fix ep buffer clear (#4450) ... * fix * fix	2025-10-21 10:56:00 +08:00
yinwei	bf03b6fcea	fix vl bug (#4485)	2025-10-20 20:13:34 +08:00
yyssys	97ee3c403a	[XPU]Fix w4a8 garbled code issue (#4493)	2025-10-20 19:41:11 +08:00
bukejiyu	de2eaf4f81	add qwen-2.5-7B-PRM/ernie-rm (#4319)	2025-10-20 15:31:03 +08:00
GoldPancake	47595a2480	[Feature] support mtp logprob (#4464) ... * support mtp logprob * fix unitest	2025-10-20 15:18:12 +08:00
Haonan Luo	1b9f351d21	Support GPT-OSS-BF16 (#4240) ... * [Feature] AppendAtten support sinks & HEAD_DIM=64 * fix bug * fix bug * fix bug * fix bug * [Feature] support gpt-oss * fix bug * add mask * support-gpt-oss * support-gpt-oss * fix long seq * support wint8 * support wint8 * support wint8 * update test * change sliding windows init pos --------- Co-authored-by: ming1753 <ideaminghp@163.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com>	2025-10-20 14:44:58 +08:00
SuperNova	80a16c4c87	[fix] adjust mctlass moe api (#4474)	2025-10-20 14:23:54 +08:00
yinwei	a64c0408b9	[XPU]Fix w4a8 precision bug && rollback moe algo (#4463) ... * fix w4a8 precision bug * add env * code stype check	2025-10-17 18:27:53 +08:00
yzwu	4b661512ca	[Iluvatar GPU] Adapt VL model (#4313)	2025-10-17 16:13:38 +08:00
lizexu123	c234b995ab	[Feature] support pooling model dummy_run (#4345) ... * support qwen3-embedding * fix ci bug * support pooling dummy_run * fix * delete print * parallel_config.max_model_len * delete is_pooling_model in dummy_run * fix * fd_model * fix embedding load * fix * fix post_process	2025-10-17 13:30:55 +08:00
chen	b134e6afe6	[BugFix]Dev fix custom ar unstable result (#4437)	2025-10-17 11:47:16 +08:00
Sunny-bot1	930f7b781c	[Optimization] Put get_block_shape_and_split_kv_block in cuda graph for append attention backend (#4443) ... * get block in cuda graph * fix sot	2025-10-17 10:59:56 +08:00
Ryan	49cea8fb1c	[SOT][Cudagraph] Remove BreakGraph of #3302 && update CustomOp (#3694) ... * rm inplace info && to(gpu) * update append_attention * unpin paddle version * add full_cuda_graph=False * add blank line --------- Co-authored-by: SigureMo <sigure.qaq@gmail.com>	2025-10-17 10:57:55 +08:00
chen	db82e9a022	[BugFix]Fix wfp8afp8 triton moe group_topk renormalized=True (#4449) ... * fix group_topk renormalized=True * check test	2025-10-16 23:17:48 +08:00
YuanRisheng	0355235fb9	[FDConfig]Remove total_block_num/dtype/block_size/enc_dec_block_num in ParallelConfig (#4400) ... * delete some attr in parallel config * delete comment --------- Co-authored-by: root <root@yqlcc01-sys-rpm12rzmwjd.yqlcc01.baidu.com>	2025-10-16 20:00:37 +08:00
zhupengyang	26ff2f8683	[XPU] refine fused moe (#4219)	2025-10-16 19:04:07 +08:00
Jianyu Li	3bbe99eae7	[Intel HPU] Enable dist sampler on intel hpu platform (#4445)	2025-10-16 19:02:27 +08:00
Lucas	a5063b96c8	[XPU] moe support VL 0-dim input (#4408)	2025-10-16 14:01:01 +08:00
gaoziyuan	fd5dd1a0f1	[Bugfix]fix ep clear buffer perf (#4389) ... * fix * Update fused_moe_backend_base.py	2025-10-16 13:05:39 +08:00
SunLei	b4b579a7ed	Feature：Add support for Pooling Model Embedding and provide an OpenAI-compatible API. (#4344) ... * feat: add OpenAIServing * feat: add ZmqOpenAIServing & OpenAIServingEmbedding * feat: Refine the basic ServingEngine class and introduce ServingContext * fix: codestyle * fix: request * fix: pooling_params * feat: _process_chat_template_kwargs * feat: support batch request * feat: pooling_params verify & default parameters --------- Co-authored-by: sunlei1024 <sunlei1024@example.com>	2025-10-15 19:42:59 +08:00