305 Commits

Author	SHA1	Message	Date
lizexu123	c563eca791	[Feature] support reward model (#5301) ... * Your commit message here * add test * update develop * support reward * support enable_chunk_prefill * support bingfa * support convert is reward * update test * delete print * fix enable_thinking * add document * fix place * fix test * fix * support enable_prefix_caching * add no-enable_prefix-caching test * fix * support enable_prefix_caching * delete print * fix document * fix * fix test * fix document and delete chinese * udpate * enable_thinking * fix test	2025-12-02 14:55:31 +08:00
K11OntheBoat	2e1680838f	[PD Disaggregation] Support PD deployment of DeepSeekv3. (#5251) ... * Support deepseekv3 cache transfer for PD deploy * clean some log info --------- Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-12-02 14:11:50 +08:00
chen	aa35ce449d	[Optimization] EP empty_input_forward Remove Communication (#5254)	2025-12-01 21:10:40 +08:00
Longzhi Wang	add524d80c	[Feature] support chunked moe (#4575) ... * [Feature] support chunked moe * update * update * fix and add test * update * fix conflict and modity test * fix fused_moe * fix fused_moe * fix docstring * fix * fix typo * fix test * fix * fix * fix test * fix test	2025-12-01 15:17:18 +08:00
Jundong Liu	6f42c37359	[Deterministic] Move paddle version batch invariant pkg to Fastdeploy (#4763) ... * Move batch invariant pkg to Fastdeploy * fix problem and pre-commit * move test * Change testcase to FD style * Add testcase for log_softmax * Add testcase for mean * Add testcase for addmm * fix pre-commit * API check v0.9 * move to layers and add comment about log_softmax * Update fastdeploy/model_executor/layers/batch_invariant_ops/batch_invariant_ops.py 存在于原版代码注释中的版本控制遗留的内容，确实应该去除 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/batch_invariant/test_batch_invariance_op_mean.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/batch_invariant/test_batch_invariance_op_logsoftmax.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/layers/batch_invariant_ops/batch_invariant_ops.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * change comment after copilot fix * fix bug about addmm * avoid global effect by enable_torch_proxy --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-01 11:25:48 +08:00
ming1753	70ec1e17c1	[Features] add audio request & fix embedding bug (#5201) ... * [Features] add audio request & fix embedding bug * fix bug	2025-12-01 11:12:17 +08:00
cmcamdy	9f4977eb74	[xpu] support mtp for xpu(mix) (#5274) ... * [XPU] support kernel for mtp(base) * [XPU] support kernel for mtp(base) * format * format * format * fix gather next token * fix step && add test * fix * mv pre/post process * add adjust batch / gather next token for mtp * fix code style * fix mtp kenrel name * fix mtp kernel test * mv xpu pre/post process * mv xpu pre/post process * [xpu] support mtp * fix code style	2025-12-01 11:03:14 +08:00
fmiao2372	2c7683d551	[Intel HPU] change MoE weights and scales from list to tensor and add… (#5289) ... * [Intel HPU] change MoE weights and scales from list to tensor and add q/k rms norm * update doc * move HPU_CHUNK_SIZE into envs	2025-11-28 19:17:05 +08:00
bukejiyu	1539fd6056	[BugFix]Set default OMP_NUM_THREADS=3 and fix extra GPU memory usage in DeepSeek (#5219) ... * fix bug * update * update * update * fix copy * update	2025-11-28 14:22:04 +08:00
lizhenyun01	aba4fc657f	[Feature] support flash_mask_attention backend (#5134) ... * [Feature] suppert flash_mask_attention backend * fix unittest * clean code	2025-11-28 10:12:16 +08:00
chen	35f85baf09	[BugFix]fix v1 loader lm head fp32 (#5270)	2025-11-27 20:12:56 +08:00
Yuanle Liu	cb56d46694	[Optimization] Refine row parallel bias and nranks and moe all_reduce (#5247) ... * rename nranks to tp_size and fix bias in v1 loader * fix * update	2025-11-26 05:09:09 -08:00
chen	209970836e	[BugFix] BF16 MoE Cutlass Backend Support EP (#5242)	2025-11-26 19:16:22 +08:00
xiaozude	d5bd64336a	[Metax] support ENABLE_V1_KVCACHE_SCHEDULER (#5163)	2025-11-24 19:19:49 +08:00
xiaoxiaohehe001	e150a418d4	support moe offline quant (#5142)	2025-11-24 18:59:18 +08:00
xiaoxiaohehe001	95f3c8c641	[Fix] Fix eplb bug and support fp8 load weight (#5178) ... * fix eplb part2 * fix eplb part2 * fix eplb part2	2025-11-24 15:31:37 +08:00
xiaoxiaohehe001	6471dade4a	[Fix] Fix noaux ep test (#5161) ... * support noaux eplb * noaux_eplb * noaux_eplb * noaux_eplb * noaux_eplb	2025-11-21 16:36:41 +08:00
bukejiyu	34f59d9800	[RL]Fix missing is_distributed attribute (#5150) ... * fix * update	2025-11-21 14:14:25 +08:00
xiaoxiaohehe001	6ca2651995	[Feature] Support noaux for eplb (#5143) ... * support noaux eplb * noaux_eplb * noaux_eplb * noaux_eplb	2025-11-21 14:10:32 +08:00
ddchenhao66	e70e2279ce	[PD Disaggregation][XPU] Add XPU support for PD disaggregation (#5113) ... * [XPU] xpu support PD disaggregation * [XPU] fix the issue of cache KV transfer process startup failure on non-zero XPU cards * [XPU] xpu support PD disaggregation in v1 scheduler --------- Co-authored-by: ddchenhao66 <dhaochen163.com>	2025-11-21 14:09:01 +08:00
Yonghua Li	43097a512a	[BugFix] [PD Disaggregation] fix v1 scheduler prefill node profile run & ipc transfer protocol (#5132) ... * [fix] fix v1 scheduler profile run for append attention in prefill node * [fix] skip send_signal if kv signal not inited for gpu and xpu * [fix] extend fix to flash_attn & mla_attn * [fix] fix v1 pd run in ipc transfer protocol * [ci] add test for v1 pd profile run using ipc transfer protocol * [style] fix code style check * [style] fix code style again * [fix] fix profile run * [update] remove --num-gpu-blocks-override in example script * [chore] rename forward_meta is_profiling to is_dummy_or_profile_run	2025-11-20 21:39:22 +08:00
Ryan	0857099191	mv import (#5146)	2025-11-20 19:25:56 +08:00
周周周	385fe6dade	[Others] clean code (#5133)	2025-11-20 18:44:08 +08:00
周周周	6fa34102e8	[Others]get_block_shape_and_split_kv_block clean code (#5123)	2025-11-20 16:40:04 +08:00
Neil Zhu	0edda75a56	[Metax] optimize cutlass moe and flash attention backend (#5128)	2025-11-20 16:12:35 +08:00
Sunny-bot1	bde97e09f7	support dynamic activation quant for w4afp8 (#5117)	2025-11-19 21:11:16 +08:00
Sunny-bot1	43f0c7557e	[Feature] Add an unquantized option for MoE and Dense quant type (#4813)	2025-11-19 16:24:03 +08:00
bukejiyu	a82f25ea7b	[RL]Resolve shape mismatch problems in RL-related modules (#5032) ... * RL fix * update	2025-11-19 11:12:48 +08:00
Daci	eab8384da6	[Feature] ThreadPoolExecutor async fill_token_bitmask (#5083) ... * ThreadPoolExecutor async fill_token_bitmask * ThreadPoolExecutor async fill_token_bitmask logging * fix test_guided_decoding * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add fill_bitmask_parallel_batch_size ENV * FD_FILL_BITMASK_BATCH fastdeploy.envs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-11-19 10:04:16 +08:00
MingkunZhang	a36c958c66	[Metax] support default_v1 loader based #4988 (#5001)	2025-11-18 09:44:30 +08:00
fmiao2372	74f33efdbf	[Intel HPU] fix bugs caused by other commits (#5074) ... * [Intel HPU] fix bugs caused by other commits * update code by copilot	2025-11-17 15:28:55 +08:00
xiaozude	68f638f8b9	[Metax] support default_v1 loader and quant_config is None for triton moe (#5030)	2025-11-17 10:38:00 +08:00
yangjianfengo1	3afb717995	【Fix】fix deepep dispatch (#5036) ... * fix dispatch * fix dispatch --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com>	2025-11-17 10:34:01 +08:00
yzwu	3b80a799ab	[Iluvatar][CI] Fix moe_expert_dispatch cannot support dequant_scale (#5012) ... Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-11-17 10:18:42 +08:00
fmiao2372	e43a5fc055	[Intel HPU] enable level 1 prefix caching and fix some bugs (#4971) ... * [Intel HPU] enable prefix caching and dense tp moe ep and fix some bugs * update code by copilot * remove dense tp and moe ep code	2025-11-14 19:42:50 +08:00
Daci	5fc12eddfe	[Optimization] xgrammar async compile, multi thread, speed up (#4835) ... * xgrammar async compile, multi thread, speed up * fix test_sampler.py & pre-commit err * add redis version check && fix request.llm_engine_recv_req_timestamp * xgrammar prefill & decode & v0 * fix test_gpu_prompt_logprobs.py * add test_guided_decoding.py * Update fastdeploy/scheduler/splitwise_scheduler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/guided_decoding/xgrammar_backend.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/guided_decoding/xgrammar_backend.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix torch xgrammar unittest env --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-11-14 18:05:26 +08:00
yangjianfengo1	ae7bee8122	【New Feature】W4afp8 supports per group quantization (#4987) ... * w4afp8 支持per group * code style * fix transpose * revert fast hardmard --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com> Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com>	2025-11-13 19:17:27 +08:00
bukejiyu	4a0d881e15	update (#4985)	2025-11-13 15:58:01 +08:00
ming1753	3148dbca06	[BugFix] fix VL fp8 bug when moe token_num is 0 (#4928) ... * [BugFix] fix VL fp8 bug when moe token_num is 0 * fix bug * format * fix bug	2025-11-12 21:19:36 +08:00
xiaozude	c45b3ccb52	[Metax] optimize flash mla (#4915)	2025-11-12 16:43:46 +08:00
MingkunZhang	9d9f5df8d0	[Metax] support default_v1 loader & thinking model (#4956) ... Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com>	2025-11-12 16:32:26 +08:00
ltd0924	5bf48de999	[KVCache] support unified cache backend (#4903) ... * [Feature] support unified cache backend * fix * fix * fix * fix * Update metax_model_runner.py * fix * update * Update test_moba_attention_backend.py --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-11-12 14:54:52 +08:00
yzwu	76e60e98f8	[Iluvatar][CI] fix safetensors_rust.SafetensorError: framework paddle is invalid (#4972)	2025-11-12 14:13:40 +08:00
bukejiyu	b09ebb2813	refactor pt loading (#4532)	2025-11-11 21:30:39 +08:00
周周周	da6b4c10e5	[ATTENTION] make buffer alloc as a function (#4945)	2025-11-11 19:17:08 +08:00
SunLei	3098aee05f	[Perf] Support tensor transmission between work and engine with zero-copy to improve efficiency (#4839) ... * feat(zmq): support tensor transmission with zero-copy for improved efficiency * perf: zmq.send disable copy * zmq recv data for debug * convert logprobs tensor to cpu	2025-11-11 15:43:11 +08:00
yzwu	3707af7a4f	[Iluvatar] add vl into ci and support v1 loader (#4774)	2025-11-11 10:50:17 +08:00
Yuanle Liu	3dc0ffa46d	[TSP] Support qwen3 moe tsp + cudagraph (#4871) ... * support qwen3_moe tsp mode * fix * fix * update * update * update * fix * support external_rmsnorm * update * fix	2025-11-10 23:37:51 +08:00
Sunny-bot1	59d2edde29	[BugFix] Add support for weight shape constraints and group size selection in Machete (#4911)	2025-11-10 20:57:35 +08:00
周周周	54536267db	[DeepEP] support P async_finish (#4899)	2025-11-10 18:24:02 +08:00