979 Commits

Author	SHA1	Message	Date
CSWYF3634076	e4c64a71cc	[BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix (#3921)	2025-09-11 15:08:24 +08:00
bukejiyu	2650f58740	[docs] Update environment variables documentation (#3957)	2025-09-10 21:17:06 -07:00
AIbin	a7392a0ff9	【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886) ... * support MLA chunk_size auto search & cuda_graph	2025-09-11 10:46:09 +08:00
chen	637d96c6ae	[Feature] Support zai-org/GLM-4.5-Air BF16 model (#3928) ... * support glm45_air	2025-09-10 19:36:10 +08:00
ltd0924	684e93269b	[Fix] fix multi api server log dir (#3967) ... * [BugFix] fix max streaming tokens invalid * fix scheduler bug * fix scheduler bug * Update multi_api_server.py	2025-09-10 17:15:30 +08:00
RAM	d3e4ae3d49	[Executor] Adjust signal sending order in RL training (#3773) ... * Adjust processing order * fix bug * fix update_parameters bug * refine code	2025-09-10 13:24:20 +08:00
Ayakouji	453487d5b0	[Feat] ernie4_5_vl_moe support CudaGraph (#3226) ... * delete dynamic control flow for decode * coda-style * fix scatter/gather typos and use input stream instead default stream * support 0-Size Tensor * update runner and model * using static mem address as input * fix mem leak * refine code * update mm_buffer * fix typo * fix buffersize * fix unk token * refine code * refine * support other arch * open cudagraph in vlci * fix * update * update * update * fix cmd * update --------- Co-authored-by: aquagull <hongyuh@qq.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-10 13:11:57 +08:00
Yuanle Liu	c3b2a60fb8	[BugFix] Fix the abnormal memory usage caused by shape errors in the triton moe backend (#4026) ... * fix device_id to in * fix triton_moe bug	2025-09-09 20:05:54 -07:00
guozhuangzhuang	f078a959b6	metrics shared folder naming (#4007) ... * Fixed the issue of metrics file conflicts between multiple instances on a single machine * Use uuid to name the metrics shared folder * Use uuid to name the metrics shared folder	2025-09-10 10:47:20 +08:00
Sunny-bot1	3b1da6e4dd	support v1 loader for machete (#3999)	2025-09-10 10:21:33 +08:00
YuanRisheng	b3fac5bde1	[V1 Loader] Ernie kv cache quant support v1 loader (#3899) ... * support c8 for ernie * add unittest * support vl * fix c8	2025-09-09 05:25:08 -07:00
Zero Rains	98bfefea02	get org_vocab_size from args (#3983)	2025-09-09 15:08:03 +08:00
Jiang-Jia-Jun	c60adf4281	Revert "【FIX】Change the name of sparse attn from moba to plas (#3845)" (#4001) ... This reverts commit e31c8f7336.	2025-09-09 11:08:23 +08:00
yangjianfengo1	e31c8f7336	【FIX】Change the name of sparse attn from moba to plas (#3845) ... * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci	2025-09-09 10:56:50 +08:00
zhuzixuan	83bd55100b	[Optimize]Error messages about Model api. (#3839) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/openai/serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * 优化报错信息。 --------- Co-authored-by: yangzichao01 <yangzichao01@baidu.com> Co-authored-by: Yzc216 <101054010+Yzc216@users.noreply.github.com> Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-09-08 15:52:26 +08:00
qwes5s5	17169a14f2	[metrics] Add serveral observability metrics (#3868) ... * Add several observability metrics * [wenxin-tools-584] 【可观测性】支持查看本节点的并发数、剩余block_size、排队请求数等信息 * adjust some metrics and md files * trigger ci * adjust ci file * trigger ci * trigger ci --------- Co-authored-by: K11OntheBoat <your_email@example.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:13:13 +08:00
Jundong Liu	3d0aaa5923	[Excutor] Experiment Feature-Support Prefill in cudagraph (#3459) ... * Support prefill in Cudagraph * Refactor GetBlockShapeAndSplitKVBlock Kernel V2 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.1 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.2 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.3 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.4 * Refactor GetBlockShapeAndSplitKVBlock Kernel V2.5 * Solve problem about encoder_num_blocks_x_cpu * Add early-exit mechanism for attention kernel * fix test case about append-attention * Update testcode, Add annotations to related tensors * move get_input_length_list * solve test_code * Add annotations about early-exit for attention kernel * Add annotations about early-exit for attention kernel2 * solve comment * solve mtp --------- Co-authored-by: RAM <gstian5555@outlook.com>	2025-09-08 13:12:24 +08:00
lzy	af49b81ffd	supports dynamic Cfp8 (#3767) ... * supports dynamic Cfp8 * add unittest	2025-09-07 20:41:29 -07:00
chenjian	b5e20e3015	[Bug fix] Fix prompt token ids dtype in v1 (#3860)	2025-09-08 11:34:13 +08:00
yinwei	7833f2f6cb	[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER (#3897) ... * fix bug * fix bug * update * update * update	2025-09-08 10:34:46 +08:00
ApplEOFDiscord	b649494655	[Feature] add HTTP GET retry (#3838) ... * add http get retry * fix coments --------- Co-authored-by: zhangjunjun04 <zhangjunjun04@baidu.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 10:11:14 +08:00
bukejiyu	e52ce1c4b1	cache feature (#3857)	2025-09-07 18:52:46 +08:00
ltd0924	0c45e225d3	mv connection_manager init (#3901) ... Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-05 21:11:48 +08:00
chen	0d989829bb	Compatible with EB 0.3B torch model arch (#3913) ... * fix * check	2025-09-05 19:04:59 +08:00
ltd0924	bd7d15f7ea	[Feature] support controller port in multi api server (#3898) ... * Update serving_chat.py * Update serving_completion.py * Update serving_completion.py * Update multi_api_server.py	2025-09-05 17:16:31 +08:00
Yuan Xiaolan	2cf55168ca	load hadamard_block_size from config (#3797)	2025-09-05 17:07:58 +08:00
AIbin	41aee08982	【Inference Optimize】Update MergedReplicatedLinear for DSK qkv_a_proj_with_mqa. (#3673) ... * support MergedReplicatedLinear * update MergedReplicatedLinear to support DSK_wint4 V1_load * update model name * update linear class * fix * fix v0 moe_bias load --------- Co-authored-by: bukejiyu <52310069+bukejiyu@users.noreply.github.com>	2025-09-04 21:16:05 -07:00
gaoziyuan	ab1929f5ff	fix mem boom in ep (#3854)	2025-09-05 11:48:21 +08:00
ltd0924	e0e7d68435	Update qwen_vl_processor.py (#3808)	2025-09-04 20:31:48 +08:00
SunLei	29628de6a7	Support for async processor added. (#3869) ... * Support for async processor added. * remove yappi code --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-04 19:58:53 +08:00
xiaolei373	ed97cf8396	Graceful shut down (#3785) ... * feat(log):add_request_and_response_log * 优雅退出-接口增加退出时长参数	2025-09-04 19:33:50 +08:00
freeliuzc	88d44a2c93	support mtp in v1_scheduler mode (#3695)	2025-09-04 17:39:59 +08:00
RichardWooSJTU	f36a388ffe	fix response processsors (#3826) ... * fix response processsors * fix ci * fix ut --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-04 16:01:25 +08:00
chenjian	22c165d6dd	[Feature] Set v1 scheduler as default in develop (#3807) ... * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * close V1 in guided_decoding * fix vl ci * close V1 in guided_decoding	2025-09-04 15:16:56 +08:00
RichardWooSJTU	0989788b29	support extend block tables (#3824)	2025-09-04 14:39:04 +08:00
gaoziyuan	6ef3b611b0	add dp config (#3822)	2025-09-04 11:46:48 +08:00
co63oc	9ec4fa0f8e	fix typo EngineSevice EngineService (#3841)	2025-09-04 11:20:36 +08:00
yyssys	abde903813	Automatically configure workers based on max-num-seqs (#3846) ... Automatically configure workers based on max-num-seqs	2025-09-03 21:12:42 +08:00
luukunn	fc598d4c5a	add reasoning parser plugin (#3811) ... * add reasoning parser plugin * fix finish reason	2025-09-03 18:31:27 +08:00
Ayakouji	31313e0f3d	[Feature] ernie4_5_vl_moe support huggingface safetensor loading (#3750) ... * update * update * update in tp * add todo * update --------- Co-authored-by: aquagull <hongyuh@qq.com>	2025-09-03 02:58:59 -07:00
YuanRisheng	0a1ce612c2	V1 loader support ep (#3801)	2025-09-03 16:05:41 +08:00
co63oc	94a61d505c	fix dcu_worker.py (#3734)	2025-09-03 10:57:42 +08:00
co63oc	ce998449e0	fix w8a8.py (#3733)	2025-09-03 10:57:26 +08:00
co63oc	5441538173	rename fused_get_rope.cu (#3752) ... * rename fused_get_rope.cu * fix * fix typos * fix * fix	2025-09-03 10:54:34 +08:00
ltd0924	2c9b169c0e	[BugFix] fix scheduler invalid (#3803) ... * [BugFix] fix max streaming tokens invalid * fix scheduler bug * fix scheduler bug	2025-09-02 20:28:51 +08:00
Longzhi Wang	e0c9a6c76c	[Feat] Support streaming transfer data using ZMQ (#3521) ... * Support streaming transfer data of ZMQ * fix typo * fix typo * support tp * add unittest * update * update * fix typo * fix typo * fix tp_num in ci machine --------- Co-authored-by: Wanglongzhi2001 <>	2025-09-02 19:52:19 +08:00
yangjianfengo1	8e1b35a09b	【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (#3771) ... * fix w4afp8 * 增加集中式配置 * codestyle * fix fa3 append attn	2025-09-02 19:17:01 +08:00
bukejiyu	b6a4115369	[v1loader]Reduce EB300B model loading time (#3700) ... * speed up eb45 * update	2025-09-02 19:13:57 +08:00
RAM	205b706ef8	[Executor] Fix bug of import paddle with RLHF (#3781)	2025-09-02 17:32:13 +08:00
Yuanle Liu	306c024ff3	[BugFix] fix error of import paddle.base.core.Config (#3761) ... * 延迟 import Config * support chunked_prefill * support chunked_prefill	2025-09-02 17:23:27 +08:00