142 Commits

Author	SHA1	Message	Date
kxz2002	83d41d23b0	initial commit (#4248)	2025-09-25 21:42:05 +08:00
K11OntheBoat	4515ad21e9	Support limit thinking lengths (#4069) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-09-25 19:55:56 +08:00
Lucas	87179cb744	[XPU] support XPU VL model inference (#4030) ... * [XPU] support XPU VL model inference * fix image op import and device check * rebase develop * fix perf	2025-09-25 14:34:15 +08:00
luukunn	18f4977aec	[fix]update apply_chat_template (#4137) ... * update apply_chat_template * fix unittest * fix unittest * fix * fix * fix unit test * fix * fix unit test * add unit test	2025-09-24 18:56:32 +08:00
ApplEOFDiscord	9566ae8827	[Bug Fix] disable prefix caching in mm model (#4167) ... * add http get retry * fix coments * disable prefix caching in mm model * fix unit test --------- Co-authored-by: zhangjunjun04 <zhangjunjun04@baidu.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-24 14:43:46 +08:00
yyssys	d6e59447f5	[XPU] Enable XPU V1 mode based on environment variable (#4213) ... * Enable XPU V1 mode based on environment variable * add default param to xft_moe_fc_block_eb for latest xvllm compatibility; update run_ci_xpu to use latest xvllm	2025-09-24 10:29:48 +08:00
bukejiyu	62d1c48363	[v1 loader]code style (#4204) ... * code style * update	2025-09-23 19:36:00 +08:00
yangjianfengo1	4325b737e7	【FIX】Change the name of sparse attn from moba to plas (#4006) (#4076) ... * 【FIX】Change the name of sparse attn from moba to plas (#4006) * 更新文档 * 【docs】 update readme (#4000) * 更新文档 * update readme * update docs * 【FIX】Change the name of sparse attn from moba to plas (#3845) * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci * code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * fix max_num_seqs * fix test load attn --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-23 10:26:40 +08:00
ltd0924	83720da79f	[Feature] support clear data (#3601) ... * [Feature] support clear data * update * fix * fix * fix * fix * fix * fix * fix	2025-09-23 10:20:02 +08:00
chenjian	918ccdb123	[Feature] Support pd ep deployment with yiyan adapter (#4029) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * Support pd ep deployment with yiyan adapter * Support pd ep deployment with yiyan adapter * refactor cache messager * support scheduler v1 in PD * suppport pd v1 + chunk prefill * suppport pd v1 + chunk prefill * add eplb * support eplb * support eplb * support eplb * support v1 * fix * fix * fix bug * remove eplb support * support prefix cache in P * fix bug * fix bug * support one stop in V1 * fix bug * fix ci * fix ci * fix * fix * fix * fix * fix --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-22 16:41:38 +08:00
lizexu123	c86945ef49	[Feature] support pool (#3827) ... * support pool * update pooling * add pooler_config and check * update * support AutoWeightsLoader load weight * fix * update * delete print * update pre-commit * fix * fix xpu * fix ModelRegistry->model_registry * fix Copilot review * fix pooler.py * delete StepPooler * fix abstract * fix default_loader_v1 * fix Pre Commit * support torch qwen3 dense * add test and fix torch-qwen * fix * fix * adapter ci: * fix review * fix pooling_params.py * fix * fix tasks.py 2025 * fix print and logger * Modefy ModelRegistry and delete AutoWeightsLoader * fix logger * fix test_embedding * fix ci bug * ernie4_5 model_registry * fix test * support Qwen3-Embedding-0.6B tp=1 load * fix extra code * fix * delete fix vocab_size * delete prepare_params_dict * fix:	2025-09-22 14:09:09 +08:00
RichardWooSJTU	91912cc2e1	fix t2i (#4163) ... Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-19 18:07:13 +08:00
YuanRisheng	24180fba0a	[FDConfig]Remove splitwise_role and engine_worker_queue_port in FDConfig (#4147) ... * remove splitwise_role and engine_worker_queue_port * fix xpu * fix xpu * fix xpu * fix unittest * resolve conflct	2025-09-19 17:01:52 +08:00
chen	66a98b44ed	ep support logprob (#4089) (#4151)	2025-09-19 14:07:31 +08:00
chenjian	618ccdbfba	[Feature] Support mixed deployment with yiyan adapter in develop (#3976) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * fix ci * fix for eb5 * fix ci * fix ci * fix ci --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-18 01:52:20 +08:00
gaoziyuan	896e3bb606	[NewFeture]add ep rollout model init and update/clear ep buffer (#4039) ... * fix gid * merge * fix test * fix bug * fix * fix ci	2025-09-17 20:24:53 +08:00
YuanRisheng	2e9e53ff7e	[FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116) ... * remove max_num_batched_tokens in parallel config * remove max_num_seqs * update test case * fix test * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-17 10:43:35 +08:00
chenjian	67e6d8c691	[Feature] Set prefix caching as default (#3814) ... * Set prefix caching as default * Set prefix caching as default * Set prefix caching as default * skip dynamic load scene * fix kill bug * fix kill bug * fix kill bug * fix * fix * fix ci	2025-09-16 20:34:27 +08:00
co63oc	17a27170bc	fix typos (#4093)	2025-09-15 18:33:30 +08:00
co63oc	8466219ec8	fix typos (#3840) ... * fix typos * ci --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-12 11:04:38 +08:00
RichardWooSJTU	82dab8a91a	Add token processor plugin support (#4059) ... * Add token processor plugin support * fix import * fix import	2025-09-12 10:17:23 +08:00
chenjian	37f1632732	[Optimize] optimize prefix cache in develop (#3890) ... * optimize prefix cache in release22 * fix * fix * fix * add ci for v1 * add unit test --------- Co-authored-by: xiegegege <46314656+xiegegege@users.noreply.github.com>	2025-09-12 10:15:59 +08:00
chen	4859f40b20	[Feature] GLM-45-AIR Support Mix Quantization(Dense wfp8afp8 and wint8 triton_moe_backend) (#4051)	2025-09-11 20:08:09 +08:00
Jiang-Jia-Jun	c60adf4281	Revert "【FIX】Change the name of sparse attn from moba to plas (#3845)" (#4001) ... This reverts commit e31c8f7336.	2025-09-09 11:08:23 +08:00
yangjianfengo1	e31c8f7336	【FIX】Change the name of sparse attn from moba to plas (#3845) ... * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci	2025-09-09 10:56:50 +08:00
qwes5s5	17169a14f2	[metrics] Add serveral observability metrics (#3868) ... * Add several observability metrics * [wenxin-tools-584] 【可观测性】支持查看本节点的并发数、剩余block_size、排队请求数等信息 * adjust some metrics and md files * trigger ci * adjust ci file * trigger ci * trigger ci --------- Co-authored-by: K11OntheBoat <your_email@example.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:13:13 +08:00
yinwei	7833f2f6cb	[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER (#3897) ... * fix bug * fix bug * update * update * update	2025-09-08 10:34:46 +08:00
Yuan Xiaolan	2cf55168ca	load hadamard_block_size from config (#3797)	2025-09-05 17:07:58 +08:00
freeliuzc	88d44a2c93	support mtp in v1_scheduler mode (#3695)	2025-09-04 17:39:59 +08:00
RichardWooSJTU	f36a388ffe	fix response processsors (#3826) ... * fix response processsors * fix ci * fix ut --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-04 16:01:25 +08:00
chenjian	22c165d6dd	[Feature] Set v1 scheduler as default in develop (#3807) ... * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * Set scheduler v1 as default * close V1 in guided_decoding * fix vl ci * close V1 in guided_decoding	2025-09-04 15:16:56 +08:00
RichardWooSJTU	0989788b29	support extend block tables (#3824)	2025-09-04 14:39:04 +08:00
co63oc	9ec4fa0f8e	fix typo EngineSevice EngineService (#3841)	2025-09-04 11:20:36 +08:00
Yuanle Liu	306c024ff3	[BugFix] fix error of import paddle.base.core.Config (#3761) ... * 延迟 import Config * support chunked_prefill * support chunked_prefill	2025-09-02 17:23:27 +08:00
kevin	1908465542	[Feature] mm and thinking model support structred output (#2749) ... * mm support structured output * update code * update code * update format * update code * update code * add enable_thinking default * update code * add structured_outputs test case * add ci install xgrammar * add ci timeout time * update test for structured_outputs * update code * add error traceback info * update error msg * update structred output code * update code * update code * update config * update torch version --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-02 16:21:09 +08:00
Jiang-Jia-Jun	0e4df5a6f4	[Feature] Setting number of apiserver workers automatically (#3790) ... Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-09-02 14:17:48 +08:00
kevin	7e751c93ae	[BugFix] Fix chunked prefill (#3759) ... * add error traceback info * update error msg * update code * default enable chunked prefill * update code * update code * add envs * update code * update enable chunked_prefill * update code * update code * update code * update code * update code --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-02 13:40:45 +08:00
co63oc	d6369b4d51	fix typos (#3684)	2025-09-01 17:50:17 +08:00
ming1753	927e8ec55e	Add more runtime information to resource manager (#3706)	2025-09-01 00:25:28 +08:00
chenjian	465065cd19	[Bug fix] Fix prefix cache in V1 (#3715) ... * [Bug fix] Fix prefix cache in V1 * fix code style	2025-08-31 21:29:33 +08:00
kevin	753772ace8	default enable chunked prefill (#3731) ... * add error traceback info * update error msg * update code * default enable chunked prefill * update code * update code * add envs * update code --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-31 13:15:13 +08:00
李泳桦	98e03fb4ea	[feat] add metrics for yiyan adapter (#3219) (#3614) ... * [feat] add metrics for yiyan adapter * [fix] fix metrics num_requests_waiting and num_requests_running * [fix] fix metrics gpu_cache_usage_perc * [refactor] change where requests_number increases * [chore] rename xxx_block_num as xxx_gpu_block_num, and update their values accordingly * [chore] delete useless code	2025-08-30 23:20:58 +08:00
SunLei	b9af95cf1c	[Feature] Add AsyncTokenizerClient&ChatResponseProcessor with remote encode&decode support. (#3674) ... * [Feature] add AsyncTokenizerClient * add decode_image * Add response_processors with remote decode support. * [Feature] add tokenizer_base_url startup argument * Revert comment removal and restore original content. * [Feature] Non-streaming requests now support remote image decoding. * Fix parameter type issue in decode_image call. * Keep completion_token_ids when return_token_ids = False. * add copyright	2025-08-30 17:06:26 +08:00
chen	7568b20098	check (#3720)	2025-08-30 16:04:20 +08:00
yangjianfengo1	3754a9906d	[Feature] block sparse attention (#3668) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config * 修复显存 * add test server * add test server * fix mlp 最后一层使用full attn	2025-08-29 19:46:30 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
Ryan	45f81b34f0	add dtype int32 (#3692)	2025-08-29 14:56:35 +08:00
Yuanle Liu	68f87240da	fix key error in mm (#3702)	2025-08-29 14:35:12 +08:00
Yuanle Liu	2fb2c0f46a	fix MultimodalRegistry (#3699)	2025-08-29 11:01:30 +08:00
Yuanle Liu	4957908275	add input_processor plugin (#3657) ... * add input_processor plugin * update * update * update * update * update * update * update * update * update * update * update	2025-08-28 22:53:57 +08:00