987 Commits

Author	SHA1	Message	Date
luukunn	aebe12a58d	[fix]update apply_chat_template (#4249) ... * [fix]Modify follow-up push parameters and Modify the verification method for thinking length (#4086) * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * add completion_token_ids * add logger * fix reasoning_max_tokens ParameterError * add unittest * add unittest * add unittest * add unittest * add unittest * add unit test * fix * [fix]update apply_chat_template (#4137) * update apply_chat_template * fix unittest * fix unittest * fix * fix * fix unit test * fix * fix unit test * add unit test	2025-09-25 16:41:56 +08:00
chen	8fdb950e9f	include_stop_str_in_output=False not return eos text (#4231)	2025-09-24 14:07:30 +08:00
Zhong Hui	a460462d2a	fix ernie vl distributed attr. (#4217)	2025-09-23 19:37:38 +08:00
李泳桦	cb8d87b945	[fix] fix clearing caches synchronization and add more logs (#4212) ... * [fix] fix clearing caches synchronization and add more logs * [chore] print cache_ready_signal in log	2025-09-23 19:36:38 +08:00
ltd0924	de4feff147	[Feature]CP support data clear (#4214) ... * Update serving_chat.py * Update serving_completion.py * Update serving_completion.py * mv connection_manager init * [BugFix] fix kv cache * fix format * [Feature] support clear data --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com> Co-authored-by: RAM <gstian5555@outlook.com>	2025-09-23 16:53:39 +08:00
chen	f38b174a75	Fix noaux_tc cuda Error 700 in CUDAGraph and Add wfp8apf8 moe quant method (#4115) ... * improve per_token_quant_fp8 performance * support moe wfp8apf8 * check glm test * fix noaux_tc op in cudagraph, support noaux_tc return the correct * check * check inf and overwrite score in noaux_tc --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-22 21:27:37 +08:00
luukunn	6b47773bd6	[fix]Modify follow-up push parameters and Modify the verification method for thinking length (#4177) ... * [fix]Modify follow-up push parameters and Modify the verification method for thinking length (#4086) * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * 续推参数 generated_token_ids 修改成 completion_token_ids;修改思考长度校验方式 * add completion_token_ids * add logger * fix reasoning_max_tokens ParameterError * add unittest * add unittest * add unittest * add unittest * add unittest * add unit test * fix	2025-09-22 21:12:05 +08:00
李泳桦	0358329946	[fix] initialize available_gpu_block_num with max_gpu_block_num (#4193)	2025-09-22 18:56:00 +08:00
RAM	01f6934162	[Executor] Adjust signal sending order in RL training (#3773) (#4066) (#4178) ... * Adjust processing order * fix bug * fix update_parameters bug * refine code	2025-09-22 14:31:36 +08:00
chen	7bdc6f41e5	fix glm all_reduce tp group (#4188)	2025-09-22 10:57:13 +08:00
ltd0924	bba279cf38	[Feature] support rdma IB transfer (#4123) ... * Update serving_chat.py * Update serving_completion.py * Update serving_completion.py * mv connection_manager init * [BugFix] fix kv cache * fix format --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-19 12:54:49 +08:00
Sunny-bot1	4f460db556	[CP2.2] Machete support group scale & wint8 & v1 loader (#4166) ... * support v1 loader for machete (#3999) * [Optimize] Support WINT8 and group scale for Machete (#3905) * [Optimize] Machete using group scale default (#4121)	2025-09-19 11:13:12 +08:00
JYChen	74d7b9151d	fix mtp (#4153) ... Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>	2025-09-18 10:53:07 +08:00
李泳桦	0fa28b1068	[fix] fix ep group all-reduce (#4140) ... * [fix] fix ep group all-reduce * [fix] fix clear/update lock not working when workers > 1 * [chore] add preemption triggered info log * [fix] fix code style * fix model_weights_signal (#4092) * fix model_weights_signal --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-18 10:34:49 +08:00
Jiang-Jia-Jun	cffde70949	Add assertion for ENABLE_V1_KVCACHE_SCHEDULER (#4146)	2025-09-17 16:02:56 +08:00
K11OntheBoat	7f9a9b37f3	Support limit thinking lengths (#4070) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-09-17 12:40:08 +08:00
gaoziyuan	b41988f4bc	fix gid (#4038)	2025-09-16 20:56:36 +08:00
李泳桦	7ccbcc5a62	[feat] support prefix cache clearing when /clear_load_weight is called (#4091) ... * [feat] support clearing prefix cache (cherry-picked from release/2.1) * [fix] fix ipc suffix, use port instead * [fix] fix prefix caching not enabled * [fix] fix code style * [fix] wait for rank0 to update weight status	2025-09-16 11:11:20 +08:00
chen	fbb4e0f8d1	[CP]Glm45 air 2.2 (#4073) ... * [Feature] Support zai-org/GLM-4.5-Air BF16 model (#3928) * support glm45_air * [Feature] GLM-45-AIR Support Mix Quantization(Dense wfp8afp8 and wint8 triton_moe_backend) (#4051) * check * fix v1 load for mix and wint8 * check --quantizations 'None' * check * support RL rollout * check v1 loader * check glm rollout_model, change wfp8afp8 per_token_cast_to_fp8 to native impl * check rollout moe gate begin layer_id * check rollout e_score_correction_bias * delete infer_to_train_mapping={} * code check	2025-09-15 18:52:58 +08:00
chenjian	4f8ff478b3	[Feature] Support mixed deployment with yiyan adapter in release22 (#3974) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * add unit test * add unit test * add unit test	2025-09-10 16:01:13 +08:00
guozhuangzhuang	c4098d56a0	Fixed the issue of metrics file conflicts between multiple instances … (#4010) ... * Fixed the issue of metrics file conflicts between multiple instances on a single machine * Use uuid to name the metrics shared folder * Use uuid to name the metrics shared folder	2025-09-10 13:48:24 +08:00
ltd0924	a6b161b007	[Fix] fix multi api server log dir (#3966) ... * fix scheduler bug * fix * Update api_server.py * Update multi_api_server.py * [Fix]	2025-09-10 13:48:17 +08:00
Yuanle Liu	7272afe3dc	Fix down projection weight shape in fused MOE layer (#4041)	2025-09-10 12:49:03 +08:00
yangjianfengo1	dfc94371ee	【FIX】Change the name of sparse attn from moba to plas (#4006) ... * 更新文档 * 【docs】 update readme (#4000) * 更新文档 * update readme * update docs * 【FIX】Change the name of sparse attn from moba to plas (#3845) * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci * code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-10 10:04:29 +08:00
Zero Rains	35b8362804	get org_vocab_size from args (#3984)	2025-09-09 15:07:51 +08:00
zhuzixuan	d43c2f2577	[Optimize]Error messages about Model api. (#3839) (#3972) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py * Update tests/entrypoints/openai/test_serving_models.py * Update tests/entrypoints/openai/test_serving_models.py * Update tests/entrypoints/openai/test_serving_models.py * Update fastdeploy/entrypoints/openai/serving_models.py * 优化报错信息。 --------- Co-authored-by: yangzichao01 <yangzichao01@baidu.com> Co-authored-by: Yzc216 <101054010+Yzc216@users.noreply.github.com> Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-09-09 10:58:11 +08:00
lizhenyun01	d40a1046de	[Feature] support rl_tp_degree (#3934) ... * [Feature] support rl_tp_degree * add rl_tp_degree in lmhead * add rl_tp_degree in bias * fix split_axis=0 in bias * fix split_axis in weight * fix bias rl_tp_degree * fix bias rl_tp_degree * change attr to dict --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 16:20:32 +08:00
luukunn	1023a67765	[BugFix] fix default parser (#3932) ... * add reasoning parser plugin * fix finish reason * fix default parser --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:12:13 +08:00
Zero Rains	d43549953c	[Cherry-Pick][Bug Fix]fix the bug for real size 0 in cudagraph (#3888) ... * fix the bug for real size 0 in cudagraph * fix cache_messager --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:06:10 +08:00
ming1753	d6bf6de5e6	[Bug Fix] Fix mm performance degradation (#3942) ... * [Bug Fix] Fix mm performance degradation * formate --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: chenjian <1435317881@qq.com>	2025-09-08 00:32:22 +08:00
chenjian	38e734e183	[Feature] support hierarchical cache in v1 (#3939)	2025-09-08 00:31:34 +08:00
chenjian	b2bb37d7c0	[Fix] when prompt token ids is numpy (#3944)	2025-09-07 23:02:03 +08:00
CSWYF3634076	c6e2a37a95	[BugFix] qwen2.5vl enable_thinking=true bug fix (#3920)	2025-09-07 21:06:36 +08:00
chenjian	8d77c1cb51	[Optimize] optimize prefix cache in release22 (#3889) ... * optimize prefix cache in release22 * optimize prefix cache in release22 * fix worker * fix * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-06 09:52:01 +08:00
chenjian	41cd3e24c9	[Feature] Enable prefix caching as default (#3816) ... * [Feature] Enable prefix caching as default * [Feature] Enable prefix caching as default * Set prefix caching as default * skip dynamic load * fix kill bug * fix kill bug * fix kill bug * fix ci * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-06 09:51:34 +08:00
freeliuzc	e2c764fd5a	update hybrid-mtp-with-ngram (#3924)	2025-09-05 23:06:57 +08:00
lizhenyun01	2d975e16b0	[BugFix] fix TaskQueue dp_id in multi node (#3919)	2025-09-05 22:29:26 +08:00
chenjian	8915c8411d	Revert "[Feature] Setting number of apiserver workers automatically (#3794)" (#3918) ... This reverts commit d1d063e4af.	2025-09-05 21:06:50 +08:00
yinwei	77c1bd0813	[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER (#3900) ... * fix bug * fix bug * update * udpate * update	2025-09-05 19:17:25 +08:00
chen	335d1c8e8f	【CP】Compatible with EB 0.3B torch model arch (#3914) ... * fix * check	2025-09-05 19:05:07 +08:00
ltd0924	173e4df982	[Fix] mv connection_manager init (#3902) ... * Update serving_chat.py * Update serving_completion.py * Update serving_completion.py * mv connection_manager init --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-05 17:42:36 +08:00
lizhenyun01	199f88ce1e	support tpep weight load (#3882)	2025-09-05 13:56:29 +08:00
ltd0924	55ebe855c0	[Feature] support controller port in multi api server (#3895) ... * fix scheduler bug * fix * Update api_server.py * Update multi_api_server.py	2025-09-05 13:38:58 +08:00
zhouchong	deb7ad205f	fix qwen_vl_processor miss image_patch_id (#3894) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-05 11:32:34 +08:00
chenjian	8567ada09e	[Fix] disable scheduler v1 in guided decoding (#3877) ... * disable scheduler v1 in guided decoding * disable scheduler v1 in guided decoding	2025-09-04 20:54:55 +08:00
lizhenyun01	d40d3a5a4f	fix DP&&TP (#3872)	2025-09-04 14:38:26 +08:00
luukunn	b8d0f1c081	[bug] fix finish reason (#3858) ... * add reasoning parser plugin * fix finish reason --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-09-04 14:36:03 +08:00
ltd0924	8550e19008	[bugfix] scheduler (#3871) ... * fix scheduler bug * fix * Update api_server.py	2025-09-04 11:34:12 +08:00
chenjian	a0c03510c0	[Bug fix] Fix prompt token ids dtype in v1 (#3861)	2025-09-04 11:02:37 +08:00
chenjian	fb1e0d6a87	[Feature] Set scheduler v1 as default (#3812) ... * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default	2025-09-04 11:02:10 +08:00