ltd0924
c35a21a99a
[Feature] support fd return decode response ( #4300 )
...
* fix
* fix
* fix
* [Feature] support clear data
* update
* fix
* fix
* fix
* fix
* [BugFix] fix clear data
* Update api_server.py
* Update api_server.py
* [Feature] support fd decode response
* Update engine.py
* Update envs.py
* Update expert_service.py
* Update common_engine.py
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
Co-authored-by: ltd0924 <luotingdan@baidu.com >
2025-09-28 16:11:50 +08:00
freeliuzc
c8985727a6
support mtp in hybird-dp-tp mode ( #4299 )
2025-09-28 15:58:45 +08:00
GoldPancake
076c30cb0f
fix top_p_candidates and support separate setting of sampling params for mtp ( #4189 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* fix top_p_candidates
* For separate setting params for mtp
* delete print
* fix
2025-09-28 11:41:20 +08:00
ltd0924
f8c6a354a1
[BUGFIX] clear request ( #4286 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* fix
* fix
* fix
* [Feature] support clear data
* update
* fix
* fix
* fix
* fix
* [BugFix] fix clear data
* Update api_server.py
* Update api_server.py
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-27 14:08:48 +08:00
freeliuzc
b176cba474
support mtp in ep64 ( #4280 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-26 15:38:03 +08:00
Yuanle Liu
dcf633c4d9
delete default value reasoning_max_tokens ( #4250 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* delete default value reasoning_max_tokens
* Adjust max_tokens and reasoning_max_tokens logic
2025-09-26 10:42:27 +08:00
Zhong Hui
213f15ef55
fix ernie vl distributed attr. ( #4259 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-25 20:06:29 +08:00
lizhenyun01
bab779011c
[CudaGraph] support cudagraph use shared pool ( #4199 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* support cudagraph use shared pool
* add envs
* change CUDAGRAPH_POOL_ID to int
* change CUDAGRAPH_POOL_ID to use_memory_pool
* unify use_unique_memory_pool
* fix use_unique_memory_pool
2025-09-24 21:32:04 +08:00
freeliuzc
e2b68b33c9
fix mtp in rl ( #4234 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-24 16:59:24 +08:00
ltd0924
1aab1c8d06
[BugFix] fix clear data ( #4227 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* [Feature] support adapter
* fix
* fix
* fix
* fix
* fix
* fix
* [BugFix] fix clear data
* Update api_server.py
2025-09-24 11:23:44 +08:00
freeliuzc
94b6e7a341
[MTP][RL]support rl reshard wenxin-tools-145 ( #4173 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* support mtp reshard in rl mode
* fix function
2025-09-23 20:40:26 +08:00
Yuanle Liu
389c5dd3a2
Each module should have its own plugins_loaded ( #4149 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-23 15:44:46 +08:00
Yuanle Liu
361104508e
support reasoning_max_tokens ( #4207 )
2025-09-23 15:44:41 +08:00
ltd0924
f489c9f8ef
[Feature] support adapter ( #4180 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* [Feature] support adapter
* fix
* fix
* fix
* fix
* fix
* fix
2025-09-22 19:32:24 +08:00
lzy
be98f6e950
supports internode_ll_two_stage ( #4143 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* supports internode_ll_two_stage
* supports internode_ll_two_stage
* supports internode_ll_two_stage
* supports internode_ll_two_stage
2025-09-22 14:55:06 +08:00
ltd0924
f75697c2d1
[Feature] support clear data ( #4185 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* fix
* fix
* fix
* [Feature] support clear data
* update
* fix
* fix
* fix
* fix
2025-09-21 20:41:27 +08:00
gaoziyuan
5027ed7239
【BugFif】fix ep decode ( #4138 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* suppoort expert num 3 pre rank
* fix ep decode
2025-09-17 14:18:31 +08:00
Yuan Xiaolan
25aa2d94aa
cp dynamic Cfp8 ( #4120 )
...
* supports dynamic Cfp8
* add unittest
* fix dynamic Cfp8 computing error
* fix Cfp8 for RL load
---------
Co-authored-by: carryyu <569782149@qq.com >
2025-09-17 11:55:47 +08:00
Yuanle Liu
d381fa8194
fix reasoning parsers plugin ( #4104 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-15 22:30:16 +08:00
freeliuzc
d2ab369427
[MTP]Support RL reshard ( #4074 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* support rl reshard
* modify model name
2025-09-15 11:47:06 +08:00
Yuanle Liu
2883746132
fix model_weights_signal ( #4092 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* fix model_weights_signal
2025-09-13 11:55:25 +08:00
chen
2485333f71
ep support logprob ( #4089 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
2025-09-12 21:11:16 +08:00
gaoziyuan
10768a4d79
[NewFeture]add ep rollout model init and update/clear ep buffer ( #3927 )
...
* add ep rollout model init && add deep update/clear
* fix test
2025-09-12 14:15:13 +08:00
gaoziyuan
447297a7b5
fix gid ( #4054 )
...
Co-authored-by: Divano <dddivano@outlook.com >
2025-09-11 16:08:00 +08:00
RAM
63d24b2210
[Executor] Adjust signal sending order in RL training ( #3773 ) ( #4066 )
...
* Adjust processing order
* fix bug
* fix update_parameters bug
* refine code
2025-09-11 15:41:32 +08:00
Yuanle Liu
48f2ab3fb3
support cuda graph ( #4056 )
...
* support cuda graph
* upstate
2025-09-11 11:38:32 +08:00
ltd0924
749f074e44
Update multi_api_server.py ( #4023 )
2025-09-10 17:15:01 +08:00
guozhuangzhuang
f06e3ee1fc
Use uuid to name the metrics shared folder ( #4025 )
...
* Use uuid to name the metrics shared folder
* Use uuid to name the metrics shared folder test case
2025-09-10 16:58:13 +08:00
freeliuzc
2f473ba966
[Feature][MTP]Support MTP for rl-model ( #4009 )
...
* qk norm for speculate decode C16
* support mtp in v1_scheduler mode
* support mtp rope_3d
* support mtp features
* add unit test && del some log
---------
Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com >
Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com >
2025-09-10 13:34:37 +08:00
Yuanle Liu
cce2410fad
Fix parameter shape for down projection weight ( #4028 )
2025-09-09 17:28:04 +08:00
Zero Rains
d8985a7a21
get org_vocab_size from args ( #3985 )
...
Co-authored-by: Yuanle Liu <yuanlehome@163.com >
2025-09-09 15:08:58 +08:00
Yuanle Liu
71a9127e13
Update args_utils.py
2025-09-08 01:41:43 -07:00
lizhenyun01
d40a1046de
[Feature] support rl_tp_degree ( #3934 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* [Feature] support rl_tp_degree
* add rl_tp_degree in lmhead
* add rl_tp_degree in bias
* fix split_axis=0 in bias
* fix split_axis in weight
* fix bias rl_tp_degree
* fix bias rl_tp_degree
* change attr to dict
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-08 16:20:32 +08:00
luukunn
1023a67765
[BugFix] fix default parser ( #3932 )
...
* add reasoning parser plugin
* fix finish reason
* fix default parser
---------
Co-authored-by: Yuanle Liu <yuanlehome@163.com >
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-08 14:12:13 +08:00
Zero Rains
d43549953c
[Cherry-Pick][Bug Fix]fix the bug for real size 0 in cudagraph ( #3888 )
...
* fix the bug for real size 0 in cudagraph
* fix cache_messager
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-08 14:06:10 +08:00
ming1753
d6bf6de5e6
[Bug Fix] Fix mm performance degradation ( #3942 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* [Bug Fix] Fix mm performance degradation
* formate
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
Co-authored-by: chenjian <1435317881@qq.com >
2025-09-08 00:32:22 +08:00
chenjian
38e734e183
[Feature] support hierarchical cache in v1 ( #3939 )
2025-09-08 00:31:34 +08:00
chenjian
b2bb37d7c0
[Fix] when prompt token ids is numpy ( #3944 )
2025-09-07 23:02:03 +08:00
CSWYF3634076
c6e2a37a95
[BugFix] qwen2.5vl enable_thinking=true bug fix ( #3920 )
2025-09-07 21:06:36 +08:00
chenjian
8d77c1cb51
[Optimize] optimize prefix cache in release22 ( #3889 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* optimize prefix cache in release22
* optimize prefix cache in release22
* fix worker
* fix
* fix
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-06 09:52:01 +08:00
chenjian
41cd3e24c9
[Feature] Enable prefix caching as default ( #3816 )
...
* [Feature] Enable prefix caching as default
* [Feature] Enable prefix caching as default
* Set prefix caching as default
* skip dynamic load
* fix kill bug
* fix kill bug
* fix kill bug
* fix ci
* fix
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-06 09:51:34 +08:00
freeliuzc
e2c764fd5a
update hybrid-mtp-with-ngram ( #3924 )
2025-09-05 23:06:57 +08:00
lizhenyun01
2d975e16b0
[BugFix] fix TaskQueue dp_id in multi node ( #3919 )
2025-09-05 22:29:26 +08:00
chenjian
8915c8411d
Revert "[Feature] Setting number of apiserver workers automatically ( #3794 )" ( #3918 )
...
This reverts commit d1d063e4af
2025-09-05 21:06:50 +08:00
yinwei
77c1bd0813
[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER ( #3900 )
...
* fix bug
* fix bug
* update
* udpate
* update
2025-09-05 19:17:25 +08:00
chen
335d1c8e8f
【CP】Compatible with EB 0.3B torch model arch ( #3914 )
...
* fix
* check
2025-09-05 19:05:07 +08:00
ltd0924
173e4df982
[Fix] mv connection_manager init ( #3902 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* Update serving_chat.py
* Update serving_completion.py
* Update serving_completion.py
* mv connection_manager init
---------
Co-authored-by: Yuanle Liu <yuanlehome@163.com >
2025-09-05 17:42:36 +08:00
lizhenyun01
199f88ce1e
support tpep weight load ( #3882 )
2025-09-05 13:56:29 +08:00
ltd0924
55ebe855c0
[Feature] support controller port in multi api server ( #3895 )
...
* fix scheduler bug
* fix
* Update api_server.py
* Update multi_api_server.py
2025-09-05 13:38:58 +08:00
zhouchong
deb7ad205f
fix qwen_vl_processor miss image_patch_id ( #3894 )
...
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-05 11:32:34 +08:00
