26 Commits

Author	SHA1	Message	Date
ltd0924	de4feff147	[Feature]CP support data clear (#4214) ... * Update serving_chat.py * Update serving_completion.py * Update serving_completion.py * mv connection_manager init * [BugFix] fix kv cache * fix format * [Feature] support clear data --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com> Co-authored-by: RAM <gstian5555@outlook.com>	2025-09-23 16:53:39 +08:00
chenjian	4f8ff478b3	[Feature] Support mixed deployment with yiyan adapter in release22 (#3974) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * add unit test * add unit test * add unit test	2025-09-10 16:01:13 +08:00
李泳桦	98e03fb4ea	[feat] add metrics for yiyan adapter (#3219) (#3614) ... * [feat] add metrics for yiyan adapter * [fix] fix metrics num_requests_waiting and num_requests_running * [fix] fix metrics gpu_cache_usage_perc * [refactor] change where requests_number increases * [chore] rename xxx_block_num as xxx_gpu_block_num, and update their values accordingly * [chore] delete useless code	2025-08-30 23:20:58 +08:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
YuanRisheng	c389a4013c	Unify server-side and model-side Config(Part-5) (#3497) ... * move config * fix xpu * fix * fix vl * fix vl * fix unitest * fix args * add unitest * fix test	2025-08-21 19:00:21 +08:00
kevin	67298cf4c0	add error traceback info (#3419) ... * add error traceback info * update error msg * update code --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-19 19:32:04 +08:00
Zero Rains	b23af29d0b	Launch expert_service before kv_cache initialization in worker_process (#3045) ... * launch expert_service before kv_cache initialization * add two signal make sure model loading and expert_service lauching finished * fix the EP bug * fix ep * update launching way * fix ep * update * roback ep * pre-commit all files --------- Co-authored-by: RAM <gstian5555@outlook.com> Co-authored-by: Divano <dddivano@outlook.com>	2025-08-11 19:38:46 +08:00
chenjian	b88537a456	fix bug for scheduler v0 (#3308)	2025-08-11 13:07:04 +08:00
chen	46c8491201	merge logprob into batch_output (#3266)	2025-08-11 10:03:00 +08:00
chenjian	c011cb8b16	[Bug Fix] Fix scheduler bug in develop (#3292) ... * Fix scheduler bug in develop * Fix scheduler bug in develop * Fix scheduler bug in develop	2025-08-10 13:55:38 +08:00
chenjian	5f0b30f6d0	support logprob in scheduler v1 (#3249) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-07 20:14:01 +08:00
SunLei	dade19d7a4	[Feature] General support for logprobs (#2974) ... * [Feature] support logprobs in chat/completions and completions endpoints * Temporarily comment out text_offset due to incorrect logic * Clean up temporary debug prints * [Feature] support logprobs in offline mode via SamplingParams * fix: serialize Logprob as dict before zmq send to fix msgpack error * refactor: remove redundant methods to simplify codebase * Fix missing fields in CompletionOutput.to_dict affecting msgpack serialization * refactor: centralize param validation in engine_client to reduce duplication * revert: rollback changes in offline_demo.py * revert: rollback changes in offline_demo.py * [bugfix] fix parameter validation for logprobs * [bugfix] fix parameter validation for logprobs * [bugfix] fix parameter validation for logprobs * [bugfix] fix parameter validation for logprobs --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-31 20:25:56 +08:00
ltd0924	ecf2fd5b9a	[BugFix] vl encoder tokens dtype problem (#3069)	2025-07-30 15:20:53 +08:00
zhuzixuan	ad7bb52a28	修复传入max_tokens=1时的报错 (#3068) ... * 修复传入max_tokens=1时的报错 * 修复传入max_tokens=1时的报错 * 修复传入max_tokens=1时的报错 * 修复传入max_tokens=1时的报错 * 修复传入max_tokens=1时的报错 * 修复传入max_tokens=1时的报错	2025-07-29 23:49:28 +08:00
chenjian	85a78d695d	[Feature] Support block scheduler v1 for FD (#2928) ... * Support FD block scheduler v1 * Support FD block scheduler v1 * Support FD block scheduler v1 * Fix according to copilot review * Fix according to review * Remove is_dummy * Fix bug when real_bsz=1 * Fix infer first token cost time --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-23 20:31:31 +08:00
GoldPancake	dc67c10a7e	[Feature][MTP]Support multi-step MTP (#2952)	2025-07-22 16:26:29 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
GoldPancake	42d4001400	[Features] Add speculative metrics (#2857)	2025-07-17 11:08:55 +08:00
ltd0924	d245d1ca6c	[LLM] support send batch data and aggregate data (#2860) ... * [LLM] support send batch data and aggregate data * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] update	2025-07-16 23:42:20 +08:00
chen	d33105baeb	[Feature] Online Chat API Support Return logprobs (#2777) ... * online chat support logprobs * check xpu * check vl_gpu_model_runner and xpu_model_runner * get_worker() check platform	2025-07-10 16:33:40 +08:00
EnflameGCU	d0f4d6ba3a	[GCU] Support gcu platform (#2702) ... baseline: e7fa57ebae Co-authored-by: yongqiangma <xing.wo@163.com>	2025-07-08 13:00:52 +08:00
liddk1121	1b54a2831e	Adapt for iluvatar gpu (#2684)	2025-07-07 16:53:14 +08:00
ltd0924	00863c43fd	[Bug] fix logger format (#2689)	2025-07-03 19:58:03 +08:00
Jiang-Jia-Jun	05c670e593	[Sync] Update to latest code (#2679) ... * [Sync] Update to latest code * Add new code files * Add new code files * update code * Try to fix build.sh * Try to fix build.sh * Update code * Update requirements.txt * Update code --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-07-03 15:43:53 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00
jiangjiajun	fb18f3092d	[LLM] Add output module and polish docs	2025-06-09 20:26:53 +08:00