18 Commits

Author	SHA1	Message	Date
ltd0924	c5591c45df	[BugFix] fix max streaming tokens invalid (#3774) ... * Update serving_chat.py * Update serving_completion.py	2025-09-02 21:00:29 +08:00
xiaolei373	f0519aec67	feat(log):add_request_and_response_log (#3391) ... * feat(log):add_request_and_response_log * [ci] Retrigger * [ci] Retrigger	2025-08-14 19:12:42 +08:00
李泳桦	eca8fc7ca6	[feat] extra parameters are all passed directly via http payload now, or in extra_body if using openai client (#3077) ... * [feat] extra parameters are all passed directly via http payload now, or in extra_body if using openai client * [fix] delete ci test case for enable_thinking * [fix] add reasoning_parser when server starts * [doc] update docs related to metadata * [fix] fix ci consistency test error with reasoning parser * [fix] cancel enable_thinking default value	2025-07-30 19:25:39 +08:00
Zero Rains	0fb37ab7e4	update flake8 version to support pre-commit in python3.12 (#3000) ... * update flake8 version to support pre-commit in python3.12 * polish code	2025-07-24 01:43:31 -07:00
ltd0924	f935d6f862	[BugFix] fix multinode deployment (#2977)	2025-07-24 15:04:04 +08:00
李泳桦	2a8a2c06de	[fix] non-streaming api now returns full output ids if return_token_ids is enabled (#2951)	2025-07-22 14:35:56 +08:00
李泳桦	8a619e9db5	[Feature] Add return_token_ids, prompt_token_ids, and delete training, raw_request in request body (#2940) ... * [feat] add return_token_ids, prompt_token_ids, delete raw_request in request body * [fix] return_token_ids not working in curl request * [test] improve some test cases of return_token_ids and prompt_token_ids * [fix] the server responds ok even if request.messages is an empty list	2025-07-21 19:31:14 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
Jiang-Jia-Jun	fbe3547c95	[Feature] Support include_stop_str_in_output in chat/completion (#2910) ... * [Feature] Support include_stop_str_in_output in chat/completion * Add ci test for include_stop_str_in_output * Update version of openai * Fix ci test --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-07-18 16:59:18 +08:00
ltd0924	9c25dcca0b	[LLM] Update Multinode Deployment (#2830) ... * [LLM] fix multinode bugs * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] update multinode deployment * [LLM] fix ci bugs * Update fastdeploy/engine/args_utils.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [LLM] update random port * [LLM] update random port * [LLM] fix ci bugs * fix ci bugs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-07-16 23:42:54 +08:00
ltd0924	d245d1ca6c	[LLM] support send batch data and aggregate data (#2860) ... * [LLM] support send batch data and aggregate data * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] fix ci bugs * [LLM] update	2025-07-16 23:42:20 +08:00
zhenwenDang	d48c03413f	Feature/logprob bug fix (#2817) ... * fix: handle missing logprobs at step 0 and incorrect finish reason with max_completion_tokens * Prevent response_logprobs.logprob_token_ids[0] from going out of bounds	2025-07-12 16:48:51 +08:00
chen	d33105baeb	[Feature] Online Chat API Support Return logprobs (#2777) ... * online chat support logprobs * check xpu * check vl_gpu_model_runner and xpu_model_runner * get_worker() check platform	2025-07-10 16:33:40 +08:00
ltd0924	68b4755587	[LLM] support multi node deploy (#2708) ... * [LLM] support multi node deploy * Update engine.py * fix bugs * fix * [LLM] support multi node deploy * [LLM] support multi node deploy --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-06 10:33:51 +08:00
ltd0924	87e638498c	[RL] update reschedule finish reason (#2709)	2025-07-04 13:47:36 +08:00
Jiang-Jia-Jun	05c670e593	[Sync] Update to latest code (#2679) ... * [Sync] Update to latest code * Add new code files * Add new code files * update code * Try to fix build.sh * Try to fix build.sh * Update code * Update requirements.txt * Update code --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>	2025-07-03 15:43:53 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00
jiangjiajun	684703fd72	[LLM] First commit the llm deployment code	2025-06-09 19:20:15 +08:00