51 Commits

Author	SHA1	Message	Date
GoldPancake	e56c4dd0a8	[Cherry-Pick] Support for request-level speculative decoding metrics monitoring.(#5518) (#5614) ... * support spec metrics monitor per request	2025-12-17 20:53:04 +08:00
qwes5s5	d67b64d5e1	add detoken switch (#5463) (#5572)	2025-12-17 17:04:45 +08:00
qwes5s5	a52aea073c	fix logprobs (#5335)	2025-12-04 10:38:51 +08:00
ming1753	5f8d4aedea	[Feature] support audio tts (#5333)	2025-12-03 21:06:48 +08:00
qwes5s5	117980dd4e	[LogProbs]Enable prompt logprobs output and modify data transmission method for the online interface. (#5089) ... * add prompt logprobs * Merge prompt_logprobs_tensors and prompt_logprobs * fix param check * trigger ci * fix unitest * fix logprobs bug	2025-12-02 13:49:51 +08:00
xiaolei373	84e2f6aa75	[CI]add clear to run-batch ci (#5307)	2025-12-01 21:18:19 +08:00
xiaolei373	b52ec268f7	[CI]fix run batch unit test (#4628)	2025-11-27 19:38:04 +08:00
fl0w2o48	e63d715fc3	[BugFix][Metrics] Fix Prometheus Multiprocess Metrics Issues and Add ZMQ Communication Metrics (#5185) ... * [Feature] add metrics for ZMQ and fix multiprocess metrics * fix test_metrics.py --------- Co-authored-by: Jiaxin Sui <95567040+plusNew001@users.noreply.github.com>	2025-11-27 15:05:09 +08:00
SunLei	c424e08dc5	[Speculative Decoding] split draft_tokens into standalone post-processing path (#5205) ... * refactor(mtp): split draft_tokens into standalone post-processing path for MTP + logprobs * Restore Request.__repr__ implementation * ci * add envs * fix unittest	2025-11-27 11:22:41 +08:00
Yonghua Li	09379183e2	[BugFix] fix work metrics not returned by metrics api (#4912) ... * [BugFix] fix work metrics not returned by metrics api * [fix] fix conflict * [fix] fix ci	2025-11-25 19:12:29 +08:00
Echo-Nie	a418d7b60b	[CI] Add Unittest (#5187) ... * add test * Delete tests/model_executor/test_w4afp8.py * Rename test_utils.py to test_tool_parsers_utils.py * add test * add test * fix platforms * Delete tests/cache_manager/test_platforms.py * dont change Removed copyright notice and license information.	2025-11-25 11:00:34 +08:00
周周周	385fe6dade	[Others] clean code (#5133)	2025-11-20 18:44:08 +08:00
kxz2002	97189079b9	[BugFix] unify max_tokens (#4968) ... * unify max tokens * modify and add unit test * modify and add unit test * modify and add unit tests --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-11-18 20:01:33 +08:00
LiqinruiG	33f96ff93a	[BugFix] rollback max_tokens and min_tokens when continue to infer (#5052) ... Co-authored-by: liqinrui <liqinrui@baidu.com>	2025-11-17 14:31:26 +08:00
zhouchong	5444af6ff6	[APIServer] metrics use port the same as api_port (#5016) ... * metrics use port the same as api_port * Be tolerant to tests that monkeypatch/partially mock args. * Reduce code redundancy --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-11-17 10:42:45 +08:00
kxz2002	9703108c28	[BugFix] adjust max_tokens and min_tokens when continue to generate tokens (#5010) ... * fix max and min tokens initial commit * fix double subtraction * add unit tests	2025-11-13 23:52:54 +08:00
kxz2002	87911b7cf1	[Feature] Enable FastDeploy to support adding the “--api-key” authentication parameter. (#4806) ... * add api key initial commit * add unit test * modify unit test * move middleware to a single file and add unit tests	2025-11-08 18:24:02 +08:00
kxz2002	9676cc87d6	fix parser register name (#4795) ... Co-authored-by: luukunn <83932082+luukunn@users.noreply.github.com>	2025-11-05 11:27:30 +08:00
luukunn	7b35488779	【DataProcessor】add options thinking_mode (#4735) ... * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add unit test	2025-11-03 14:30:07 +08:00
kxz2002	a2870ed4a9	[Feature] Unify the registration name recognition for tool_parser and reasoning_parser to “-” (#4668) ... * parser register name unify * change ernie_x1 to ernie-x1 * change ernie4_5_vl to ernie-45-vl * fix unit test	2025-10-31 10:45:27 +08:00
kxz2002	82bd7e5db4	[BugFix] Fix finish reason in _create_chat_completion_choice (#4582) ... * fix n_param _create_chat_completion_choicel * fix unit test * fix final_res * modify unit tests	2025-10-31 10:42:19 +08:00
kxz2002	c30bfb294f	[Feature] add a new reasoning parser (#4571) ... * add new reasoning_parser initial commit * add parser file content * add register * ernie_test_reasoning_parser * support <tool_call> token and add tool_parser * add and fix unit tests * modify reasoning_parser * modify reasoning parser and tool parser * modify unit tests * modify reasoning_parser and tool_parser * modify unit tests * fix tool_parser * modify the logic of reasoning_parser and tool_parser * add and modify unit tests * standardize code style * simplify reasoning_parser and tool_parser * modify unit test	2025-10-29 18:16:50 +08:00
ApplEOFDiscord	14f8cddaf1	[Feature] add mm token usage (#4570) ... * add mm token usage * fix unit test * fix unit test * fix unit test * fix model path * fix unit test * fix unit test * fix unit test * remove uncomment * change var name * fix code style * fix code style * fix code style * fix code style * fix unit test	2025-10-29 14:37:12 +08:00
xiaolei373	14e7d88ea4	[feature] support reward api (#4518) ... Co-authored-by: SunLei <sunlei5788@gmail.com>	2025-10-29 00:20:28 +08:00
SunLei	2a9ed72533	feat: add support for API usage with multimodal models (#4548) ... * feat: add support for API usage with multimodal models * completion_tokens contains num_image_tokens * remove test_request.py * fix: paddle.device.is_compiled_with_cuda() * fix test_unstream_without_logprobs	2025-10-28 20:23:46 +08:00
xiaolei373	31180a6a13	fix_run_batch_unittest (#4613)	2025-10-28 10:38:06 +08:00
kxz2002	327fa4c255	[DataProcessor] add reasoning_tokens into usage info (#4520) ... * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test	2025-10-25 16:57:58 +08:00
luukunn	bbf06b9ff7	[BugFix]Fix finish reason (#4543) ... * fix finish reason * add unit test * add unit test * fix unie test * fix unit test	2025-10-23 14:04:43 +08:00
SunLei	809c1ac7ec	feat: add post-processing step for pool_output (#4462) ... * feat: add post-processing step for pool_output * bugfix * fix: test_serving_embedding * fix test_request_to_batch_dicts * fix: code style	2025-10-21 20:24:26 +08:00
SunLei	ee915220bd	[Speculative Decoding] Add draft_logprobs Support for Speculative Decode MTP (#4467) ... * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix: postprocess for speculative decode * test: test_speculative_decoding_use_logprobs * fix: test_completion_echo * fix test_max_streaming_tokens --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 14:57:50 +08:00
kxz2002	b5b993e48e	【feature】support n parameter (#4273) ... * support n parameter * pre-commit check * pre-commit check * restore format_and_add_data * update n_param * bug fix index - str to int * bug fix del child_task * bug fix metrics * add debug info * add debug info2 * remove debug info * change connecting symbol to '-' * bugfix change connecting symbol * bugfix change connecting symbol2 * unit tests fix * unit test fix2 * unittest add param n=2 * n param add unit tests and adapt to echo * pre-commit fix * resolve review * adjust stop reason * add unittest for _create_chat_completion_choice * modify unittest * solve confict * solve conflict * resolve conflict --------- Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: gaoziyuan <m13689897706@163.com>	2025-10-17 20:51:59 +08:00
LiqinruiG	4251ac5e95	【Fix】 remove text_after_process & raw_prediction (#4421) ... * remove text_after_process & raw_prediction * remove text_after_process & raw_prediction	2025-10-16 19:00:18 +08:00
SunLei	b4b579a7ed	Feature：Add support for Pooling Model Embedding and provide an OpenAI-compatible API. (#4344) ... * feat: add OpenAIServing * feat: add ZmqOpenAIServing & OpenAIServingEmbedding * feat: Refine the basic ServingEngine class and introduce ServingContext * fix: codestyle * fix: request * fix: pooling_params * feat: _process_chat_template_kwargs * feat: support batch request * feat: pooling_params verify & default parameters --------- Co-authored-by: sunlei1024 <sunlei1024@example.com>	2025-10-15 19:42:59 +08:00
qwes5s5	abb62624b8	[fix] Fixed the issue of excessive/redundant spans being returned for streaming requests. (#4375) ... * fix stream span * fix stream span	2025-10-15 11:47:47 +08:00
ltd0924	28d1b6cd97	[BugFix] fix multinode bugs (#4377) ... * [BugFix] fix multinode bugs * Update test_config.py * Update test_config.py * Update test_config.py --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-10-15 11:43:39 +08:00
xiaolei373	55124f8491	Add cli run batch (#4237) ... * feat(log):add_request_and_response_log * [cli] add run batch cli --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-26 14:27:25 +08:00
zhuzixuan	12a3587cca	[Supplements and upgrades]Improvement of X1 parsers (#4172) ... * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser	2025-09-26 13:37:37 +08:00
ltd0924	83720da79f	[Feature] support clear data (#3601) ... * [Feature] support clear data * update * fix * fix * fix * fix * fix * fix * fix	2025-09-23 10:20:02 +08:00
co63oc	17a27170bc	fix typos (#4093)	2025-09-15 18:33:30 +08:00
xiaolei373	9ac539471d	[format] Valid para format error info (#4035) ... * feat(log):add_request_and_response_log * 报错信息与OpenAI对齐	2025-09-12 19:05:17 +08:00
ltd0924	cab7a633fe	[CI] add multi api server test (#4049) ... * [BugFix] fix max streaming tokens invalid * fix scheduler bug * fix scheduler bug * Update multi_api_server.py * Create test_multi_api_server.py * fix	2025-09-12 11:18:38 +08:00
zhuzixuan	a47976e82d	[Echo] Support more types of prompt echo (#4022) ... * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. * wenxin-tools-700 When the prompt type is list[int] or list[list[int]], it needs to support echoing after decoding. --------- Co-authored-by: luukunn <83932082+luukunn@users.noreply.github.com>	2025-09-11 19:34:44 +08:00
zhuzixuan	83bd55100b	[Optimize]Error messages about Model api. (#3839) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/openai/serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * 优化报错信息。 --------- Co-authored-by: yangzichao01 <yangzichao01@baidu.com> Co-authored-by: Yzc216 <101054010+Yzc216@users.noreply.github.com> Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-09-08 15:52:26 +08:00
RichardWooSJTU	f36a388ffe	fix response processsors (#3826) ... * fix response processsors * fix ci * fix ut --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-04 16:01:25 +08:00
ltd0924	bf0cf5167a	[BugFix] fix max streaming tokens invalid (#3789)	2025-09-02 13:57:32 +08:00
SunLei	b9af95cf1c	[Feature] Add AsyncTokenizerClient&ChatResponseProcessor with remote encode&decode support. (#3674) ... * [Feature] add AsyncTokenizerClient * add decode_image * Add response_processors with remote decode support. * [Feature] add tokenizer_base_url startup argument * Revert comment removal and restore original content. * [Feature] Non-streaming requests now support remote image decoding. * Fix parameter type issue in decode_image call. * Keep completion_token_ids when return_token_ids = False. * add copyright	2025-08-30 17:06:26 +08:00
Jiang-Jia-Jun	93e1b63200	Revert "[UnitTest][Copilot] Improve unit test coverage for entrypoints module…" (#3564) ... This reverts commit 36325e9ea7.	2025-08-23 10:44:23 +08:00
Copilot	36325e9ea7	[UnitTest][Copilot] Improve unit test coverage for entrypoints modules (#3546) ... * Initial plan * Add comprehensive unit tests for entrypoints utilities Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Complete entrypoints test coverage improvement with tool parser tests Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Apply pre-commit formatting to test files - fix trailing whitespace and long lines --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-22 19:20:51 +08:00
Yzc216	466cbb5a99	[Feature] Models api (#3073) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/openai/serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-21 17:02:56 +08:00
ltd0924	51f68ae593	[Feature] add dealer manager to reuse the connection (#3471) ... * [BugFix] fix control signal release failed * [BugFix] fix control signal release failed * update * update * update * [Feature] add dealer manager to reuse the connection * fix * fix * fix * fix * fix * fix * Create test_dealer_connection_manager.py * Delete test/entrypoints/openai directory * Update test_dealer_connection_manager.py * Update test_dealer_connection_manager.py	2025-08-21 13:11:13 +08:00