342 Commits

Author	SHA1	Message	Date
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
Ryan	28de91b50f	[Graph Optimization] SOT+CUDAGraph support ERNIE4.5T VL 28B / 424B (#4645) ... * 45TVL support sot+CUDAGraph * mv unitest from ce_deploy 2 e2e * add test_EB_VL_Lite_sot_serving * rm useless line * add openai_client * fix unitest && reduce computing resources	2025-10-31 11:38:43 +08:00
Longzhi Wang	b61a272385	[BugFix] fix unittest of get_save_output_v1 (#4701) ... * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1	2025-10-31 11:23:49 +08:00
kxz2002	a2870ed4a9	[Feature] Unify the registration name recognition for tool_parser and reasoning_parser to “-” (#4668) ... * parser register name unify * change ernie_x1 to ernie-x1 * change ernie4_5_vl to ernie-45-vl * fix unit test	2025-10-31 10:45:27 +08:00
kxz2002	82bd7e5db4	[BugFix] Fix finish reason in _create_chat_completion_choice (#4582) ... * fix n_param _create_chat_completion_choicel * fix unit test * fix final_res * modify unit tests	2025-10-31 10:42:19 +08:00
plusNew001	ea866e4b34	[XPU] [CI] Add Vl case (#4649) ... * Enhance CI script with health checks and logging Updated the CI script to include health checks and logging for the VL model testing process. * Add test for OpenAI chat completions * Refactor chat completion user message structure * Fix variable name for exit code in CI script * Update text prompt to Chinese for artifact question * Update service port and response assertions in tests * Refactor assertion for response content comparison * Update run_45vl.py * Change service HTTP port from 8123 to 8188	2025-10-31 10:38:09 +08:00
zhouchong	35286ce31a	fix total_block_num init error in worker_process (#4687)	2025-10-30 19:53:09 +08:00
周周周	8b9c9463cd	add real gate_correction_bias weight to mock un-balanced dispatch (#4676)	2025-10-30 15:13:21 +08:00
Haonan Luo	d7d0112bbf	[CI] Add test for paddleocr_vl (#4627)	2025-10-30 13:40:04 +08:00
RAM	cd3b7cc392	[Graph Optimization] Add the CUDAGraph usage switch for Draft Model (#4601) ... * add draft model using cudagraph switch * set default as false * capture draft model in ci * fix bug	2025-10-30 11:44:50 +08:00
GoldPancake	fddda50cb9	Add ut for speculative sampler (#4650)	2025-10-30 10:37:49 +08:00
周周周	dab04ab413	add noaux_tc to unitest fused_moe (#4656)	2025-10-29 21:50:25 +08:00
kxz2002	c30bfb294f	[Feature] add a new reasoning parser (#4571) ... * add new reasoning_parser initial commit * add parser file content * add register * ernie_test_reasoning_parser * support <tool_call> token and add tool_parser * add and fix unit tests * modify reasoning_parser * modify reasoning parser and tool parser * modify unit tests * modify reasoning_parser and tool_parser * modify unit tests * fix tool_parser * modify the logic of reasoning_parser and tool_parser * add and modify unit tests * standardize code style * simplify reasoning_parser and tool_parser * modify unit test	2025-10-29 18:16:50 +08:00
ApplEOFDiscord	14f8cddaf1	[Feature] add mm token usage (#4570) ... * add mm token usage * fix unit test * fix unit test * fix unit test * fix model path * fix unit test * fix unit test * fix unit test * remove uncomment * change var name * fix code style * fix code style * fix code style * fix code style * fix unit test	2025-10-29 14:37:12 +08:00
RAM	fc5cd1adb1	[BugFix] Fix graph opt test case (#4634) ... * fix bug and refine code * add debug count * refine code * fix ci test case	2025-10-29 13:28:04 +08:00
xiaolei373	14e7d88ea4	[feature] support reward api (#4518) ... Co-authored-by: SunLei <sunlei5788@gmail.com>	2025-10-29 00:20:28 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
SunLei	2a9ed72533	feat: add support for API usage with multimodal models (#4548) ... * feat: add support for API usage with multimodal models * completion_tokens contains num_image_tokens * remove test_request.py * fix: paddle.device.is_compiled_with_cuda() * fix test_unstream_without_logprobs	2025-10-28 20:23:46 +08:00
YuBaoku	e1ac90d787	[CI] Revert test_rollout_model directory change (#4626)	2025-10-28 20:14:00 +08:00
zhouchong	567f61072c	[CI][BugFix] fix port conflicts in concurrent ci test and add more unit test on async_llm (#4616) ... * fix:port conflicts in concurrent ci test * add more unit test on async_llm	2025-10-28 19:04:24 +08:00
yyssys	cd6d1f633c	[XPU]add xpu ci w4a8 case (#4501)	2025-10-28 19:02:29 +08:00
Zhang Yulong	b4014834a9	Extend sleep time to 10 seconds in switch_service (#4618) ... Increase sleep duration before switching services.	2025-10-28 15:19:21 +08:00
YuBaoku	b2c6c41447	[CI] Relocate server test cases from ci_use directory to e2e (#4608)	2025-10-28 11:37:30 +08:00
xiaolei373	31180a6a13	fix_run_batch_unittest (#4613)	2025-10-28 10:38:06 +08:00
ming1753	7681375a19	[BugFix] PaddleOCR-VL fix FD_DEBUG type and support v1 loader (#4605) ... * [Bug Fix] PaddleOCRVL fix FD_DEBUG type and support HF model * fix bug * fix bug * fix bug	2025-10-28 09:47:47 +08:00
K11OntheBoat	64d1aa973b	[Unitest]Add unitest of Attention Layer (#4494)	2025-10-27 19:18:50 +08:00
kevin	8aab4e367f	[Feature] mm support prefix cache (#4134) ... * support mm prefix caching * update code * fix mm_hashes * support encoder cache * add encoder cache * update code * update encoder cache * fix features bug * fix worker bug * support processor cache, need to optimize yet * refactor multimodal data cache * update code * update code * update v1 scheduler * update code * update code * update codestyle * support turn off processor cache and encoder cache * update pre-commit * fix code * solve review * update code * update code * update test case * set processor cache in GiB * update test case * support mm prefix caching for qwen model * fix code style check * update pre-commit * fix unit test * fix unit test * add ci test case * fix rescheduled bug * change text_after_process to prompt_tokens * fix unit test * fix chat template * change model path * [EP] fix adapter bugs (#4572) * Update expert_service.py * Update common_engine.py * Update expert_service.py * fix v1 hang bug (#4573) * fix import image_ops error on some platforms (#4559) * [CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) * add collect-env * del files * [Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) * add new branch for sot * reorder * fix batch bug * [XPU]Moe uses a new operator (#4585) * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response * [Feature] Support Paddle-OCR (#4396) * init * update code * fix code style & disable thinking * adapt for common_engine.update_mm_requests_chunk_size * use 3d rope * use flash_attn_unpadded * opt siglip * update to be compatible with the latest codebase * fix typo * optim OCR performance * fix bug * fix bug * fix bug * fix bug * normlize name * modify xpu rope * revert logger * fix bug * fix bug * fix bug * support default_v1 * optim performance * fix bug --------- Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> * [DataProcessor] add reasoning_tokens into usage info (#4520) * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test * perf: Optimize task queue communication from engine to worker (#4531) * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Clean up ports after processing results (#4587) * [CI] Add /re-run command in PR comments to restart failed CI workflows (#4593) * [Others] api server exits when worker process is dead (#3271) * [fix] fix terminal hangs when worker process is dead * [chore] change sleep time of monitor * [chore] remove redundant comments * update docs --------- Co-authored-by: ApplEOFDiscord <wwy640130@163.com> Co-authored-by: ApplEOFDiscord <31272106+ApplEOFDiscord@users.noreply.github.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com> Co-authored-by: yinwei <yinwei_hust@163.com> Co-authored-by: JYChen <zoooo0820@qq.com> Co-authored-by: qwes5s5 <45442318+qwes5s5@users.noreply.github.com> Co-authored-by: Ryan <zihaohuang@aliyun.com> Co-authored-by: yyssys <atyangshuang@foxmail.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com> Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> Co-authored-by: kxz2002 <115912648+kxz2002@users.noreply.github.com> Co-authored-by: SunLei <sunlei5788@gmail.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: Zhang Yulong <35552275+ZhangYulongg@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: 李泳桦 <39643373+liyonghua0910@users.noreply.github.com>	2025-10-27 17:39:51 +08:00
chen	5c63a089f6	[Feature] Support logprobs_mode (#4567)	2025-10-27 14:27:48 +08:00
CSWYF3634076	acd331780c	[V1 loader] Qwen25 VL support v1 loader and torch style safetensors load (#4388) ... * [BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix * [Docs]offine infer add apply_chat_template add_generation_prompt parameter * [Model]qwen2.5VL support --use-cudagraph * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v2 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v3 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v4 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v5 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v6 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v7 * qwen25vl v1 loader * qwen25vl v1 loader v2 * qwen25vl v1 loader v3 * qwen25vl v1 loader fix tp2 weight PySafeSlice * qwen25vl v1 loader no test * qwen25vl v1 loader add unit test * qwen25vl v1 loader add unit test v2 * qwen25vl v1 loader add torch unit test v3 * qwen25vl v1 loader add torch unit test v4 * qwen25vl v1 loader add torch unit test v5 * qwen25vl v1 loader add torch unit test v6	2025-10-27 10:54:15 +08:00
Zhang Yulong	83b720804b	Clean up ports after processing results (#4587)	2025-10-27 10:13:24 +08:00
SunLei	dc1a9c7287	perf: Optimize task queue communication from engine to worker (#4531) ... * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-25 22:45:38 +08:00
kxz2002	327fa4c255	[DataProcessor] add reasoning_tokens into usage info (#4520) ... * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test	2025-10-25 16:57:58 +08:00
yyssys	822dea8d5f	[XPU]Moe uses a new operator (#4585) ... * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response	2025-10-24 23:01:46 +08:00
Ryan	f42ed6d5f2	[Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) ... * add new branch for sot * reorder * fix batch bug	2025-10-24 18:36:52 +08:00
qwes5s5	e02a812880	[CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) ... * add collect-env * del files	2025-10-24 16:46:45 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
luukunn	bbf06b9ff7	[BugFix]Fix finish reason (#4543) ... * fix finish reason * add unit test * add unit test * fix unie test * fix unit test	2025-10-23 14:04:43 +08:00
RAM	8a02ab43a8	[FDConfig]Turn on the CUDAGraph + RL switch (#4508) ... * Turn on the CUDAGraph + RL switch * reduce max_num_seqs and number of request	2025-10-23 11:08:07 +08:00
周周周	d7bcedf421	small change in test_fusedmoe.py (#4538)	2025-10-22 17:49:18 +08:00
Yuanle Liu	8e02a509c3	[CI] stable test_rollout_model.py (#4536) ... * stable test_rollout_model.py * update baseline * update baseline	2025-10-22 01:59:44 -07:00
zhouchong	dce988824d	[Feature] Support AsyncLLM (#4458) ... * add async_llm * apply review * update engine config * Adapt to latest engine.py changes * add more unit tests * Increase unit test coverage --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-22 15:50:12 +08:00
yzwu	dc7facaa7f	[Iluvatar GPU] fix ci error caused by rebuild_padding param and cuda graph (#4504)	2025-10-21 21:41:41 +08:00
SunLei	809c1ac7ec	feat: add post-processing step for pool_output (#4462) ... * feat: add post-processing step for pool_output * bugfix * fix: test_serving_embedding * fix test_request_to_batch_dicts * fix: code style	2025-10-21 20:24:26 +08:00
plusNew001	2bd3fb6315	[XPU]add xpu ci ep case (#4432) ... * add xpu ci case * Add xDeepEP download and build steps Download and build xDeepEP before running tests. * Fix formatting and add missing sleep command * Update Docker image version in CI workflow * Modify run_ci_xpu.sh for log cleanup and error handling Clean up log files before running tests and output worker log on failure. * Enhance test_ep.py with process management and assertions Refactor test function to include process cleanup and assertions. * Replace test_fastdeploy_llm with test_fd_ep * Fix conditional statement in run_ci_xpu.sh * Update test_ep.py for string handling and formatting Fix string encoding issues and improve readability. * Rename test_ep.py to run_ep.py * Change test script from test_ep.py to run_ep.py	2025-10-21 19:19:40 +08:00
Copilot	175391389f	Add comprehensive unit tests for limit_thinking_content_length operators (#4510) ... * Initial plan * Add comprehensive unit tests for limit_thinking_content_length functions Co-authored-by: yuanlehome <23653004+yuanlehome@users.noreply.github.com> * fix (#4514) --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: yuanlehome <23653004+yuanlehome@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-10-21 18:55:57 +08:00
ltd0924	fb76cdfb4f	[Fearture] Support mm model close prefix cache (#4459) ... * [Feature] support prefix cache in DP * fix * Update common_engine.py * Update common_engine.py * Update common_engine.py * Update common_engine.py * [BugFix] fix workers more than 1 * fix * Update api_server.py * fix * Update api_server.py * fix * [Fearture] Support mm model close prefix cache * Update api_server.py * Update engine_client.py * Update engine_client.py * add test * Update test_chat.py * fix * fix * Update test_chat.py * Update test_chat.py --------- Co-authored-by: ltd0924 <luotingdan@baidu.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 15:37:59 +08:00
Divano	2b53c4d684	【CI】Add test cases for n parameter and streaming validation (#4503) ... * add repitation early stop cases * add repitation early stop cases * add structure test openai * add n parameters case	2025-10-21 15:33:29 +08:00
SunLei	ee915220bd	[Speculative Decoding] Add draft_logprobs Support for Speculative Decode MTP (#4467) ... * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix: postprocess for speculative decode * test: test_speculative_decoding_use_logprobs * fix: test_completion_echo * fix test_max_streaming_tokens --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 14:57:50 +08:00
RAM	775edcc09a	[Executor] Default use CUDAGraph (#3594) ... * add start intercept * Adjustment GraphOptConfig * pre-commit * default use cudagraph * set default value * default use cuda graph * pre-commit * fix test case bug * disable rl * fix moba attention * only support gpu * Temporarily disable PD Disaggregation * set max_num_seqs of test case as 1 * set max_num_seqs and temperature * fix max_num_batched_tokens bug * close cuda graph * success run wint2 * profile run with max_num_batched_tokens * 1.add c++ memchecker 2.success run wint2 * updatee a800 yaml * update docs * 1. delete check 2. fix plas attn test case * default use use_unique_memory_pool * add try-except for warmup * ban mtp, mm, rl * fix test case mock * fix ci bug * fix form_model_get_output_topp0 bug * fix ci bug * refine deepseek ci * refine code * Disable PD * fix sot yaml	2025-10-21 14:25:45 +08:00
YuBaoku	70a29ec49e	[CI] update ernie-4_5-vl baseline (#4495) ... * [CI] update ernie-4_5-vl baseline * [CI] update Qwen2.5-VL-7B-Instruct baseline	2025-10-21 10:18:29 +08:00