3609 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
kevin	8aab4e367f	[Feature] mm support prefix cache (#4134) ... * support mm prefix caching * update code * fix mm_hashes * support encoder cache * add encoder cache * update code * update encoder cache * fix features bug * fix worker bug * support processor cache, need to optimize yet * refactor multimodal data cache * update code * update code * update v1 scheduler * update code * update code * update codestyle * support turn off processor cache and encoder cache * update pre-commit * fix code * solve review * update code * update code * update test case * set processor cache in GiB * update test case * support mm prefix caching for qwen model * fix code style check * update pre-commit * fix unit test * fix unit test * add ci test case * fix rescheduled bug * change text_after_process to prompt_tokens * fix unit test * fix chat template * change model path * [EP] fix adapter bugs (#4572) * Update expert_service.py * Update common_engine.py * Update expert_service.py * fix v1 hang bug (#4573) * fix import image_ops error on some platforms (#4559) * [CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) * add collect-env * del files * [Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) * add new branch for sot * reorder * fix batch bug * [XPU]Moe uses a new operator (#4585) * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response * [Feature] Support Paddle-OCR (#4396) * init * update code * fix code style & disable thinking * adapt for common_engine.update_mm_requests_chunk_size * use 3d rope * use flash_attn_unpadded * opt siglip * update to be compatible with the latest codebase * fix typo * optim OCR performance * fix bug * fix bug * fix bug * fix bug * normlize name * modify xpu rope * revert logger * fix bug * fix bug * fix bug * support default_v1 * optim performance * fix bug --------- Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> * [DataProcessor] add reasoning_tokens into usage info (#4520) * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test * perf: Optimize task queue communication from engine to worker (#4531) * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Clean up ports after processing results (#4587) * [CI] Add /re-run command in PR comments to restart failed CI workflows (#4593) * [Others] api server exits when worker process is dead (#3271) * [fix] fix terminal hangs when worker process is dead * [chore] change sleep time of monitor * [chore] remove redundant comments * update docs --------- Co-authored-by: ApplEOFDiscord <wwy640130@163.com> Co-authored-by: ApplEOFDiscord <31272106+ApplEOFDiscord@users.noreply.github.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com> Co-authored-by: yinwei <yinwei_hust@163.com> Co-authored-by: JYChen <zoooo0820@qq.com> Co-authored-by: qwes5s5 <45442318+qwes5s5@users.noreply.github.com> Co-authored-by: Ryan <zihaohuang@aliyun.com> Co-authored-by: yyssys <atyangshuang@foxmail.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com> Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> Co-authored-by: kxz2002 <115912648+kxz2002@users.noreply.github.com> Co-authored-by: SunLei <sunlei5788@gmail.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: Zhang Yulong <35552275+ZhangYulongg@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: 李泳桦 <39643373+liyonghua0910@users.noreply.github.com>	2025-10-27 17:39:51 +08:00
YuBaoku	a4fb3d4ff0	[CI] Fix path error of /re-run (#4606) ... * [CI] Add /re-run command in PR comments to restart failed CI workflows * [CI] Fix /re-run command	2025-10-27 17:03:11 +08:00
chen	5c63a089f6	[Feature] Support logprobs_mode (#4567)	2025-10-27 14:27:48 +08:00
CSWYF3634076	acd331780c	[V1 loader] Qwen25 VL support v1 loader and torch style safetensors load (#4388) ... * [BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix * [Docs]offine infer add apply_chat_template add_generation_prompt parameter * [Model]qwen2.5VL support --use-cudagraph * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v2 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v3 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v4 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v5 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v6 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v7 * qwen25vl v1 loader * qwen25vl v1 loader v2 * qwen25vl v1 loader v3 * qwen25vl v1 loader fix tp2 weight PySafeSlice * qwen25vl v1 loader no test * qwen25vl v1 loader add unit test * qwen25vl v1 loader add unit test v2 * qwen25vl v1 loader add torch unit test v3 * qwen25vl v1 loader add torch unit test v4 * qwen25vl v1 loader add torch unit test v5 * qwen25vl v1 loader add torch unit test v6	2025-10-27 10:54:15 +08:00
Lucas	5c6105f4a2	[XPU] bind some OPs for VL model with pybind (#4522)	2025-10-27 10:50:08 +08:00
李泳桦	cdc40cdc2a	[Others] api server exits when worker process is dead (#3271) ... * [fix] fix terminal hangs when worker process is dead * [chore] change sleep time of monitor * [chore] remove redundant comments	2025-10-27 10:23:48 +08:00
YuBaoku	ebae69b1f8	[CI] Add /re-run command in PR comments to restart failed CI workflows (#4593)	2025-10-27 10:18:56 +08:00
Zhang Yulong	83b720804b	Clean up ports after processing results (#4587)	2025-10-27 10:13:24 +08:00
SunLei	dc1a9c7287	perf: Optimize task queue communication from engine to worker (#4531) ... * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-25 22:45:38 +08:00
kxz2002	327fa4c255	[DataProcessor] add reasoning_tokens into usage info (#4520) ... * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test	2025-10-25 16:57:58 +08:00
ming1753	e4e3cede7f	[Feature] Support Paddle-OCR (#4396) ... * init * update code * fix code style & disable thinking * adapt for common_engine.update_mm_requests_chunk_size * use 3d rope * use flash_attn_unpadded * opt siglip * update to be compatible with the latest codebase * fix typo * optim OCR performance * fix bug * fix bug * fix bug * fix bug * normlize name * modify xpu rope * revert logger * fix bug * fix bug * fix bug * support default_v1 * optim performance * fix bug --------- Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com>	2025-10-24 23:34:30 +08:00
yyssys	822dea8d5f	[XPU]Moe uses a new operator (#4585) ... * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response	2025-10-24 23:01:46 +08:00
Ryan	f42ed6d5f2	[Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) ... * add new branch for sot * reorder * fix batch bug	2025-10-24 18:36:52 +08:00
qwes5s5	e02a812880	[CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) ... * add collect-env * del files	2025-10-24 16:46:45 +08:00
JYChen	83d45af1f3	fix import image_ops error on some platforms (#4559)	2025-10-24 16:09:20 +08:00
yinwei	5fbc653238	fix v1 hang bug (#4573)	2025-10-24 15:35:10 +08:00
ltd0924	b60ce4922b	[EP] fix adapter bugs (#4572) ... * Update expert_service.py * Update common_engine.py * Update expert_service.py	2025-10-24 12:30:08 +08:00
李泳桦	8edc5cca91	[BugFix] fix create_cache_tensor for ep (#4542) ... * [fix] fix create_cache_tensor for ep * [fix] fix again	2025-10-24 11:31:13 +08:00
xiaozude	f7069b8057	[Metax] adapt DeepSeek (#4498)	2025-10-24 10:14:53 +08:00
Sunny-bot1	8718fa34b2	support static C8 (#4568)	2025-10-23 22:01:03 +08:00
RAM	e36343d807	[FDConfig]Turn on the CUDAGraph + PD Disaggregation switch (#4530)	2025-10-23 21:05:14 +08:00
RAM	9dc5c3e370	[Graph Optimization] Support CUDAGraph Padding + MTP (#4545) ... * Support CUDAGraph Padding + MTP * support orther write cache kernel	2025-10-23 20:57:26 +08:00
RichardWooSJTU	5a8c60454e	[BugFix] Fix decode_type which has been deleted in req and optimize token client retry scheme (#4564)	2025-10-23 05:08:10 -07:00
zhupengyang	3a43dbf82d	[XPU] merge apply_tp, ops support token_num = 0 (#4507)	2025-10-23 19:09:58 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
YuBaoku	a240425db9	[CI] Optimize coverage upload reporting (#4547) ... * [CI] Optimize coverage upload reporting * [CI] fix upload reporting * [CI] fix code style	2025-10-23 17:01:48 +08:00
ddchenhao66	5443b2cffb	[XPU] xpu support think length limit (#4539) ... * [XPU] xpu support think length limit * [XPU] xpu c++ code files format --------- Co-authored-by: ddchenhao66 <dhaochen163.com>	2025-10-23 15:58:11 +08:00
tianlef	2676a918f0	[Doc]fix deepseek ce (#4560)	2025-10-23 14:09:11 +08:00
luukunn	bbf06b9ff7	[BugFix]Fix finish reason (#4543) ... * fix finish reason * add unit test * add unit test * fix unie test * fix unit test	2025-10-23 14:04:43 +08:00
YuanRisheng	ac4f5ca272	delete useless code (#4544) ... Co-authored-by: root <root@yqlcc01-sys-rpm12rzmwjd.yqlcc01.baidu.com>	2025-10-23 13:40:34 +08:00
RAM	8a02ab43a8	[FDConfig]Turn on the CUDAGraph + RL switch (#4508) ... * Turn on the CUDAGraph + RL switch * reduce max_num_seqs and number of request	2025-10-23 11:08:07 +08:00
plusNew001	918e4e9850	[XPU] Change XPU stable third-party version (#4524) ... * add xpu ci case * Add xDeepEP download and build steps Download and build xDeepEP before running tests. * Fix formatting and add missing sleep command * Update Docker image version in CI workflow * Modify run_ci_xpu.sh for log cleanup and error handling Clean up log files before running tests and output worker log on failure. * Enhance test_ep.py with process management and assertions Refactor test function to include process cleanup and assertions. * Replace test_fastdeploy_llm with test_fd_ep * Fix conditional statement in run_ci_xpu.sh * Update test_ep.py for string handling and formatting Fix string encoding issues and improve readability. * Rename test_ep.py to run_ep.py * Change test script from test_ep.py to run_ep.py * Update dependency versions for stable release * Install pytest-timeout and modify test execution Added pytest-timeout installation and updated test command.	2025-10-22 19:43:03 +08:00
zhupengyang	3a6883ac1a	c++ code format (#4527)	2025-10-22 17:59:50 +08:00
周周周	d7bcedf421	small change in test_fusedmoe.py (#4538)	2025-10-22 17:49:18 +08:00
Yuanle Liu	8e02a509c3	[CI] stable test_rollout_model.py (#4536) ... * stable test_rollout_model.py * update baseline * update baseline	2025-10-22 01:59:44 -07:00
zhouchong	dce988824d	[Feature] Support AsyncLLM (#4458) ... * add async_llm * apply review * update engine config * Adapt to latest engine.py changes * add more unit tests * Increase unit test coverage --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-22 15:50:12 +08:00
guozhuangzhuang	b6cd3aec70	[Feature] support fd return decode response (#4407) ... * [Feature] support fd return decode response * Resolving conflicts * fix * fix * fix * fix * fix --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-22 14:22:08 +08:00
yyssys	cd9195d54c	[XPU]Modify the xpu memory display unit of log (#4534)	2025-10-22 12:46:01 +08:00
YuBaoku	f69c9cd122	[CI] Remove redundant .coveragerc file (#4521)	2025-10-21 23:24:05 +08:00
Yuanle Liu	3b58310c26	enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525) ... * fix custom_ops * paddleformers>=0.3.1	2025-10-21 22:06:06 +08:00
yzwu	dc7facaa7f	[Iluvatar GPU] fix ci error caused by rebuild_padding param and cuda graph (#4504)	2025-10-21 21:41:41 +08:00
RAM	d70aacfbdc	[FDConfig] Turn on the CUDAGraph + MultiModel switch (#4512)	2025-10-21 06:21:26 -07:00
SunLei	809c1ac7ec	feat: add post-processing step for pool_output (#4462) ... * feat: add post-processing step for pool_output * bugfix * fix: test_serving_embedding * fix test_request_to_batch_dicts * fix: code style	2025-10-21 20:24:26 +08:00
plusNew001	2bd3fb6315	[XPU]add xpu ci ep case (#4432) ... * add xpu ci case * Add xDeepEP download and build steps Download and build xDeepEP before running tests. * Fix formatting and add missing sleep command * Update Docker image version in CI workflow * Modify run_ci_xpu.sh for log cleanup and error handling Clean up log files before running tests and output worker log on failure. * Enhance test_ep.py with process management and assertions Refactor test function to include process cleanup and assertions. * Replace test_fastdeploy_llm with test_fd_ep * Fix conditional statement in run_ci_xpu.sh * Update test_ep.py for string handling and formatting Fix string encoding issues and improve readability. * Rename test_ep.py to run_ep.py * Change test script from test_ep.py to run_ep.py	2025-10-21 19:19:40 +08:00
Copilot	175391389f	Add comprehensive unit tests for limit_thinking_content_length operators (#4510) ... * Initial plan * Add comprehensive unit tests for limit_thinking_content_length functions Co-authored-by: yuanlehome <23653004+yuanlehome@users.noreply.github.com> * fix (#4514) --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: yuanlehome <23653004+yuanlehome@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-10-21 18:55:57 +08:00
RAM	7cbe6b2472	[FDConfig] Turn on the CUDAGraph + Speculative Decoding switch (#4511)	2025-10-21 03:34:16 -07:00
tianlef	153f15db39	[Doc]add deepseek wint4 ce (#4517)	2025-10-21 16:41:51 +08:00
ltd0924	fb76cdfb4f	[Fearture] Support mm model close prefix cache (#4459) ... * [Feature] support prefix cache in DP * fix * Update common_engine.py * Update common_engine.py * Update common_engine.py * Update common_engine.py * [BugFix] fix workers more than 1 * fix * Update api_server.py * fix * Update api_server.py * fix * [Fearture] Support mm model close prefix cache * Update api_server.py * Update engine_client.py * Update engine_client.py * add test * Update test_chat.py * fix * fix * Update test_chat.py * Update test_chat.py --------- Co-authored-by: ltd0924 <luotingdan@baidu.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 15:37:59 +08:00
Divano	2b53c4d684	【CI】Add test cases for n parameter and streaming validation (#4503) ... * add repitation early stop cases * add repitation early stop cases * add structure test openai * add n parameters case	2025-10-21 15:33:29 +08:00
SunLei	ee915220bd	[Speculative Decoding] Add draft_logprobs Support for Speculative Decode MTP (#4467) ... * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix: postprocess for speculative decode * test: test_speculative_decoding_use_logprobs * fix: test_completion_echo * fix test_max_streaming_tokens --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 14:57:50 +08:00