352 Commits

Author	SHA1	Message	Date
bukejiyu	69c2f3cda1	[CI]test common model (#4697) ... * ut * update	2025-11-03 16:48:36 +08:00
luukunn	7b35488779	【DataProcessor】add options thinking_mode (#4735) ... * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add unit test	2025-11-03 14:30:07 +08:00
yinwei	377f3bf5f2	[XPU] add v1 support for bf16 (#4744) ... * support v1 loader * update code style * update code	2025-11-03 14:13:17 +08:00
freeliuzc	11398790d3	[Speculative Decoding][MTP]Support attn mask offset (#4641) ... * [MTP]Merge support attn (#4591) * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import * support attn_mask_offset in mtp mode * add update_attn_mask op * fix unit test && fix code-style	2025-11-03 10:08:01 +08:00
YuBaoku	9eff788658	[CI] fix some ci yaml (#4747)	2025-11-02 21:28:04 +08:00
lizexu123	4ac6de9a3c	[Feature] support pooling model runner (#4590) ... * support qwen3-embedding * support qwen3-embedding-0.6b * fix * fix bug * fix test_return_token_ids.py and update enable_thinking * fix mtp dummy_run * merge develop * fix np.float32 * delete FD_DISABLE_CHUNKED_PREFILL and FD_USE_GET_SAVE_OUTPUT_V1 * delete and build_stream_transfer_data * fix test_update_v1: * fix * fix * update dummy_run post_process * delete test_update_v1 * fix * fix dummy_run * fix model_path * fix model_path * fix dummy_run	2025-10-31 22:32:05 +08:00
YuBaoku	acef624049	[CI] Fix rollout_model test logic (#4730)	2025-10-31 22:25:24 +08:00
Yuanle Liu	b301bd6c31	[BugFix] fix thinking bug (#4710) ... * fix thinking bug * fix ut * update * fix	2025-10-31 22:00:31 +08:00
xiaolei373	dde7ba3f9e	[CI]add_tokenizer_cli_unitest (#4620)	2025-10-31 13:57:51 +08:00
周周周	10de7a3b82	add flops and bandwidth to test_ffn.py (#4704)	2025-10-31 12:13:59 +08:00
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
Ryan	28de91b50f	[Graph Optimization] SOT+CUDAGraph support ERNIE4.5T VL 28B / 424B (#4645) ... * 45TVL support sot+CUDAGraph * mv unitest from ce_deploy 2 e2e * add test_EB_VL_Lite_sot_serving * rm useless line * add openai_client * fix unitest && reduce computing resources	2025-10-31 11:38:43 +08:00
Longzhi Wang	b61a272385	[BugFix] fix unittest of get_save_output_v1 (#4701) ... * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1	2025-10-31 11:23:49 +08:00
kxz2002	a2870ed4a9	[Feature] Unify the registration name recognition for tool_parser and reasoning_parser to “-” (#4668) ... * parser register name unify * change ernie_x1 to ernie-x1 * change ernie4_5_vl to ernie-45-vl * fix unit test	2025-10-31 10:45:27 +08:00
kxz2002	82bd7e5db4	[BugFix] Fix finish reason in _create_chat_completion_choice (#4582) ... * fix n_param _create_chat_completion_choicel * fix unit test * fix final_res * modify unit tests	2025-10-31 10:42:19 +08:00
plusNew001	ea866e4b34	[XPU] [CI] Add Vl case (#4649) ... * Enhance CI script with health checks and logging Updated the CI script to include health checks and logging for the VL model testing process. * Add test for OpenAI chat completions * Refactor chat completion user message structure * Fix variable name for exit code in CI script * Update text prompt to Chinese for artifact question * Update service port and response assertions in tests * Refactor assertion for response content comparison * Update run_45vl.py * Change service HTTP port from 8123 to 8188	2025-10-31 10:38:09 +08:00
zhouchong	35286ce31a	fix total_block_num init error in worker_process (#4687)	2025-10-30 19:53:09 +08:00
周周周	8b9c9463cd	add real gate_correction_bias weight to mock un-balanced dispatch (#4676)	2025-10-30 15:13:21 +08:00
Haonan Luo	d7d0112bbf	[CI] Add test for paddleocr_vl (#4627)	2025-10-30 13:40:04 +08:00
RAM	cd3b7cc392	[Graph Optimization] Add the CUDAGraph usage switch for Draft Model (#4601) ... * add draft model using cudagraph switch * set default as false * capture draft model in ci * fix bug	2025-10-30 11:44:50 +08:00
GoldPancake	fddda50cb9	Add ut for speculative sampler (#4650)	2025-10-30 10:37:49 +08:00
周周周	dab04ab413	add noaux_tc to unitest fused_moe (#4656)	2025-10-29 21:50:25 +08:00
kxz2002	c30bfb294f	[Feature] add a new reasoning parser (#4571) ... * add new reasoning_parser initial commit * add parser file content * add register * ernie_test_reasoning_parser * support <tool_call> token and add tool_parser * add and fix unit tests * modify reasoning_parser * modify reasoning parser and tool parser * modify unit tests * modify reasoning_parser and tool_parser * modify unit tests * fix tool_parser * modify the logic of reasoning_parser and tool_parser * add and modify unit tests * standardize code style * simplify reasoning_parser and tool_parser * modify unit test	2025-10-29 18:16:50 +08:00
ApplEOFDiscord	14f8cddaf1	[Feature] add mm token usage (#4570) ... * add mm token usage * fix unit test * fix unit test * fix unit test * fix model path * fix unit test * fix unit test * fix unit test * remove uncomment * change var name * fix code style * fix code style * fix code style * fix code style * fix unit test	2025-10-29 14:37:12 +08:00
RAM	fc5cd1adb1	[BugFix] Fix graph opt test case (#4634) ... * fix bug and refine code * add debug count * refine code * fix ci test case	2025-10-29 13:28:04 +08:00
xiaolei373	14e7d88ea4	[feature] support reward api (#4518) ... Co-authored-by: SunLei <sunlei5788@gmail.com>	2025-10-29 00:20:28 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
SunLei	2a9ed72533	feat: add support for API usage with multimodal models (#4548) ... * feat: add support for API usage with multimodal models * completion_tokens contains num_image_tokens * remove test_request.py * fix: paddle.device.is_compiled_with_cuda() * fix test_unstream_without_logprobs	2025-10-28 20:23:46 +08:00
YuBaoku	e1ac90d787	[CI] Revert test_rollout_model directory change (#4626)	2025-10-28 20:14:00 +08:00
zhouchong	567f61072c	[CI][BugFix] fix port conflicts in concurrent ci test and add more unit test on async_llm (#4616) ... * fix:port conflicts in concurrent ci test * add more unit test on async_llm	2025-10-28 19:04:24 +08:00
yyssys	cd6d1f633c	[XPU]add xpu ci w4a8 case (#4501)	2025-10-28 19:02:29 +08:00
Zhang Yulong	b4014834a9	Extend sleep time to 10 seconds in switch_service (#4618) ... Increase sleep duration before switching services.	2025-10-28 15:19:21 +08:00
YuBaoku	b2c6c41447	[CI] Relocate server test cases from ci_use directory to e2e (#4608)	2025-10-28 11:37:30 +08:00
xiaolei373	31180a6a13	fix_run_batch_unittest (#4613)	2025-10-28 10:38:06 +08:00
ming1753	7681375a19	[BugFix] PaddleOCR-VL fix FD_DEBUG type and support v1 loader (#4605) ... * [Bug Fix] PaddleOCRVL fix FD_DEBUG type and support HF model * fix bug * fix bug * fix bug	2025-10-28 09:47:47 +08:00
K11OntheBoat	64d1aa973b	[Unitest]Add unitest of Attention Layer (#4494)	2025-10-27 19:18:50 +08:00
kevin	8aab4e367f	[Feature] mm support prefix cache (#4134) ... * support mm prefix caching * update code * fix mm_hashes * support encoder cache * add encoder cache * update code * update encoder cache * fix features bug * fix worker bug * support processor cache, need to optimize yet * refactor multimodal data cache * update code * update code * update v1 scheduler * update code * update code * update codestyle * support turn off processor cache and encoder cache * update pre-commit * fix code * solve review * update code * update code * update test case * set processor cache in GiB * update test case * support mm prefix caching for qwen model * fix code style check * update pre-commit * fix unit test * fix unit test * add ci test case * fix rescheduled bug * change text_after_process to prompt_tokens * fix unit test * fix chat template * change model path * [EP] fix adapter bugs (#4572) * Update expert_service.py * Update common_engine.py * Update expert_service.py * fix v1 hang bug (#4573) * fix import image_ops error on some platforms (#4559) * [CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) * add collect-env * del files * [Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) * add new branch for sot * reorder * fix batch bug * [XPU]Moe uses a new operator (#4585) * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response * [Feature] Support Paddle-OCR (#4396) * init * update code * fix code style & disable thinking * adapt for common_engine.update_mm_requests_chunk_size * use 3d rope * use flash_attn_unpadded * opt siglip * update to be compatible with the latest codebase * fix typo * optim OCR performance * fix bug * fix bug * fix bug * fix bug * normlize name * modify xpu rope * revert logger * fix bug * fix bug * fix bug * support default_v1 * optim performance * fix bug --------- Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> * [DataProcessor] add reasoning_tokens into usage info (#4520) * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test * perf: Optimize task queue communication from engine to worker (#4531) * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Clean up ports after processing results (#4587) * [CI] Add /re-run command in PR comments to restart failed CI workflows (#4593) * [Others] api server exits when worker process is dead (#3271) * [fix] fix terminal hangs when worker process is dead * [chore] change sleep time of monitor * [chore] remove redundant comments * update docs --------- Co-authored-by: ApplEOFDiscord <wwy640130@163.com> Co-authored-by: ApplEOFDiscord <31272106+ApplEOFDiscord@users.noreply.github.com> Co-authored-by: ltd0924 <32387785+ltd0924@users.noreply.github.com> Co-authored-by: yinwei <yinwei_hust@163.com> Co-authored-by: JYChen <zoooo0820@qq.com> Co-authored-by: qwes5s5 <45442318+qwes5s5@users.noreply.github.com> Co-authored-by: Ryan <zihaohuang@aliyun.com> Co-authored-by: yyssys <atyangshuang@foxmail.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com> Co-authored-by: root <root@szzj-acg-tge1-fdda9.szzj.baidu.com> Co-authored-by: zhangyue66 <zhangyue66@baidu.com> Co-authored-by: kxz2002 <115912648+kxz2002@users.noreply.github.com> Co-authored-by: SunLei <sunlei5788@gmail.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: Zhang Yulong <35552275+ZhangYulongg@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: 李泳桦 <39643373+liyonghua0910@users.noreply.github.com>	2025-10-27 17:39:51 +08:00
chen	5c63a089f6	[Feature] Support logprobs_mode (#4567)	2025-10-27 14:27:48 +08:00
CSWYF3634076	acd331780c	[V1 loader] Qwen25 VL support v1 loader and torch style safetensors load (#4388) ... * [BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix * [Docs]offine infer add apply_chat_template add_generation_prompt parameter * [Model]qwen2.5VL support --use-cudagraph * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v2 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v3 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v4 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v5 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v6 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v7 * qwen25vl v1 loader * qwen25vl v1 loader v2 * qwen25vl v1 loader v3 * qwen25vl v1 loader fix tp2 weight PySafeSlice * qwen25vl v1 loader no test * qwen25vl v1 loader add unit test * qwen25vl v1 loader add unit test v2 * qwen25vl v1 loader add torch unit test v3 * qwen25vl v1 loader add torch unit test v4 * qwen25vl v1 loader add torch unit test v5 * qwen25vl v1 loader add torch unit test v6	2025-10-27 10:54:15 +08:00
Zhang Yulong	83b720804b	Clean up ports after processing results (#4587)	2025-10-27 10:13:24 +08:00
SunLei	dc1a9c7287	perf: Optimize task queue communication from engine to worker (#4531) ... * perf: Optimize task queue communication from engine to worker * perf: get_tasks to numpy * perf: get_tasks remove to_numpy * fix: request & replace ENV * remove test_e2w_perf.py * fix code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-25 22:45:38 +08:00
kxz2002	327fa4c255	[DataProcessor] add reasoning_tokens into usage info (#4520) ... * add reasoning_tokens into usage info initial commit * add unit tests * modify unit test * modify and add unit tests * fix unit test * move steam usage to processor * modify processor * modify test_logprobs * modify test_logprobs.py * modify stream reasoning tokens accumulation * fix unit test	2025-10-25 16:57:58 +08:00
yyssys	822dea8d5f	[XPU]Moe uses a new operator (#4585) ... * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response	2025-10-24 23:01:46 +08:00
Ryan	f42ed6d5f2	[Graph Optimization] Add dy_runnable and introduce cudagraph_switch_threshold for cudagraph mode switching (#4578) ... * add new branch for sot * reorder * fix batch bug	2025-10-24 18:36:52 +08:00
qwes5s5	e02a812880	[CLI]Update parameters in bench latecy cli tool and fix collect-env cli tool (#4558) ... * add collect-env * del files	2025-10-24 16:46:45 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
luukunn	bbf06b9ff7	[BugFix]Fix finish reason (#4543) ... * fix finish reason * add unit test * add unit test * fix unie test * fix unit test	2025-10-23 14:04:43 +08:00
RAM	8a02ab43a8	[FDConfig]Turn on the CUDAGraph + RL switch (#4508) ... * Turn on the CUDAGraph + RL switch * reduce max_num_seqs and number of request	2025-10-23 11:08:07 +08:00
周周周	d7bcedf421	small change in test_fusedmoe.py (#4538)	2025-10-22 17:49:18 +08:00
Yuanle Liu	8e02a509c3	[CI] stable test_rollout_model.py (#4536) ... * stable test_rollout_model.py * update baseline * update baseline	2025-10-22 01:59:44 -07:00