370 Commits

Author	SHA1	Message	Date
Zhang Yulong	5aa73d32f4	Update deploy.py (#4850)	2025-11-06 19:09:28 +08:00
YuBaoku	819b2dbbae	Revert "【New Feature】W4afp8 supports per group quantization (#4272)" (#4854) ... This reverts commit 93fcf7e4ec.	2025-11-06 17:48:28 +08:00
Juncai	08ca0f6aea	[Feature] [PD] add simple router and refine splitwise deployment (#4709) ... * add simple router and refine splitwise deployment * fix	2025-11-06 14:56:02 +08:00
plusNew001	fc8bef2c95	[XPU][CI]Change ci vl model to 28 b (#4764) ... * Update XPU_VISIBLE_DEVICES and model parameters * Update base response and adjust max tokens * Implement process cleanup in CI workflow Add process cleanup commands to prevent port conflicts * Remove process cleanup commands from CI workflow Removed old process cleanup commands to prevent port conflicts.	2025-11-06 14:12:23 +08:00
Echo-Nie	354ddc8bc5	[CI] Add unittest for activation, native_paddle_backend, w4a8, w4afp8, platforms/utils (#4812) ... * add unnitest for activation, native_paddle_backend, w4a8, w4afp8, platforms/utils * Remove activation function retrieval tests Removed tests for valid and unsupported activation function retrieval. * move w4a8, w4afp8 to quantization * fix code style	2025-11-06 14:08:00 +08:00
SunLei	782818c031	fix: ci port conflict (#4840)	2025-11-06 11:56:17 +08:00
kxz2002	5bdd40da5d	[BugFix] Fix ernie_vl_reasoning_parsers.py 'end_token' to 'think_end_token' (#4805) ... * fix ernie_vl_reasoning_parsers.py 'end_token' to 'think_end_token' * add unit tests	2025-11-06 11:28:55 +08:00
yangjianfengo1	93fcf7e4ec	【New Feature】W4afp8 supports per group quantization (#4272) ... * w4afp8 支持per group * code style * 精度完成 * revert append attn utils * ffn1 动态量化 * ffn2 支持动态量化 * code style * code style * 修改单测 * 修改单测 * fix bug * Implement conditional parameter creation for layers Add parameter creation for up_gate_proj_in_scale when ep_size > 1. * code style * fix conflict * code style * code style * 修复w4aint8 精度 * fix ci --------- Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com>	2025-11-05 21:00:23 +08:00
chenjian	cc8f5312f5	[Feature] Add timestamp for profiler (#4726) ... * [Feature] Add timestamp for profiler * fix bug for offine inference * fix for ci * fix * fix ci	2025-11-05 12:04:59 +08:00
周周周	876e4a8935	remove input_ids from ForwardMeta (#4793)	2025-11-05 11:55:51 +08:00
kxz2002	9676cc87d6	fix parser register name (#4795) ... Co-authored-by: luukunn <83932082+luukunn@users.noreply.github.com>	2025-11-05 11:27:30 +08:00
zhupengyang	2fd254e5b7	support ep+tp at op layer (#4688)	2025-11-05 11:15:57 +08:00
周周周	937eb3c6ed	[get_padding_offset.] clean get_padding_offset.cu (#4777) ... [get_padding_offset.] clean get_padding_offset.cu (#4777)	2025-11-05 10:47:40 +08:00
Haonan Luo	2c281e617c	Update Unit Test for PaddleOCR-VL (#4802) ... * fix paddleocr prefix cache bug * add test for paddleocr_vl * disable prefix-caching in ocr * add test for paddleocr_vl * Fix top_p for rejection sampling * add test for ocr processor; fix top_p for rejection sampling * add test for ocr processor; fix top_p for rejection sampling * add test for ocr processor; fix top_p for rejection sampling * add test for ocr processor; fix top_p for rejection sampling * add test for ocr processor; fix top_p for rejection sampling --------- Co-authored-by: ming1753 <ideaminghp@163.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com>	2025-11-04 22:40:15 +08:00
YuBaoku	722110a952	[CI] Refactor CE wheel upload for multiple target paths (#4790) ... * [CI] Refactor CE wheel upload for multiple target paths * [CI] fix test_streaming_with_stop_str error	2025-11-04 18:56:38 +08:00
kxz2002	8a40374bfe	[BugFix] Fix ernie4_5_vl_processor.py and qwen_vl_processor.py can not disable thinking (#4762) ... * fix ernie4_5_vl_processor.py and qwen_vl_processor.py * add unit test	2025-11-04 16:00:32 +08:00
bukejiyu	41bfa1090d	[CI]delete test_common_model (#4794) ... * fix * update * update	2025-11-04 13:57:55 +08:00
plusNew001	9887025926	Update run_w4a8.py (#4783)	2025-11-03 21:41:00 +08:00
bukejiyu	69c2f3cda1	[CI]test common model (#4697) ... * ut * update	2025-11-03 16:48:36 +08:00
luukunn	7b35488779	【DataProcessor】add options thinking_mode (#4735) ... * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add thinking_mode * add unit test	2025-11-03 14:30:07 +08:00
yinwei	377f3bf5f2	[XPU] add v1 support for bf16 (#4744) ... * support v1 loader * update code style * update code	2025-11-03 14:13:17 +08:00
freeliuzc	11398790d3	[Speculative Decoding][MTP]Support attn mask offset (#4641) ... * [MTP]Merge support attn (#4591) * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import * support attn_mask_offset in mtp mode * add update_attn_mask op * fix unit test && fix code-style	2025-11-03 10:08:01 +08:00
YuBaoku	9eff788658	[CI] fix some ci yaml (#4747)	2025-11-02 21:28:04 +08:00
lizexu123	4ac6de9a3c	[Feature] support pooling model runner (#4590) ... * support qwen3-embedding * support qwen3-embedding-0.6b * fix * fix bug * fix test_return_token_ids.py and update enable_thinking * fix mtp dummy_run * merge develop * fix np.float32 * delete FD_DISABLE_CHUNKED_PREFILL and FD_USE_GET_SAVE_OUTPUT_V1 * delete and build_stream_transfer_data * fix test_update_v1: * fix * fix * update dummy_run post_process * delete test_update_v1 * fix * fix dummy_run * fix model_path * fix model_path * fix dummy_run	2025-10-31 22:32:05 +08:00
YuBaoku	acef624049	[CI] Fix rollout_model test logic (#4730)	2025-10-31 22:25:24 +08:00
Yuanle Liu	b301bd6c31	[BugFix] fix thinking bug (#4710) ... * fix thinking bug * fix ut * update * fix	2025-10-31 22:00:31 +08:00
xiaolei373	dde7ba3f9e	[CI]add_tokenizer_cli_unitest (#4620)	2025-10-31 13:57:51 +08:00
周周周	10de7a3b82	add flops and bandwidth to test_ffn.py (#4704)	2025-10-31 12:13:59 +08:00
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
Ryan	28de91b50f	[Graph Optimization] SOT+CUDAGraph support ERNIE4.5T VL 28B / 424B (#4645) ... * 45TVL support sot+CUDAGraph * mv unitest from ce_deploy 2 e2e * add test_EB_VL_Lite_sot_serving * rm useless line * add openai_client * fix unitest && reduce computing resources	2025-10-31 11:38:43 +08:00
Longzhi Wang	b61a272385	[BugFix] fix unittest of get_save_output_v1 (#4701) ... * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1	2025-10-31 11:23:49 +08:00
kxz2002	a2870ed4a9	[Feature] Unify the registration name recognition for tool_parser and reasoning_parser to “-” (#4668) ... * parser register name unify * change ernie_x1 to ernie-x1 * change ernie4_5_vl to ernie-45-vl * fix unit test	2025-10-31 10:45:27 +08:00
kxz2002	82bd7e5db4	[BugFix] Fix finish reason in _create_chat_completion_choice (#4582) ... * fix n_param _create_chat_completion_choicel * fix unit test * fix final_res * modify unit tests	2025-10-31 10:42:19 +08:00
plusNew001	ea866e4b34	[XPU] [CI] Add Vl case (#4649) ... * Enhance CI script with health checks and logging Updated the CI script to include health checks and logging for the VL model testing process. * Add test for OpenAI chat completions * Refactor chat completion user message structure * Fix variable name for exit code in CI script * Update text prompt to Chinese for artifact question * Update service port and response assertions in tests * Refactor assertion for response content comparison * Update run_45vl.py * Change service HTTP port from 8123 to 8188	2025-10-31 10:38:09 +08:00
zhouchong	35286ce31a	fix total_block_num init error in worker_process (#4687)	2025-10-30 19:53:09 +08:00
周周周	8b9c9463cd	add real gate_correction_bias weight to mock un-balanced dispatch (#4676)	2025-10-30 15:13:21 +08:00
Haonan Luo	d7d0112bbf	[CI] Add test for paddleocr_vl (#4627)	2025-10-30 13:40:04 +08:00
RAM	cd3b7cc392	[Graph Optimization] Add the CUDAGraph usage switch for Draft Model (#4601) ... * add draft model using cudagraph switch * set default as false * capture draft model in ci * fix bug	2025-10-30 11:44:50 +08:00
GoldPancake	fddda50cb9	Add ut for speculative sampler (#4650)	2025-10-30 10:37:49 +08:00
周周周	dab04ab413	add noaux_tc to unitest fused_moe (#4656)	2025-10-29 21:50:25 +08:00
kxz2002	c30bfb294f	[Feature] add a new reasoning parser (#4571) ... * add new reasoning_parser initial commit * add parser file content * add register * ernie_test_reasoning_parser * support <tool_call> token and add tool_parser * add and fix unit tests * modify reasoning_parser * modify reasoning parser and tool parser * modify unit tests * modify reasoning_parser and tool_parser * modify unit tests * fix tool_parser * modify the logic of reasoning_parser and tool_parser * add and modify unit tests * standardize code style * simplify reasoning_parser and tool_parser * modify unit test	2025-10-29 18:16:50 +08:00
ApplEOFDiscord	14f8cddaf1	[Feature] add mm token usage (#4570) ... * add mm token usage * fix unit test * fix unit test * fix unit test * fix model path * fix unit test * fix unit test * fix unit test * remove uncomment * change var name * fix code style * fix code style * fix code style * fix code style * fix unit test	2025-10-29 14:37:12 +08:00
RAM	fc5cd1adb1	[BugFix] Fix graph opt test case (#4634) ... * fix bug and refine code * add debug count * refine code * fix ci test case	2025-10-29 13:28:04 +08:00
xiaolei373	14e7d88ea4	[feature] support reward api (#4518) ... Co-authored-by: SunLei <sunlei5788@gmail.com>	2025-10-29 00:20:28 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
SunLei	2a9ed72533	feat: add support for API usage with multimodal models (#4548) ... * feat: add support for API usage with multimodal models * completion_tokens contains num_image_tokens * remove test_request.py * fix: paddle.device.is_compiled_with_cuda() * fix test_unstream_without_logprobs	2025-10-28 20:23:46 +08:00
YuBaoku	e1ac90d787	[CI] Revert test_rollout_model directory change (#4626)	2025-10-28 20:14:00 +08:00
zhouchong	567f61072c	[CI][BugFix] fix port conflicts in concurrent ci test and add more unit test on async_llm (#4616) ... * fix:port conflicts in concurrent ci test * add more unit test on async_llm	2025-10-28 19:04:24 +08:00
yyssys	cd6d1f633c	[XPU]add xpu ci w4a8 case (#4501)	2025-10-28 19:02:29 +08:00
Zhang Yulong	b4014834a9	Extend sleep time to 10 seconds in switch_service (#4618) ... Increase sleep duration before switching services.	2025-10-28 15:19:21 +08:00