53 Commits

Author	SHA1	Message	Date
plusNew001	bd28f18785	[XPU][CI] Ci release update (#5085) ... * Update assertion to check for specific keywords * Remove unused base_response variable Remove unused variable 'base_response'. * Update assertion to check for multiple keywords * Update run_w4a8.py	2025-11-17 19:01:13 +08:00
plusNew001	bbd9c96ab9	[XPU] [CI]Change CI to multi-concurrency (#4924) ... * Refactor GPU ID logic in CI workflow Updated GPU ID assignment logic and removed unused port variables. * Refactor process handling and update model paths Refactor process termination logic into a function and update model paths for various tests. * Modify service_http_port calculation and clean up code * Modify test_45vl to use dynamic service port and response * Refactor model path and enable tensor parallelism Updated model path retrieval and added tensor parallelism configuration. * Modify service port configuration and cleanup code * Update run_45vl.py * Update health check endpoint to use port variable * Update CI script for paddlepaddle-xpu installation * Replace variable usage with ENDPOINT in script * Refactor EP test script for clarity and efficiency Updated echo statements and removed redundant code for EP tests. * Refactor test_fd_ep function for clarity	2025-11-12 14:43:55 +08:00
plusNew001	c6e9717f33	[XPU][CI]Update test assertion and base response value (#4908)	2025-11-10 12:59:04 +08:00
plusNew001	329e999f2d	[XPU][CI] Release ci bug fix (#4892) ... * Update run_45T.py * Refactor test_w4a8 to simplify assertions Comment out base response variables and update assertion to check for '人工智能'.	2025-11-07 17:49:41 +08:00
YuBaoku	71bbedaf50	[Cherry-Pick][BugFix][CI] fix vl moe(#4867) (#4869) ... * [CI] update paddlepaddle_gpu==3.2.1 and fix rollout_model test logic * [Cherry-Pick][BugFix][CI] fix vl moe(#4867)	2025-11-07 00:03:36 +08:00
YuBaoku	7cee8030af	[CI] Disable unstable test jobs and cases (#4799) ... [CI] Disable unstable test jobs and cases	2025-11-05 10:28:53 +08:00
plusNew001	ffa57dbfac	Modify base_response_104 for better clarity (#4789) ... Updated the base_response_104 message to improve wording.	2025-11-03 22:08:32 +08:00
YuBaoku	8632b778f5	[CI] update paddlepaddle_gpu==3.2.1 and fix rollout_model test logic (#4738)	2025-11-02 21:30:23 +08:00
yinwei	b4aa189483	[XPU] Support V1 Loader in Bf16 (#4746) ... * add v1 support for bf16 * update * update * update * update * update * update code	2025-11-01 16:13:25 +08:00
plusNew001	7b013c63e2	[XPU ][CI] Release XPU ci update (#4722) ... * Refactor CI script for paddlepaddle-xpu installation Updated the CI script to install specific paddlepaddle-xpu version and modified the testing commands for better performance and error handling. * Add test script for OpenAI client interaction * Remove empty line at the beginning of run_45vl.py	2025-10-31 15:36:14 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
YuBaoku	e1ac90d787	[CI] Revert test_rollout_model directory change (#4626)	2025-10-28 20:14:00 +08:00
yyssys	cd6d1f633c	[XPU]add xpu ci w4a8 case (#4501)	2025-10-28 19:02:29 +08:00
YuBaoku	b2c6c41447	[CI] Relocate server test cases from ci_use directory to e2e (#4608)	2025-10-28 11:37:30 +08:00
yyssys	822dea8d5f	[XPU]Moe uses a new operator (#4585) ... * [XPU]Moe uses a new operator * [XPU]Moe uses a new operator * update response	2025-10-24 23:01:46 +08:00
Sunny-bot1	4ffe41a747	WINT4/WINT8 dense gemm default use Machete (#4451)	2025-10-23 17:57:59 +08:00
Yuanle Liu	8e02a509c3	[CI] stable test_rollout_model.py (#4536) ... * stable test_rollout_model.py * update baseline * update baseline	2025-10-22 01:59:44 -07:00
yzwu	dc7facaa7f	[Iluvatar GPU] fix ci error caused by rebuild_padding param and cuda graph (#4504)	2025-10-21 21:41:41 +08:00
plusNew001	2bd3fb6315	[XPU]add xpu ci ep case (#4432) ... * add xpu ci case * Add xDeepEP download and build steps Download and build xDeepEP before running tests. * Fix formatting and add missing sleep command * Update Docker image version in CI workflow * Modify run_ci_xpu.sh for log cleanup and error handling Clean up log files before running tests and output worker log on failure. * Enhance test_ep.py with process management and assertions Refactor test function to include process cleanup and assertions. * Replace test_fastdeploy_llm with test_fd_ep * Fix conditional statement in run_ci_xpu.sh * Update test_ep.py for string handling and formatting Fix string encoding issues and improve readability. * Rename test_ep.py to run_ep.py * Change test script from test_ep.py to run_ep.py	2025-10-21 19:19:40 +08:00
RAM	775edcc09a	[Executor] Default use CUDAGraph (#3594) ... * add start intercept * Adjustment GraphOptConfig * pre-commit * default use cudagraph * set default value * default use cuda graph * pre-commit * fix test case bug * disable rl * fix moba attention * only support gpu * Temporarily disable PD Disaggregation * set max_num_seqs of test case as 1 * set max_num_seqs and temperature * fix max_num_batched_tokens bug * close cuda graph * success run wint2 * profile run with max_num_batched_tokens * 1.add c++ memchecker 2.success run wint2 * updatee a800 yaml * update docs * 1. delete check 2. fix plas attn test case * default use use_unique_memory_pool * add try-except for warmup * ban mtp, mm, rl * fix test case mock * fix ci bug * fix form_model_get_output_topp0 bug * fix ci bug * refine deepseek ci * refine code * Disable PD * fix sot yaml	2025-10-21 14:25:45 +08:00
YuBaoku	70a29ec49e	[CI] update ernie-4_5-vl baseline (#4495) ... * [CI] update ernie-4_5-vl baseline * [CI] update Qwen2.5-VL-7B-Instruct baseline	2025-10-21 10:18:29 +08:00
yinwei	a64c0408b9	[XPU]Fix w4a8 precision bug && rollback moe algo (#4463) ... * fix w4a8 precision bug * add env * code stype check	2025-10-17 18:27:53 +08:00
YuBaoku	01510876ab	[CI] Fix partial instability issues (#4461)	2025-10-17 14:17:06 +08:00
plusNew001	a21e16ee5f	[XPU] fix XPU CI bug (#4358) ... * Update assertions for response content in test_45t fix XPU CI bug * Comment out base_response print statement Comment out the print statement for base_response. * Refactor assertion for clarity in run_45T.py * Add blank line before main function call	2025-10-11 14:48:14 +08:00
yinwei	20c7b741f4	[XPU] Support W4A8C8-TP4-300B Model (#4068) ... * support w4a8 * delete ep block attn * delete moe_topk_select * update note * update * delte useless info * update * add some note * fix some format * update scale info * add ans baseline --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-10 15:41:32 +08:00
memoryCoderC	4ec00df2b0	[Feature] add config api (#4254)	2025-09-26 11:21:02 +08:00
K11OntheBoat	4515ad21e9	Support limit thinking lengths (#4069) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-09-25 19:55:56 +08:00
CSWYF3634076	5ff10c8ced	[Model] Qwen2.5VL support --use-cudagraph and unit testing (#4087) ... * [BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix * [Docs]offine infer add apply_chat_template add_generation_prompt parameter * [Model]qwen2.5VL support --use-cudagraph * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v2 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v3 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v4 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v5 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v6 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v7	2025-09-24 19:45:01 +08:00
chen	ec99474e71	[Test]add glm45_air logprob test and rollout model (#4175) ... * add glm45_air logprob test * add glm rollout model and pretrainedmodel for rl * add glm rollout model and test * check * delete cudagraph in glm45 * add UT for glm rollout model * revert glm UT	2025-09-23 21:06:07 +08:00
plusNew001	2c34a557f4	[XPU]change xpu ci model (#4117) ... * change xpu ci model * change xpu ci model * change xpu ci model * change xpu ci model * Update model path and XPU settings in run_ci_xpu.sh * Increase health check timeout to 10 minutes Increased the timeout duration for health checks from 5 minutes to 10 minutes in two places. * Implement test for OpenAI chat completion Add a test function for the OpenAI client chat response. * Change script to use pytest for running tests * Update health check timeout to 15 minutes Increase the timeout for health checks from 10 minutes to 15 minutes. * Add pytest installation to CI script * Modify base response in test_45t function Updated the base response message for the test. * Add V0 and V1 mode test echo statements --------- Co-authored-by: root <root@yq01-inf-hic-k8s-a100-aa24-0591.yq01.baidu.com>	2025-09-23 10:21:17 +08:00
yzwu	504461b6b5	[Iluvatar GPU] Optimize attention performance and fix moe load ckpt error (#3651)	2025-09-22 21:13:59 +08:00
co63oc	c4830ef24c	fix typos (#4176) ... * fix typos * fix	2025-09-22 14:27:17 +08:00
chenjian	618ccdbfba	[Feature] Support mixed deployment with yiyan adapter in develop (#3976) ... * [Feature] Support mixed deployment with yiyan adapter in release2.2 * fix metrics * add unit test * add unit test * add unit test * fix ci * fix for eb5 * fix ci * fix ci * fix ci --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-18 01:52:20 +08:00
YuBaoku	2745f37017	[CI] enhance clean port and add waiting time (#4152)	2025-09-17 20:31:49 +08:00
YUNSHEN XIE	c01a756912	mv test to tests (#4129)	2025-09-16 20:45:40 +08:00
zhouchong	958abebeab	Support offline inference with streaming output (#4071) ... * Support offline inference with streaming output * add unit test --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-15 10:27:03 +08:00
YuBaoku	fec58639db	[CI] skip test_structured_outputs* temporarily (#4055)	2025-09-11 18:07:50 +08:00
qwes5s5	17169a14f2	[metrics] Add serveral observability metrics (#3868) ... * Add several observability metrics * [wenxin-tools-584] 【可观测性】支持查看本节点的并发数、剩余block_size、排队请求数等信息 * adjust some metrics and md files * trigger ci * adjust ci file * trigger ci * trigger ci --------- Co-authored-by: K11OntheBoat <your_email@example.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-08 14:13:13 +08:00
Zhang Yulong	349aa6348b	add cache queue port (#3904) ... * add cache queue port * add cache queue port * add cache queue port	2025-09-05 21:17:06 +08:00
Zhang Yulong	4c160aa4dd	Update test_ernie_21b_mtp.py (#3885)	2025-09-04 20:20:36 +08:00
kevin	1908465542	[Feature] mm and thinking model support structred output (#2749) ... * mm support structured output * update code * update code * update format * update code * update code * add enable_thinking default * update code * add structured_outputs test case * add ci install xgrammar * add ci timeout time * update test for structured_outputs * update code * add error traceback info * update error msg * update structred output code * update code * update code * update config * update torch version --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-02 16:21:09 +08:00
Zhang Yulong	adc246127b	Update test_ernie_21b_mtp.py (#3783) ... 暂时跳过多卡MTP case	2025-09-01 20:39:40 +08:00
Zhang Yulong	085fe070f2	add CI cases (#3714)	2025-09-01 10:06:49 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
Yuanle Liu	cbce94a00e	rename ernie_xxx to ernie4_5_xxx (#3621) ... * rename ernie_xxx to ernie4_5_xxx * ci fix	2025-08-26 19:29:27 +08:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
bukejiyu	77514e3e1e	[V1 Loader] support weight_only (#3413) ... * support wint4/wint8 * delete smoe case * update ci * print log	2025-08-23 13:13:41 +08:00
Zhang Yulong	3cc182236a	update ci (#3519)	2025-08-21 20:05:50 +08:00
Zhang Yulong	a5692e8b7d	Add PD CI case (#3490) ... * Create test_ernie_03b_pd.py * Update test_ernie_03b_pd.py	2025-08-21 18:48:34 +08:00
李泳桦	e4f0b755b4	[fix] setting disable_chat_template while passing prompt_token_ids led to response error (#3228) ... * [fix] setting disable_chat_template while passing prompt_token_ids led to response error * [fix] code syntax * [test] add test case for this bug * [test] add test case for empty message list * [test] fix test case for empty message list	2025-08-21 17:30:51 +08:00