290 Commits

Author	SHA1	Message	Date
GoldPancake	47595a2480	[Feature] support mtp logprob (#4464) ... * support mtp logprob * fix unitest	2025-10-20 15:18:12 +08:00
Haonan Luo	1b9f351d21	Support GPT-OSS-BF16 (#4240) ... * [Feature] AppendAtten support sinks & HEAD_DIM=64 * fix bug * fix bug * fix bug * fix bug * [Feature] support gpt-oss * fix bug * add mask * support-gpt-oss * support-gpt-oss * fix long seq * support wint8 * support wint8 * support wint8 * update test * change sliding windows init pos --------- Co-authored-by: ming1753 <ideaminghp@163.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com> Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com>	2025-10-20 14:44:58 +08:00
zhuzixuan	1e59905e34	Optimization of ‘tools’ in request fields (#4380) ... * Remove multiple 'tools' * Remove multiple 'tools' * Remove multiple 'tools' * Remove multiple 'tools'	2025-10-20 11:04:08 +08:00
YuBaoku	c4fc0073cf	[CI] Handle unit test issues (#4483)	2025-10-20 10:13:21 +08:00
周周周	817210e47f	[ATTN]delete code and add ffn and moe layer level test (#4440) ... * delete code * delete code * delete code * commit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit * copmmit	2025-10-19 16:23:11 +08:00
kxz2002	b5b993e48e	【feature】support n parameter (#4273) ... * support n parameter * pre-commit check * pre-commit check * restore format_and_add_data * update n_param * bug fix index - str to int * bug fix del child_task * bug fix metrics * add debug info * add debug info2 * remove debug info * change connecting symbol to '-' * bugfix change connecting symbol * bugfix change connecting symbol2 * unit tests fix * unit test fix2 * unittest add param n=2 * n param add unit tests and adapt to echo * pre-commit fix * resolve review * adjust stop reason * add unittest for _create_chat_completion_choice * modify unittest * solve confict * solve conflict * resolve conflict --------- Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: gaoziyuan <m13689897706@163.com>	2025-10-17 20:51:59 +08:00
kxz2002	8ccfd975b5	LLM.chat add "tools" param (#4415) ... * llm add tools param initial commit * llm add tools param bugfix * offline add tools add unittests * fix preprocessor * move tools paramter into tasks * change variable name	2025-10-17 20:25:03 +08:00
yinwei	a64c0408b9	[XPU]Fix w4a8 precision bug && rollback moe algo (#4463) ... * fix w4a8 precision bug * add env * code stype check	2025-10-17 18:27:53 +08:00
yzwu	4b661512ca	[Iluvatar GPU] Adapt VL model (#4313)	2025-10-17 16:13:38 +08:00
YuBaoku	01510876ab	[CI] Fix partial instability issues (#4461)	2025-10-17 14:17:06 +08:00
chen	b134e6afe6	[BugFix]Dev fix custom ar unstable result (#4437)	2025-10-17 11:47:16 +08:00
Ryan	6160145f82	[SOT] Change warnings to errors and remove fallback operations (#4378) ... * Change warnings to errors and remove fallback operations * fix unitest * fix codestyle	2025-10-17 11:27:04 +08:00
Ryan	49cea8fb1c	[SOT][Cudagraph] Remove BreakGraph of #3302 && update CustomOp (#3694) ... * rm inplace info && to(gpu) * update append_attention * unpin paddle version * add full_cuda_graph=False * add blank line --------- Co-authored-by: SigureMo <sigure.qaq@gmail.com>	2025-10-17 10:57:55 +08:00
chen	db82e9a022	[BugFix]Fix wfp8afp8 triton moe group_topk renormalized=True (#4449) ... * fix group_topk renormalized=True * check test	2025-10-16 23:17:48 +08:00
xiaolei373	dbca63f862	[bugfix] kill cache_transfer_manager process (#4401)	2025-10-16 20:45:24 +08:00
LiqinruiG	4251ac5e95	【Fix】 remove text_after_process & raw_prediction (#4421) ... * remove text_after_process & raw_prediction * remove text_after_process & raw_prediction	2025-10-16 19:00:18 +08:00
SunLei	b4b579a7ed	Feature：Add support for Pooling Model Embedding and provide an OpenAI-compatible API. (#4344) ... * feat: add OpenAIServing * feat: add ZmqOpenAIServing & OpenAIServingEmbedding * feat: Refine the basic ServingEngine class and introduce ServingContext * fix: codestyle * fix: request * fix: pooling_params * feat: _process_chat_template_kwargs * feat: support batch request * feat: pooling_params verify & default parameters --------- Co-authored-by: sunlei1024 <sunlei1024@example.com>	2025-10-15 19:42:59 +08:00
freeliuzc	744287e1a9	fix param (#4419)	2025-10-15 18:44:24 +08:00
AIbin	6938df9c23	【Fix CI Bug】Fix ci bug (#4413) ... * Support DSK-v3.2 model * Support DSK-v3.2 * Support DSK-v3.2 * Support DSK-3.2 * fix CI bug * fix_CI_BUG * update ci bug	2025-10-15 14:19:04 +08:00
freeliuzc	582aebd48b	[MTP]support mtp chunk_prefill_v1 (#4366) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output, fix unit test * fix unit test * fix save_output	2025-10-15 13:21:32 +08:00
qwes5s5	abb62624b8	[fix] Fixed the issue of excessive/redundant spans being returned for streaming requests. (#4375) ... * fix stream span * fix stream span	2025-10-15 11:47:47 +08:00
ltd0924	28d1b6cd97	[BugFix] fix multinode bugs (#4377) ... * [BugFix] fix multinode bugs * Update test_config.py * Update test_config.py * Update test_config.py --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>	2025-10-15 11:43:39 +08:00
YuBaoku	4b647d17de	[CI] Fix partial instability issues (#4418)	2025-10-15 11:13:43 +08:00
co63oc	73c8e0849f	【Hackathon 9th No.67】add speculate_verify (#4326) ... * add speculate_verify * fix * fix	2025-10-14 14:13:17 +08:00
Sunny-bot1	a751d977bc	[Optimization] Fuse get_max_len and get_kv_max_len (#4369) ... * opt split_q_block * fuse max_lens and max kv_len	2025-10-13 20:35:00 +08:00
ooo oo	2d641078c3	【Hackathon 9th No.20】add unit tests for masked_per_token_quant (#4111) ... * test: add unit tests for masked_per_token_quant * apply review	2025-10-13 14:51:11 +08:00
plusNew001	a21e16ee5f	[XPU] fix XPU CI bug (#4358) ... * Update assertions for response content in test_45t fix XPU CI bug * Comment out base_response print statement Comment out the print statement for base_response. * Refactor assertion for clarity in run_45T.py * Add blank line before main function call	2025-10-11 14:48:14 +08:00
YuanRisheng	a2ec2c4152	[FDConfig]Remove max_model_len in FDConfig (#4350) ... * modify max_model_len * fix unittest * fix unittest --------- Co-authored-by: root <root@yqlcc01-sys-rpm12rzmwjd.yqlcc01.baidu.com>	2025-10-11 14:04:17 +08:00
AIbin	368049673b	Add DeepSeek model end-to-end CI (#4360) ... Add DeepSeek-v3-5layers end-to-end CI	2025-10-11 08:33:37 +08:00
qwes5s5	6fd3e72da1	[FastDeploy Cli] Bench Command eval and throughput (#4239) ... * bench command * bench command * bench command * bench command * bench command --------- Co-authored-by: K11OntheBoat <your_email@example.com>	2025-10-10 16:17:44 +08:00
yinwei	20c7b741f4	[XPU] Support W4A8C8-TP4-300B Model (#4068) ... * support w4a8 * delete ep block attn * delete moe_topk_select * update note * update * delte useless info * update * add some note * fix some format * update scale info * add ans baseline --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-10 15:41:32 +08:00
AIbin	7b1689f437	schedule_bugfix (#4336)	2025-10-09 20:40:10 +08:00
xiaolei373	1282ebe1b1	add_cli_tokenizer (#4278)	2025-09-28 20:47:35 +08:00
xiaolei373	55124f8491	Add cli run batch (#4237) ... * feat(log):add_request_and_response_log * [cli] add run batch cli --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-26 14:27:25 +08:00
zhuzixuan	12a3587cca	[Supplements and upgrades]Improvement of X1 parsers (#4172) ... * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser * reasoning_parser	2025-09-26 13:37:37 +08:00
memoryCoderC	4ec00df2b0	[Feature] add config api (#4254)	2025-09-26 11:21:02 +08:00
K11OntheBoat	4515ad21e9	Support limit thinking lengths (#4069) ... Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>	2025-09-25 19:55:56 +08:00
ooo oo	e36eccfdad	【Hackathon 9th No.21、23】add unit tests for fused_hadamard_quant_fp8, moe_fused_hadamard_quant_fp8 (#4094) ... * test: add unit tests for fused_hadamard_quant_fp8 * test: add unit tests for moe_fused_hadamard_quant_fp8 * tests: simulate CUDA kernel's hadamard32_warp using butterfly operations * apply review * apply review	2025-09-25 12:15:00 +08:00
CSWYF3634076	5ff10c8ced	[Model] Qwen2.5VL support --use-cudagraph and unit testing (#4087) ... * [BugFix] qwen2.5vl enable_thinking=true and image_patch_id bug fix * [Docs]offine infer add apply_chat_template add_generation_prompt parameter * [Model]qwen2.5VL support --use-cudagraph * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v2 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v3 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v4 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v5 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v6 * [Model]qwen2.5VL support --use-cudagraph buffer and qwenvl test v7	2025-09-24 19:45:01 +08:00
luukunn	18f4977aec	[fix]update apply_chat_template (#4137) ... * update apply_chat_template * fix unittest * fix unittest * fix * fix * fix unit test * fix * fix unit test * add unit test	2025-09-24 18:56:32 +08:00
chen	7c1fd19f0f	[OPs] MoE support wfp8afp8(channelwise) and improve per_token_quant_fp8 (#4238)	2025-09-24 16:39:51 +08:00
memoryCoderC	8b0ce8e3ab	[Feature] add cli command serve (#4226)	2025-09-24 14:50:45 +08:00
co63oc	a1c5d930bb	【Hackathon 9th No.24】add rebuild_padding (#4107)	2025-09-24 12:08:17 +08:00
chen	ec99474e71	[Test]add glm45_air logprob test and rollout model (#4175) ... * add glm45_air logprob test * add glm rollout model and pretrainedmodel for rl * add glm rollout model and test * check * delete cudagraph in glm45 * add UT for glm rollout model * revert glm UT	2025-09-23 21:06:07 +08:00
bukejiyu	62d1c48363	[v1 loader]code style (#4204) ... * code style * update	2025-09-23 19:36:00 +08:00
chen	1a6283424e	Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)	2025-09-23 18:41:33 +08:00
lizexu123	c96a535a5d	[Feature] support qwen3-embedding model load (#4202) ... * support qwen3-embedding * fix ci bug * fix * fix ci bug * fix ci bug * fix	2025-09-23 00:14:35 -07:00
yangjianfengo1	4325b737e7	【FIX】Change the name of sparse attn from moba to plas (#4006) (#4076) ... * 【FIX】Change the name of sparse attn from moba to plas (#4006) * 更新文档 * 【docs】 update readme (#4000) * 更新文档 * update readme * update docs * 【FIX】Change the name of sparse attn from moba to plas (#3845) * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci * code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * fix max_num_seqs * fix test load attn --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-23 10:26:40 +08:00
plusNew001	2c34a557f4	[XPU]change xpu ci model (#4117) ... * change xpu ci model * change xpu ci model * change xpu ci model * change xpu ci model * Update model path and XPU settings in run_ci_xpu.sh * Increase health check timeout to 10 minutes Increased the timeout duration for health checks from 5 minutes to 10 minutes in two places. * Implement test for OpenAI chat completion Add a test function for the OpenAI client chat response. * Change script to use pytest for running tests * Update health check timeout to 15 minutes Increase the timeout for health checks from 10 minutes to 15 minutes. * Add pytest installation to CI script * Modify base response in test_45t function Updated the base response message for the test. * Add V0 and V1 mode test echo statements --------- Co-authored-by: root <root@yq01-inf-hic-k8s-a100-aa24-0591.yq01.baidu.com>	2025-09-23 10:21:17 +08:00
ltd0924	83720da79f	[Feature] support clear data (#3601) ... * [Feature] support clear data * update * fix * fix * fix * fix * fix * fix * fix	2025-09-23 10:20:02 +08:00