133 Commits

Author	SHA1	Message	Date
yangjianfengo1	3754a9906d	[Feature] block sparse attention (#3668) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config * 修复显存 * add test server * add test server * fix mlp 最后一层使用full attn	2025-08-29 19:46:30 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
Ryan	45f81b34f0	add dtype int32 (#3692)	2025-08-29 14:56:35 +08:00
李泳桦	88297240e7	[feat] completion api supports passing input token ids in either prompt or prompt_token_ids (#3311) ... * [feat] completion api supports passing input token ids in either `prompt` or `prompt_token_ids` * [fix] update comment * [fix] fix type error * [test] add a unittest file for serving api test * [test] try to fix ci error * [chore] rename test function names * [test] try to fix ci error * [test] try to fix ci error * [test] add tests for qwen	2025-08-29 14:19:42 +08:00
周周周	17b414c2df	MoE Default use triton's blockwise fp8 in TP Case (#3678)	2025-08-29 11:07:30 +08:00
Echo-Nie	43d5bd62b4	【Hackathon 9th No.70】supplementary unit test for CPUPlatform and CUDAPlatform (#3580) ... * 功能模块 CUDAPlatform、CPUPlatform 单测补充 * update the "is_cuda" to "is_cuda_and_available" * fix pre-commit --------- Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-08-29 10:34:05 +08:00
Yuanle Liu	4957908275	add input_processor plugin (#3657) ... * add input_processor plugin * update * update * update * update * update * update * update * update * update * update * update	2025-08-28 22:53:57 +08:00
Divano	17731a8acd	add concurrency cases (#3689)	2025-08-28 18:30:19 +08:00
Liumengyuan	e93d4cfcdd	Add with_output version AppendAttention (#3302) ... * get use_output from fd_config * add clear TODO description * add mask_offset para to align with develop * fix bug * fix use_output logic * fix sot bug	2025-08-28 17:10:18 +08:00
bukejiyu	73cf6096da	fix (#3676) ... * fix * update	2025-08-28 17:06:32 +08:00
co63oc	d4fc893fe3	fix typos (#3633) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-28 14:42:24 +08:00
Echo-Nie	7afcd4b776	【Hackathon 9th No.77】supplementary unit test for get_filtered_metrics (#3578) ... * 功能模块 fastdeploy/metrics/metrics/get_filtered_metrics 单测补充 * fix pre-commit --------- Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-08-28 10:39:02 +08:00
ltd0924	3d92fb09f7	[BugFix] fix parameter is 0 (#3592) ... * Update engine_client.py * fix * Update common_engine.py	2025-08-28 09:52:36 +08:00
Sunny-bot1	479c8b85d3	[Optimize]support machete weight only gemm (#3561) ... * support machete weight only gemm * add generate * update * fix * change file location * add sm_version limit * fix * fix * fix ci * fix coverage * fix xpu	2025-08-28 09:49:58 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
Jiang-Jia-Jun	c694fa2879	Revert "[Feature] block sparse attention (#3209)" (#3647) ... This reverts commit 646a0c2fd8.	2025-08-27 17:35:04 +08:00
lzy	1265f6c192	deepgemm don't support tp+ep (for ci) (#3638) ... * deepgemm don't support tp+ep (for ci) * deepgemm don't support tp+ep (for ci)	2025-08-27 16:39:19 +08:00
xjkmfa	afb9f327ef	【CI case】for echo finish_reason text_after_process and raw_prediction check (#3630) ... * Add ci case for min token and max token * 【CI case】include total_tokens in the last packet of completion interface stream output * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check --------- Co-authored-by: xujing43 <xujing43@baidu.com>	2025-08-27 15:21:16 +08:00
YUNSHEN XIE	85afa72763	fix publish task (#3635) ... * fix publish task * disable ut	2025-08-27 11:14:53 +08:00
yangjianfengo1	646a0c2fd8	[Feature] block sparse attention (#3209) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config	2025-08-26 07:16:04 -07:00
RAM	f0a362af18	[CUDAGraph]Switch the scope so that output buffer of CUDAGraph can automatically release (#3612) ... * fix typo * fix typo * add print dot files * fix bug * Switch the scope so that output buffer of cudagraph can automatically release * Revert "add print dot files" This reverts commit dc21809eb5.	2025-08-26 21:28:19 +08:00
Yuanle Liu	cbce94a00e	rename ernie_xxx to ernie4_5_xxx (#3621) ... * rename ernie_xxx to ernie4_5_xxx * ci fix	2025-08-26 19:29:27 +08:00
YuanRisheng	642480f5f6	[CI] Standard unittest (#3606) ... * standard unittest * fix bugs * fix script	2025-08-26 19:03:11 +08:00
bukejiyu	3200a80de3	[v1 loader]support fp8 (#3593) ... * support fp8 * update ci	2025-08-26 02:42:46 -07:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
Ryan	a5b4866ff1	[CudaGraph][SOT] Add unit tests for splitting the static graph into piecewise graphs that support cuda_graph (#3590) ... * add unitest * change sot_warmup_sizes * wtf; add missed commit	2025-08-26 11:25:04 +08:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
lizexu123	c43a4bec00	[Features] support hugging face qwen3 dense and qwen2 model (#3574) ... * support qwen2 and qwen3 hugging face * fix moe * defualt_v1 loader * hugging_face_format deprecated * modify hugging_face_foramt to model_format * model_format auto * fix environemt * fix bug * fix qwen3-0.6 bug * model_format is str * fix	2025-08-26 10:54:53 +08:00
RAM	2fa173e327	[Executor] CUDAGraph support RL training (#3265) ... * add clear graph opt backend * cuda graph support rl * add branch * 1.fix dynamic_weight_manager bug 2.add clear api for CasualLM * open test case * fix typo * update mkdocs.yaml * [Docs]Update mkdocs.yml * update test case * use unittest in graph test case	2025-08-25 20:59:30 +08:00
chen	9cab3f47ff	[Feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing (#3552) ... * [feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing * infer engine support temp_scaled_logprobs and top_p_normalized_logprobs * delete some code * code check * code check and add doc * fix tokenizer.decoder(-1), return 'Invalid Token' * add ci for temp_scaled and top_p logprobs * check test * check seq len time shape * logprob clip inf --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com>	2025-08-25 14:11:49 +08:00
Yuan Xiaolan	9205c88da1	support w4afp8 EP inference (#3044)	2025-08-25 11:27:45 +08:00
YUNSHEN XIE	46664985fc	Modify the existing coverage collection method (#3573) ... fix cov report	2025-08-25 10:35:35 +08:00
YuBaoku	7821534ff5	[CI] add sot test (#3579) ... * [CI] add sot test * [CI] add sot test	2025-08-25 10:14:50 +08:00
bukejiyu	bdbac0aa3d	support qwen2 weight only (#3571)	2025-08-24 11:14:34 +08:00
bukejiyu	77514e3e1e	[V1 Loader] support weight_only (#3413) ... * support wint4/wint8 * delete smoe case * update ci * print log	2025-08-23 13:13:41 +08:00
Jiang-Jia-Jun	93e1b63200	Revert "[UnitTest][Copilot] Improve unit test coverage for entrypoints module…" (#3564) ... This reverts commit 36325e9ea7.	2025-08-23 10:44:23 +08:00
YUNSHEN XIE	cb166053ba	fix test name (#3493) ... * fix test name * update * update * fix * fix * update * update * update * update * update * fix * update	2025-08-22 23:43:47 +08:00
Copilot	36325e9ea7	[UnitTest][Copilot] Improve unit test coverage for entrypoints modules (#3546) ... * Initial plan * Add comprehensive unit tests for entrypoints utilities Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Complete entrypoints test coverage improvement with tool parser tests Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Apply pre-commit formatting to test files - fix trailing whitespace and long lines --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-22 19:20:51 +08:00
lddfym	27666ee586	[Feature] Add Qwen25-VL Processor (#3501) ... * add qwen-2.5-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor position_ids * add qwen25-vl processor * add qwen25-vl processor * position_ids * add test for qwen25-vl * organize comments * formatted * qwen_vl_processor * add qwen_vl_processor unittest * update model path * update model path * update qwen_vl_processor unittest * add unittest and bug fix * add unittest and bug fix * Update fastdeploy/input/qwen_mm_processor/image_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/input/qwen_vl_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-22 16:49:42 +08:00
freeliuzc	76759108c9	[Feature][SpeculativeDecoding]Support tree-attention (#3514) ... * support tree-attention * fix merge bug * fix unit-test api * fix merge bug	2025-08-22 13:36:41 +08:00
YuanRisheng	85fbf5455a	[V1 Loader]Ernie VL support loader v1 (#3494) ... * ernie vl support new loader * add unittest * fix test	2025-08-22 11:16:57 +08:00
Zhang Yulong	3cc182236a	update ci (#3519)	2025-08-21 20:05:50 +08:00
YuanRisheng	c389a4013c	Unify server-side and model-side Config(Part-5) (#3497) ... * move config * fix xpu * fix * fix vl * fix vl * fix unitest * fix args * add unitest * fix test	2025-08-21 19:00:21 +08:00
yangjianfengo1	e5aa7087db	【bug fix】修复w4a8编译慢 (#3510) ... * 修复w4a8编译 * code style * 修复tma copy	2025-08-21 18:50:14 +08:00
Zhang Yulong	a5692e8b7d	Add PD CI case (#3490) ... * Create test_ernie_03b_pd.py * Update test_ernie_03b_pd.py	2025-08-21 18:48:34 +08:00
李泳桦	e4f0b755b4	[fix] setting disable_chat_template while passing prompt_token_ids led to response error (#3228) ... * [fix] setting disable_chat_template while passing prompt_token_ids led to response error * [fix] code syntax * [test] add test case for this bug * [test] add test case for empty message list * [test] fix test case for empty message list	2025-08-21 17:30:51 +08:00
luukunn	371fb3f853	[Feature] add tool parser (#3483) ... * add tool parser * add x1 enable_thinking * restart ci * fix vl reasoning parser * modify call style * modify call style * add offline enablethinking * fix completion * fix * fix unit test * fix unit test * fix unit test * fix vl reasoning parser * fix vl reasoning parser	2025-08-21 17:25:44 +08:00
Yzc216	466cbb5a99	[Feature] Models api (#3073) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/openai/serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-21 17:02:56 +08:00
Zhang Yulong	b7eee3aec1	Update CI (#3474) ... * update CI cases * update CI cases * update CI cases * update CI cases * Merge upstream/develop and resolve directory rename conflict * Merge upstream/develop and resolve directory rename conflict * Merge upstream/develop and resolve directory rename conflict * update deploy * update deploy * update deploy * update deploy * update deploy	2025-08-21 16:49:20 +08:00
ltd0924	51f68ae593	[Feature] add dealer manager to reuse the connection (#3471) ... * [BugFix] fix control signal release failed * [BugFix] fix control signal release failed * update * update * update * [Feature] add dealer manager to reuse the connection * fix * fix * fix * fix * fix * fix * Create test_dealer_connection_manager.py * Delete test/entrypoints/openai directory * Update test_dealer_connection_manager.py * Update test_dealer_connection_manager.py	2025-08-21 13:11:13 +08:00