126 Commits

Author	SHA1	Message	Date
Divano	17731a8acd	add concurrency cases (#3689)	2025-08-28 18:30:19 +08:00
Liumengyuan	e93d4cfcdd	Add with_output version AppendAttention (#3302) ... * get use_output from fd_config * add clear TODO description * add mask_offset para to align with develop * fix bug * fix use_output logic * fix sot bug	2025-08-28 17:10:18 +08:00
bukejiyu	73cf6096da	fix (#3676) ... * fix * update	2025-08-28 17:06:32 +08:00
co63oc	d4fc893fe3	fix typos (#3633) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-28 14:42:24 +08:00
Echo-Nie	7afcd4b776	【Hackathon 9th No.77】supplementary unit test for get_filtered_metrics (#3578) ... * 功能模块 fastdeploy/metrics/metrics/get_filtered_metrics 单测补充 * fix pre-commit --------- Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-08-28 10:39:02 +08:00
ltd0924	3d92fb09f7	[BugFix] fix parameter is 0 (#3592) ... * Update engine_client.py * fix * Update common_engine.py	2025-08-28 09:52:36 +08:00
Sunny-bot1	479c8b85d3	[Optimize]support machete weight only gemm (#3561) ... * support machete weight only gemm * add generate * update * fix * change file location * add sm_version limit * fix * fix * fix ci * fix coverage * fix xpu	2025-08-28 09:49:58 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
Jiang-Jia-Jun	c694fa2879	Revert "[Feature] block sparse attention (#3209)" (#3647) ... This reverts commit 646a0c2fd8.	2025-08-27 17:35:04 +08:00
lzy	1265f6c192	deepgemm don't support tp+ep (for ci) (#3638) ... * deepgemm don't support tp+ep (for ci) * deepgemm don't support tp+ep (for ci)	2025-08-27 16:39:19 +08:00
xjkmfa	afb9f327ef	【CI case】for echo finish_reason text_after_process and raw_prediction check (#3630) ... * Add ci case for min token and max token * 【CI case】include total_tokens in the last packet of completion interface stream output * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check --------- Co-authored-by: xujing43 <xujing43@baidu.com>	2025-08-27 15:21:16 +08:00
YUNSHEN XIE	85afa72763	fix publish task (#3635) ... * fix publish task * disable ut	2025-08-27 11:14:53 +08:00
yangjianfengo1	646a0c2fd8	[Feature] block sparse attention (#3209) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config	2025-08-26 07:16:04 -07:00
RAM	f0a362af18	[CUDAGraph]Switch the scope so that output buffer of CUDAGraph can automatically release (#3612) ... * fix typo * fix typo * add print dot files * fix bug * Switch the scope so that output buffer of cudagraph can automatically release * Revert "add print dot files" This reverts commit dc21809eb5.	2025-08-26 21:28:19 +08:00
Yuanle Liu	cbce94a00e	rename ernie_xxx to ernie4_5_xxx (#3621) ... * rename ernie_xxx to ernie4_5_xxx * ci fix	2025-08-26 19:29:27 +08:00
YuanRisheng	642480f5f6	[CI] Standard unittest (#3606) ... * standard unittest * fix bugs * fix script	2025-08-26 19:03:11 +08:00
bukejiyu	3200a80de3	[v1 loader]support fp8 (#3593) ... * support fp8 * update ci	2025-08-26 02:42:46 -07:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
Ryan	a5b4866ff1	[CudaGraph][SOT] Add unit tests for splitting the static graph into piecewise graphs that support cuda_graph (#3590) ... * add unitest * change sot_warmup_sizes * wtf; add missed commit	2025-08-26 11:25:04 +08:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
lizexu123	c43a4bec00	[Features] support hugging face qwen3 dense and qwen2 model (#3574) ... * support qwen2 and qwen3 hugging face * fix moe * defualt_v1 loader * hugging_face_format deprecated * modify hugging_face_foramt to model_format * model_format auto * fix environemt * fix bug * fix qwen3-0.6 bug * model_format is str * fix	2025-08-26 10:54:53 +08:00
RAM	2fa173e327	[Executor] CUDAGraph support RL training (#3265) ... * add clear graph opt backend * cuda graph support rl * add branch * 1.fix dynamic_weight_manager bug 2.add clear api for CasualLM * open test case * fix typo * update mkdocs.yaml * [Docs]Update mkdocs.yml * update test case * use unittest in graph test case	2025-08-25 20:59:30 +08:00
chen	9cab3f47ff	[Feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing (#3552) ... * [feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing * infer engine support temp_scaled_logprobs and top_p_normalized_logprobs * delete some code * code check * code check and add doc * fix tokenizer.decoder(-1), return 'Invalid Token' * add ci for temp_scaled and top_p logprobs * check test * check seq len time shape * logprob clip inf --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com>	2025-08-25 14:11:49 +08:00
Yuan Xiaolan	9205c88da1	support w4afp8 EP inference (#3044)	2025-08-25 11:27:45 +08:00
YUNSHEN XIE	46664985fc	Modify the existing coverage collection method (#3573) ... fix cov report	2025-08-25 10:35:35 +08:00
YuBaoku	7821534ff5	[CI] add sot test (#3579) ... * [CI] add sot test * [CI] add sot test	2025-08-25 10:14:50 +08:00
bukejiyu	bdbac0aa3d	support qwen2 weight only (#3571)	2025-08-24 11:14:34 +08:00
bukejiyu	77514e3e1e	[V1 Loader] support weight_only (#3413) ... * support wint4/wint8 * delete smoe case * update ci * print log	2025-08-23 13:13:41 +08:00
Jiang-Jia-Jun	93e1b63200	Revert "[UnitTest][Copilot] Improve unit test coverage for entrypoints module…" (#3564) ... This reverts commit 36325e9ea7.	2025-08-23 10:44:23 +08:00
YUNSHEN XIE	cb166053ba	fix test name (#3493) ... * fix test name * update * update * fix * fix * update * update * update * update * update * fix * update	2025-08-22 23:43:47 +08:00
Copilot	36325e9ea7	[UnitTest][Copilot] Improve unit test coverage for entrypoints modules (#3546) ... * Initial plan * Add comprehensive unit tests for entrypoints utilities Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Complete entrypoints test coverage improvement with tool parser tests Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Apply pre-commit formatting to test files - fix trailing whitespace and long lines --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-22 19:20:51 +08:00
lddfym	27666ee586	[Feature] Add Qwen25-VL Processor (#3501) ... * add qwen-2.5-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor position_ids * add qwen25-vl processor * add qwen25-vl processor * position_ids * add test for qwen25-vl * organize comments * formatted * qwen_vl_processor * add qwen_vl_processor unittest * update model path * update model path * update qwen_vl_processor unittest * add unittest and bug fix * add unittest and bug fix * Update fastdeploy/input/qwen_mm_processor/image_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/input/qwen_vl_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-22 16:49:42 +08:00
freeliuzc	76759108c9	[Feature][SpeculativeDecoding]Support tree-attention (#3514) ... * support tree-attention * fix merge bug * fix unit-test api * fix merge bug	2025-08-22 13:36:41 +08:00
YuanRisheng	85fbf5455a	[V1 Loader]Ernie VL support loader v1 (#3494) ... * ernie vl support new loader * add unittest * fix test	2025-08-22 11:16:57 +08:00
Zhang Yulong	3cc182236a	update ci (#3519)	2025-08-21 20:05:50 +08:00
YuanRisheng	c389a4013c	Unify server-side and model-side Config(Part-5) (#3497) ... * move config * fix xpu * fix * fix vl * fix vl * fix unitest * fix args * add unitest * fix test	2025-08-21 19:00:21 +08:00
yangjianfengo1	e5aa7087db	【bug fix】修复w4a8编译慢 (#3510) ... * 修复w4a8编译 * code style * 修复tma copy	2025-08-21 18:50:14 +08:00
Zhang Yulong	a5692e8b7d	Add PD CI case (#3490) ... * Create test_ernie_03b_pd.py * Update test_ernie_03b_pd.py	2025-08-21 18:48:34 +08:00
李泳桦	e4f0b755b4	[fix] setting disable_chat_template while passing prompt_token_ids led to response error (#3228) ... * [fix] setting disable_chat_template while passing prompt_token_ids led to response error * [fix] code syntax * [test] add test case for this bug * [test] add test case for empty message list * [test] fix test case for empty message list	2025-08-21 17:30:51 +08:00
luukunn	371fb3f853	[Feature] add tool parser (#3483) ... * add tool parser * add x1 enable_thinking * restart ci * fix vl reasoning parser * modify call style * modify call style * add offline enablethinking * fix completion * fix * fix unit test * fix unit test * fix unit test * fix vl reasoning parser * fix vl reasoning parser	2025-08-21 17:25:44 +08:00
Yzc216	466cbb5a99	[Feature] Models api (#3073) ... * add v1/models interface related * add model parameters * default model verification * unit test * check model err_msg * unit test * type annotation * model parameter in response * modify document description * modify document description * unit test * verification * verification update * model_name * pre-commit * update test case * update test case * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/entrypoints/openai/test_serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/openai/serving_models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-21 17:02:56 +08:00
Zhang Yulong	b7eee3aec1	Update CI (#3474) ... * update CI cases * update CI cases * update CI cases * update CI cases * Merge upstream/develop and resolve directory rename conflict * Merge upstream/develop and resolve directory rename conflict * Merge upstream/develop and resolve directory rename conflict * update deploy * update deploy * update deploy * update deploy * update deploy	2025-08-21 16:49:20 +08:00
ltd0924	51f68ae593	[Feature] add dealer manager to reuse the connection (#3471) ... * [BugFix] fix control signal release failed * [BugFix] fix control signal release failed * update * update * update * [Feature] add dealer manager to reuse the connection * fix * fix * fix * fix * fix * fix * Create test_dealer_connection_manager.py * Delete test/entrypoints/openai directory * Update test_dealer_connection_manager.py * Update test_dealer_connection_manager.py	2025-08-21 13:11:13 +08:00
YUNSHEN XIE	3a6058e445	Add stable ci (#3460) ... * add stable ci * fix * update * fix * rename tests dir;fix stable ci bug * add timeout limit * update	2025-08-20 08:57:17 +08:00
jiangjiajun	684703fd72	[LLM] First commit the llm deployment code	2025-06-09 19:20:15 +08:00
DefTruth	ad9c70675e	[tests] Update cpp release tasks scripts (#1918) ... [tests] Update release tasks scripts	2023-05-09 17:14:06 +08:00
yeliang2258	81fbd54c9d	[Other] Add tests for TIMVX (#1605) ... * add tests for timvx * add mobilenetv1 test * update code * fix log info * update log * fix test --------- Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com>	2023-04-14 10:31:36 +08:00
DefTruth	ee85a3cade	[Benchmark] Add precision evaluation api from benchmark (#1310) ... * [Benchmark] Init benchmark precision api * [Benchmark] Init benchmark precision api * [Benchmark] Add benchmark precision api * [Benchmark] Calculate the statis of diff * [Benchmark] Calculate the statis of diff * [Benchmark] Calculate the statis of diff * [Benchmark] Calculate the statis of diff * [Benchmark] Calculate the statis of diff * [Benchmark] Add SplitDataLine utils * [Benchmark] Add LexSortByXY func * [Benchmark] Add LexSortByXY func * [Benchmark] Add LexSortDetectionResultByXY func * [Benchmark] Add LexSortDetectionResultByXY func * [Benchmark] Add tensor diff presicion test * [Benchmark] fixed conflicts * [Benchmark] fixed calc tensor diff * fixed build bugs * fixed ci bugs when WITH_TESTING=ON	2023-02-16 17:16:14 +08:00
yeliang2258	ef852579a9	[Bug Fix] Fix bugs in acc eval (#1276) ... fix bugs in acc eval	2023-02-09 10:04:04 +08:00
guxukai	1c115bb237	[Model] Add facedet model: CenterFace (#1131) ... * cpp example run success * add landmarks * fix reviewed problem * add pybind * add readme in examples * fix reviewed problem * new file: tests/models/test_centerface.py * fix reviewed problem 230202	2023-02-07 14:05:08 +08:00