144 Commits

Author	SHA1	Message	Date
bukejiyu	051e4a881c	ignore (#3949)	2025-09-07 23:57:48 +08:00
chenjian	41cd3e24c9	[Feature] Enable prefix caching as default (#3816) ... * [Feature] Enable prefix caching as default * [Feature] Enable prefix caching as default * Set prefix caching as default * skip dynamic load * fix kill bug * fix kill bug * fix kill bug * fix ci * fix --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-06 09:51:34 +08:00
Zhang Yulong	11b18e5ef0	add cache queue port (#3904) (#3926) ... * add cache queue port * add cache queue port * add cache queue port	2025-09-06 00:00:12 +08:00
chenjian	fb1e0d6a87	[Feature] Set scheduler v1 as default (#3812) ... * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default	2025-09-04 11:02:10 +08:00
SunLei	8c0e7d6fe9	Support for async processor added. (#3870) ... * Support for async processor added. * remove yappi code	2025-09-04 10:35:08 +08:00
yangjianfengo1	b56b015d85	fix port (#3865) ... Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-09-04 10:02:08 +08:00
lizhenyun01	bed09ae8f8	fix mask_offset in append_attn (#3745) ... * fix mask_offset in append_attn * fix test	2025-08-31 15:03:16 +08:00
SunLei	b9af95cf1c	[Feature] Add AsyncTokenizerClient&ChatResponseProcessor with remote encode&decode support. (#3674) ... * [Feature] add AsyncTokenizerClient * add decode_image * Add response_processors with remote decode support. * [Feature] add tokenizer_base_url startup argument * Revert comment removal and restore original content. * [Feature] Non-streaming requests now support remote image decoding. * Fix parameter type issue in decode_image call. * Keep completion_token_ids when return_token_ids = False. * add copyright	2025-08-30 17:06:26 +08:00
lizexu123	b21e085f3e	[Code Simplification] delete print (#3729)	2025-08-30 16:19:07 +08:00
lizexu123	455205f991	[Features] support hugging face qwen3 moe (#3649) ... * split ut * qwen3-30B-A3B * fix * add test * add test_torch_model.py * fix test_torch_model.py * delete print * fix moe * delete init.py * fix * fix --------- Co-authored-by: bukejiyu <395822456@qq.com> Co-authored-by: bukejiyu <52310069+bukejiyu@users.noreply.github.com>	2025-08-30 15:26:05 +08:00
YUNSHEN XIE	a18afcfdd9	Optimize coverage jobs (#3683)	2025-08-30 00:12:40 +08:00
yangjianfengo1	3754a9906d	[Feature] block sparse attention (#3668) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config * 修复显存 * add test server * add test server * fix mlp 最后一层使用full attn	2025-08-29 19:46:30 +08:00
Yuan Xiaolan	c71ee0831c	add w4afp8 offline script (#3636)	2025-08-29 17:56:05 +08:00
Ryan	45f81b34f0	add dtype int32 (#3692)	2025-08-29 14:56:35 +08:00
李泳桦	88297240e7	[feat] completion api supports passing input token ids in either prompt or prompt_token_ids (#3311) ... * [feat] completion api supports passing input token ids in either `prompt` or `prompt_token_ids` * [fix] update comment * [fix] fix type error * [test] add a unittest file for serving api test * [test] try to fix ci error * [chore] rename test function names * [test] try to fix ci error * [test] try to fix ci error * [test] add tests for qwen	2025-08-29 14:19:42 +08:00
周周周	17b414c2df	MoE Default use triton's blockwise fp8 in TP Case (#3678)	2025-08-29 11:07:30 +08:00
Echo-Nie	43d5bd62b4	【Hackathon 9th No.70】supplementary unit test for CPUPlatform and CUDAPlatform (#3580) ... * 功能模块 CUDAPlatform、CPUPlatform 单测补充 * update the "is_cuda" to "is_cuda_and_available" * fix pre-commit --------- Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-08-29 10:34:05 +08:00
Yuanle Liu	4957908275	add input_processor plugin (#3657) ... * add input_processor plugin * update * update * update * update * update * update * update * update * update * update * update	2025-08-28 22:53:57 +08:00
Divano	17731a8acd	add concurrency cases (#3689)	2025-08-28 18:30:19 +08:00
Liumengyuan	e93d4cfcdd	Add with_output version AppendAttention (#3302) ... * get use_output from fd_config * add clear TODO description * add mask_offset para to align with develop * fix bug * fix use_output logic * fix sot bug	2025-08-28 17:10:18 +08:00
bukejiyu	73cf6096da	fix (#3676) ... * fix * update	2025-08-28 17:06:32 +08:00
co63oc	d4fc893fe3	fix typos (#3633) ... Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-28 14:42:24 +08:00
Echo-Nie	7afcd4b776	【Hackathon 9th No.77】supplementary unit test for get_filtered_metrics (#3578) ... * 功能模块 fastdeploy/metrics/metrics/get_filtered_metrics 单测补充 * fix pre-commit --------- Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-08-28 10:39:02 +08:00
ltd0924	3d92fb09f7	[BugFix] fix parameter is 0 (#3592) ... * Update engine_client.py * fix * Update common_engine.py	2025-08-28 09:52:36 +08:00
Sunny-bot1	479c8b85d3	[Optimize]support machete weight only gemm (#3561) ... * support machete weight only gemm * add generate * update * fix * change file location * add sm_version limit * fix * fix * fix ci * fix coverage * fix xpu	2025-08-28 09:49:58 +08:00
Zero Rains	e37e86b3b8	[V1 Loader]support param create and load for wint2 and xpu backend (#3581) ... * support wint2 backend' * [V1 Loader]support param create and load for wint2 and xpu backend * update weight shape name * update * update * update baseline.txt * update model name * update baseline.txt * fix codestyle * remove debug coode	2025-08-28 09:49:36 +08:00
Jiang-Jia-Jun	c694fa2879	Revert "[Feature] block sparse attention (#3209)" (#3647) ... This reverts commit 646a0c2fd8.	2025-08-27 17:35:04 +08:00
lzy	1265f6c192	deepgemm don't support tp+ep (for ci) (#3638) ... * deepgemm don't support tp+ep (for ci) * deepgemm don't support tp+ep (for ci)	2025-08-27 16:39:19 +08:00
xjkmfa	afb9f327ef	【CI case】for echo finish_reason text_after_process and raw_prediction check (#3630) ... * Add ci case for min token and max token * 【CI case】include total_tokens in the last packet of completion interface stream output * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check * echo&finish_reason&text_after_process&raw_prediction check --------- Co-authored-by: xujing43 <xujing43@baidu.com>	2025-08-27 15:21:16 +08:00
YUNSHEN XIE	85afa72763	fix publish task (#3635) ... * fix publish task * disable ut	2025-08-27 11:14:53 +08:00
yangjianfengo1	646a0c2fd8	[Feature] block sparse attention (#3209) ... * 支持稀疏attn * fix bug * code style * fix moba attn get kv shape * 修复a100编译 * codestyle * code style * code style * code style * fix conflict * 增加单侧 * code style * 增加eblite 加载时间 * fix bug * for ci * for ci * for ci * for ci * 支持mlp block size 128 * 增加小算子单测 * fix 单测 mlp * 将环境变量加入到config里面 * fix rollout config	2025-08-26 07:16:04 -07:00
RAM	f0a362af18	[CUDAGraph]Switch the scope so that output buffer of CUDAGraph can automatically release (#3612) ... * fix typo * fix typo * add print dot files * fix bug * Switch the scope so that output buffer of cudagraph can automatically release * Revert "add print dot files" This reverts commit dc21809eb5.	2025-08-26 21:28:19 +08:00
Yuanle Liu	cbce94a00e	rename ernie_xxx to ernie4_5_xxx (#3621) ... * rename ernie_xxx to ernie4_5_xxx * ci fix	2025-08-26 19:29:27 +08:00
YuanRisheng	642480f5f6	[CI] Standard unittest (#3606) ... * standard unittest * fix bugs * fix script	2025-08-26 19:03:11 +08:00
bukejiyu	3200a80de3	[v1 loader]support fp8 (#3593) ... * support fp8 * update ci	2025-08-26 02:42:46 -07:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
Ryan	a5b4866ff1	[CudaGraph][SOT] Add unit tests for splitting the static graph into piecewise graphs that support cuda_graph (#3590) ... * add unitest * change sot_warmup_sizes * wtf; add missed commit	2025-08-26 11:25:04 +08:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
lizexu123	c43a4bec00	[Features] support hugging face qwen3 dense and qwen2 model (#3574) ... * support qwen2 and qwen3 hugging face * fix moe * defualt_v1 loader * hugging_face_format deprecated * modify hugging_face_foramt to model_format * model_format auto * fix environemt * fix bug * fix qwen3-0.6 bug * model_format is str * fix	2025-08-26 10:54:53 +08:00
RAM	2fa173e327	[Executor] CUDAGraph support RL training (#3265) ... * add clear graph opt backend * cuda graph support rl * add branch * 1.fix dynamic_weight_manager bug 2.add clear api for CasualLM * open test case * fix typo * update mkdocs.yaml * [Docs]Update mkdocs.yml * update test case * use unittest in graph test case	2025-08-25 20:59:30 +08:00
chen	9cab3f47ff	[Feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing (#3552) ... * [feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing * infer engine support temp_scaled_logprobs and top_p_normalized_logprobs * delete some code * code check * code check and add doc * fix tokenizer.decoder(-1), return 'Invalid Token' * add ci for temp_scaled and top_p logprobs * check test * check seq len time shape * logprob clip inf --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com>	2025-08-25 14:11:49 +08:00
Yuan Xiaolan	9205c88da1	support w4afp8 EP inference (#3044)	2025-08-25 11:27:45 +08:00
YUNSHEN XIE	46664985fc	Modify the existing coverage collection method (#3573) ... fix cov report	2025-08-25 10:35:35 +08:00
YuBaoku	7821534ff5	[CI] add sot test (#3579) ... * [CI] add sot test * [CI] add sot test	2025-08-25 10:14:50 +08:00
bukejiyu	bdbac0aa3d	support qwen2 weight only (#3571)	2025-08-24 11:14:34 +08:00
bukejiyu	77514e3e1e	[V1 Loader] support weight_only (#3413) ... * support wint4/wint8 * delete smoe case * update ci * print log	2025-08-23 13:13:41 +08:00
Jiang-Jia-Jun	93e1b63200	Revert "[UnitTest][Copilot] Improve unit test coverage for entrypoints module…" (#3564) ... This reverts commit 36325e9ea7.	2025-08-23 10:44:23 +08:00
YUNSHEN XIE	cb166053ba	fix test name (#3493) ... * fix test name * update * update * fix * fix * update * update * update * update * update * fix * update	2025-08-22 23:43:47 +08:00
Copilot	36325e9ea7	[UnitTest][Copilot] Improve unit test coverage for entrypoints modules (#3546) ... * Initial plan * Add comprehensive unit tests for entrypoints utilities Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Complete entrypoints test coverage improvement with tool parser tests Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Apply pre-commit formatting to test files - fix trailing whitespace and long lines --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-08-22 19:20:51 +08:00
lddfym	27666ee586	[Feature] Add Qwen25-VL Processor (#3501) ... * add qwen-2.5-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor * add qwen25-vl processor position_ids * add qwen25-vl processor * add qwen25-vl processor * position_ids * add test for qwen25-vl * organize comments * formatted * qwen_vl_processor * add qwen_vl_processor unittest * update model path * update model path * update qwen_vl_processor unittest * add unittest and bug fix * add unittest and bug fix * Update fastdeploy/input/qwen_mm_processor/image_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/input/qwen_vl_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-22 16:49:42 +08:00