32 Commits

Author	SHA1	Message	Date
Yuanle Liu	0cb9ad186e	[Cherry-Pick][BugFix] fix speculate_limit_thinking_content_length #5590 (#5615)	2025-12-18 01:50:18 -08:00
Yuanle Liu	1776d410d0	fix limit_thinking bug (#5469)	2025-12-10 11:56:35 +08:00
GoldPancake	8545b705ed	fix top_p_candidates (#5400) ... Co-authored-by: freeliuzc <lzc842650834@gmail.com>	2025-12-05 20:01:05 +08:00
GoldPancake	cfc5b0ccf9	[BugFix] fix mtp logprob bugs in chunk prefill (#5244) ... * fix mtp logprob bugs in chunk prefill * fix * fix	2025-11-27 11:31:29 +08:00
freeliuzc	f1e36ff2f7	[Speculative Decoding][MTP]Support stop_seqs and pd-split mode (#5029) ... * support multi_stop_seqs in speculative decoding * support mtp tp with ep split * fix custom op register * fix spec stop_seqs params	2025-11-20 15:26:01 +08:00
freeliuzc	11398790d3	[Speculative Decoding][MTP]Support attn mask offset (#4641) ... * [MTP]Merge support attn (#4591) * support mask_offset in speculate decoding * fix dummpy run output * add unit test * fix unit test import * support attn_mask_offset in mtp mode * add update_attn_mask op * fix unit test && fix code-style	2025-11-03 10:08:01 +08:00
freeliuzc	f44f4bafd1	support mtp in splitewise and scheduler_v1 mode (#4743)	2025-11-03 10:07:15 +08:00
Yuanle Liu	b301bd6c31	[BugFix] fix thinking bug (#4710) ... * fix thinking bug * fix ut * update * fix	2025-10-31 22:00:31 +08:00
GoldPancake	1f3ce65b58	[Feature] support mtp distribution equivalence verification (#4699)	2025-10-31 11:45:04 +08:00
Yuanle Liu	cef3164c3b	Optimizing the performance of think length limit using custom operators (#4279) ... * delete impl * delete min_length&max_length * support limit thinking content strategy * fix * fix * fix * update * fix set_value_by_flags_and_idx * fix * fix * fix * fix * update * fix * fix * fix typo * fix ci * fix * fix * support mtp * fix * fix * update * update	2025-10-20 21:09:13 +08:00
GoldPancake	47595a2480	[Feature] support mtp logprob (#4464) ... * support mtp logprob * fix unitest	2025-10-20 15:18:12 +08:00
freeliuzc	582aebd48b	[MTP]support mtp chunk_prefill_v1 (#4366) ... * support mtp chunk_prefill_v1 * fix mtp chunkprefill output, fix unit test * fix unit test * fix save_output	2025-10-15 13:21:32 +08:00
freeliuzc	365601ea5a	[MTP]support more branchs in topp kernel (#4352)	2025-10-11 11:33:52 +08:00
RAM	aa27b03bc0	[Executor]CUDAGraph support Speculate Decode (#3769) ... * success run ngram * Revert "[Code Simplification] remove cum_offsets (#3410)" This reverts commit 32b39620bc. * success run ngram5 tp4 42bs * success run ngram5 tp4 42bs * mtp draft commit * add decorator for target model * enable draft model in cudagraph v0.5 * revert revrt cum_offset * enable target model in cudagraph v0.9 And clean debug code * Revert "success run ngram" This reverts commit 8351e83993. * add reverted code * enable target model in cudagraph v0.9 * solve comment * fix bid < 0 * Enable Target Model Padding And Draft Model in cudagraph * solve problem * delete rebuild padding debug note * fast compile * Add capture list for mtp * success run 256 tp1 mtp * Enable Lite TP2 Bsz256 * realy enable tp2 bsz 256 * fix problem * Solve problem for Draft model in cudagraph * Solve comment * replace emptytensor as zeros * Solve comments * Revert "fast compile" This reverts commit 834639a7ff. * fix bug * fix merge bug * fix typo * fix bug --------- Co-authored-by: lizexu <2694294196@qq.com> Co-authored-by: littledgg <1658565283@qq.com> Co-authored-by: zeroRains <linjunlu@zerorains.top> Co-authored-by: gongshaotian <gstain5555@outlook.com>	2025-10-09 21:18:29 +08:00
co63oc	30a1c1783f	rename eagle_get_base_model_hidden_states.cu (#3753)	2025-09-07 10:24:58 +08:00
freeliuzc	88d44a2c93	support mtp in v1_scheduler mode (#3695)	2025-09-04 17:39:59 +08:00
lizexu123	4c998c3636	[Code Simplification] delete cum_offsets_out (#3815) ... * fix * fix	2025-09-03 16:15:33 +08:00
co63oc	5441538173	rename fused_get_rope.cu (#3752) ... * rename fused_get_rope.cu * fix * fix typos * fix * fix	2025-09-03 10:54:34 +08:00
co63oc	aa067a3106	rename speculate_token_penalty_multi_scores.cu (#3735)	2025-09-02 18:12:11 +08:00
co63oc	f296aff6cf	rename speculate_stop_generation_multi_stop_seqs (#3743)	2025-09-02 18:04:29 +08:00
co63oc	d6369b4d51	fix typos (#3684)	2025-09-01 17:50:17 +08:00
freeliuzc	52eda7fdb3	[Feature][MTP]support new speculative decoding method named hybrid mtp with ngram (#3610)	2025-08-26 14:29:22 +08:00
lizexu123	32b39620bc	[Code Simplification] remove cum_offsets (#3410)	2025-08-18 20:21:25 +08:00
freeliuzc	a12d0bc549	[Feature][MTP]update multi-draft-token strategy (#3369) ... * update multi-draft-token strategy * fix format --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-08-18 13:59:56 +08:00
Sunny-bot1	74aa31d15b	[Feature] support bad_words (#3055) ... * support bad_words * support online infer bad_words * update * add CI test * update * update * update --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>	2025-07-30 09:31:29 +08:00
Zero Rains	25698d56d1	polish code with new pre-commit rule (#2923)	2025-07-19 23:19:27 +08:00
周周周	1339e56282	[XPU] Remove padding_offsets from get_padding_offset.cu (#2911)	2025-07-18 14:16:44 +08:00
周周周	ddb10ac509	[Inference, rename] remove padding_offsets from atten use batch_id_per_token (#2880) ... * remove padding_offsets from atten	2025-07-17 18:41:31 +08:00
freeliuzc	7cdd8d290d	[MTP] optimize mtp infer speed (#2840)	2025-07-14 19:50:22 +08:00
GoldPancake	f7cad30a38	[Feature] Add speculative decoding simulation benchmark. (#2751) ... * Add speculative decoding simulation benchmark * Fix the name of the parameter	2025-07-09 12:08:43 +08:00
Jiang-Jia-Jun	92c2cfa2e7	Sync v2.0 version of code to github repo	2025-06-29 23:29:37 +00:00
jiangjiajun	684703fd72	[LLM] First commit the llm deployment code	2025-06-09 19:20:15 +08:00