apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
Files
aa27b03bc00af15f522781df99c92b4fd0019b17
FastDeploy/custom_ops/gpu_ops
History
RAM aa27b03bc0
Some checks failed
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details
[Executor]CUDAGraph support Speculate Decode (#3769) 
* success run ngram

* Revert "[Code Simplification] remove cum_offsets (#3410)"

This reverts commit 32b39620bc.

* success run ngram5 tp4 42bs

* success run ngram5 tp4 42bs

* mtp draft commit

* add decorator for target model

* enable draft model in cudagraph v0.5

* revert revrt cum_offset

* enable target model in cudagraph v0.9 And clean debug code

* Revert "success run ngram"

This reverts commit 8351e83993.

* add reverted code

* enable target model in cudagraph v0.9

* solve comment

* fix bid < 0

* Enable Target Model Padding And Draft Model in cudagraph

* solve problem

* delete rebuild padding debug note

* fast compile

* Add capture list for mtp

* success run 256 tp1 mtp

* Enable Lite TP2 Bsz256

* realy enable tp2 bsz 256

* fix problem

* Solve problem for Draft model in cudagraph

* Solve comment

* replace emptytensor as zeros

* Solve comments

* Revert "fast compile"

This reverts commit 834639a7ff.

* fix bug

* fix merge bug

* fix typo

* fix bug

---------

Co-authored-by: lizexu <2694294196@qq.com>
Co-authored-by: littledgg <1658565283@qq.com>
Co-authored-by: zeroRains <linjunlu@zerorains.top>
Co-authored-by: gongshaotian <gstain5555@outlook.com>
2025-10-09 21:18:29 +08:00
..
append_attn
[Executor]CUDAGraph support Speculate Decode (#3769)
2025-10-09 21:18:29 +08:00
common
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
custom_all_reduce
[NewFeature]custom_allreduce support cudagraph recapture (#4305)
2025-09-29 15:56:54 +08:00
cutlass_extensions
fix typos (#3684)
2025-09-01 17:50:17 +08:00
cutlass_kernels
fix typos (#3684)
2025-09-01 17:50:17 +08:00
flash_mask_attn
[New Feature] fa3 支持flash mask (#3184)
2025-08-05 12:20:48 +08:00
fp8_gemm_with_cutlass
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
glog
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
int8_gemm_with_cutlass
fix typos (#3684)
2025-09-01 17:50:17 +08:00
machete
[Optimize] Support WINT8 and group scale for Machete (#3905)
2025-09-15 12:01:34 +08:00
mla_attn
【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886)
2025-09-11 10:46:09 +08:00
moba_attn
fix typos (#3951)
2025-09-08 15:22:41 +08:00
moe
[NewFeture]add ep rollout model init and update/clear ep buffer (#4039)
2025-09-17 20:24:53 +08:00
quantization
[OPs] MoE support wfp8afp8(channelwise) and improve per_token_quant_fp8 (#4238)
2025-09-24 16:39:51 +08:00
sample_kernels
[OPs] Universal optimization and Fix early_stop cuda 700 (#3375)
2025-08-14 22:40:44 +08:00
speculate_decoding
[Executor]CUDAGraph support Speculate Decode (#3769)
2025-10-09 21:18:29 +08:00
w4afp8_gemm
【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (#3771)
2025-09-02 19:17:01 +08:00
wfp8afp8_sparse_gemm
【New Feature】支持Fp8 group Gemm 24稀疏 (#3463)
2025-08-19 02:54:47 -07:00
append_attention.cu
[Executor]CUDAGraph support Speculate Decode (#3769)
2025-10-09 21:18:29 +08:00
beam_search_softmax.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
cpp_extensions.cc
[Executor]CUDAGraph support Speculate Decode (#3769)
2025-10-09 21:18:29 +08:00
cuda_multiprocess.h
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
dequant_int8.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
enforce_generation.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
env.h
【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886)
2025-09-11 10:46:09 +08:00
extract_text_token_output.cu
[Code Simplification] Refactor Post-processing in VL Model Forward Method (#2937)
2025-08-01 17:28:07 +08:00
fused_get_rotary_embedding.cu
rename fused_get_rope.cu (#3752)
2025-09-03 10:54:34 +08:00
fused_hadamard_quant_fp8.cu
[Optimize] Optimize tensorwise fp8 performance (#2729)
2025-07-07 20:06:28 +08:00
fused_rotary_position_encoding.cu
[Feature] DeepseekV3 use pd_build_static_op (#2948)
2025-07-22 15:03:41 +08:00
gather_idx.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_data_ptr_ipc.cu
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
get_img_boundaries.cc
[Feature] Multimodal Scheduler V1 (#3019)
2025-07-30 16:05:55 +08:00
get_mm_split_fuse.cc
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
get_output_ep.cc
[OPs] Universal optimization and Fix early_stop cuda 700 (#3375)
2025-08-14 22:40:44 +08:00
get_output_msg_with_topk.cc
ep support logprob (#4089) (#4151)
2025-09-19 14:07:31 +08:00
get_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_padding_offset_system.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_padding_offset.cu
[Feat] ernie4_5_vl_moe support CudaGraph (#3226)
2025-09-10 13:11:57 +08:00
get_position_ids_and_mask_encoder_batch.cu
[Feature] DeepseekV3 use pd_build_static_op (#2948)
2025-07-22 15:03:41 +08:00
helper.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
helper.h
[Metax] support cutlass moe & optimize flash attention (#4208)
2025-09-29 11:22:43 +08:00
init_signal_layerwise.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
ipc_sent_key_value_cache_by_remote_ptr.cu
[BugFix]remove redundant includes (#4312)
2025-09-30 17:54:19 +08:00
merge_prefill_decode_output.cu
集中式支持fa3 (#3112)
2025-08-01 18:03:36 +08:00
msg_utils.h
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
multi_head_latent_attention.cu
【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886)
2025-09-11 10:46:09 +08:00
ngram_mask.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
noaux_tc.cu
Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)
2025-09-23 18:41:33 +08:00
noauxtc_kernel.h
Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)
2025-09-23 18:41:33 +08:00
open_shm_and_get_meta_signal.cc
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
per_token_quant_fp8.cu
Support 45t fp8 8 GPU (#3659)
2025-08-28 10:52:53 +08:00
read_data_ipc.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
read_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
read_temp_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
rebuild_padding.cu
[Executor]CUDAGraph support Speculate Decode (#3769)
2025-10-09 21:18:29 +08:00
recover_decode_task.cu
support mtp in v1_scheduler mode (#3695)
2025-09-04 17:39:59 +08:00
remote_cache_kv_ipc.cc
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
remote_cache_kv_ipc.h
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
save_output_msg_with_topk.cc
fix typos (#3684)
2025-09-01 17:50:17 +08:00
save_with_output_msg.cc
fix typos (#3684)
2025-09-01 17:50:17 +08:00
save_with_output_msg.h
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
save_with_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
scaled_gemm_f8_i4_f16_gemm.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
scaled_gemm_f8_i4_f16_weight_quantize.cu
fix scaled_gemm_f8_i4_f16_weight_quantize input (#3685)
2025-08-29 11:04:04 +08:00
seqs2seqs.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_data_ipc.cu
dcu adapter ernie45t (#2756)
2025-07-09 18:56:27 +08:00
set_flags.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_mask_value.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_value_by_flags_and_idx.cu
rename fused_get_rope.cu (#3752)
2025-09-03 10:54:34 +08:00
share_external_data.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step_reschedule.cu
fix typos (#3684)
2025-09-01 17:50:17 +08:00
step_system_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
stop_generation_multi_ends.cu
[stop sequence] support stop sequence (#3025)
2025-07-29 14:17:37 +08:00
stop_generation.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
swap_cache_batch.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
swap_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
system2group.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
text_image_gather_scatter.cu
[Feat] ernie4_5_vl_moe support CudaGraph (#3226)
2025-09-10 13:11:57 +08:00
text_image_index_out.cu
[Feat] ernie4_5_vl_moe support CudaGraph (#3226)
2025-09-10 13:11:57 +08:00
token_penalty_multi_scores.cu
[Feature] support bad_words (#3055)
2025-07-30 09:31:29 +08:00
token_penalty_only_once.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
token_transfer.hpp
fix typos (#3684)
2025-09-01 17:50:17 +08:00
transfer_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
tune_cublaslt_gemm.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
unset_data_ipc.cu
[feat] support prefix cache clearing when /clear_load_weight is called (#4008)
2025-09-28 19:42:53 +08:00
update_inputs_beam.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
update_inputs_v1.cu
[Feature] Support pd ep deployment with yiyan adapter (#4029)
2025-09-22 16:41:38 +08:00
update_inputs.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
update_split_fuse_input.cu
fix typos (#3684)
2025-09-01 17:50:17 +08:00