FastDeploy

append_attn

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

common

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

custom_all_reduce

Add custom op declaration for all_reduce (#3473 )

2025-08-20 20:29:58 +08:00

cutlass_extensions

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

cutlass_kernels

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

flash_mask_attn

[New Feature] fa3 支持flash mask (#3184 )

2025-08-05 12:20:48 +08:00

fp8_gemm_with_cutlass

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

glog

…

int8_gemm_with_cutlass

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

machete

[Optimize]support machete weight only gemm (#3561 )

2025-08-28 09:49:58 +08:00

mla_attn

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

moba_attn

fix typos (#3951 )

2025-09-08 15:22:41 +08:00

moe

[Feat] ernie4_5_vl_moe support CudaGraph (#3226 )

2025-09-10 13:11:57 +08:00

quantization

…

sample_kernels

[OPs] Universal optimization and Fix early_stop cuda 700 (#3375 )

2025-08-14 22:40:44 +08:00

speculate_decoding

rename eagle_get_base_model_hidden_states.cu (#3753 )

2025-09-07 10:24:58 +08:00

w4afp8_gemm

【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (#3771 )

2025-09-02 19:17:01 +08:00

wfp8afp8_sparse_gemm

【New Feature】支持Fp8 group Gemm 24稀疏 (#3463 )

2025-08-19 02:54:47 -07:00

append_attention.cu

supports dynamic Cfp8 (#3767 )

2025-09-07 20:41:29 -07:00

beam_search_softmax.cu

…

cpp_extensions.cc

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

cuda_multiprocess.h

…

dequant_int8.cu

…

enforce_generation.cu

…

env.h

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

extract_text_token_output.cu

[Code Simplification] Refactor Post-processing in VL Model Forward Method (#2937 )

2025-08-01 17:28:07 +08:00

fused_get_rotary_embedding.cu

rename fused_get_rope.cu (#3752 )

2025-09-03 10:54:34 +08:00

fused_hadamard_quant_fp8.cu

…

fused_rotary_position_encoding.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

gather_idx.cu

…

get_data_ptr_ipc.cu

…

get_img_boundaries.cc

[Feature] Multimodal Scheduler V1 (#3019 )

2025-07-30 16:05:55 +08:00

get_mm_split_fuse.cc

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

get_output_ep.cc

[OPs] Universal optimization and Fix early_stop cuda 700 (#3375 )

2025-08-14 22:40:44 +08:00

get_output_msg_with_topk.cc

…

get_output.cc

…

get_padding_offset_system.cu

…

get_padding_offset.cu

[Feat] ernie4_5_vl_moe support CudaGraph (#3226 )

2025-09-10 13:11:57 +08:00

get_position_ids_and_mask_encoder_batch.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

helper.cu

…

helper.h

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

init_signal_layerwise.cc

…

ipc_sent_key_value_cache_by_remote_ptr.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

merge_prefill_decode_output.cu

集中式支持fa3 (#3112 )

2025-08-01 18:03:36 +08:00

msg_utils.h

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

multi_head_latent_attention.cu

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

ngram_mask.cu

…

noaux_tc.cu

[Bugfix] Fix model accuracy in some ops (#3231 )

2025-08-08 17:30:37 +08:00

noauxtc_kernel.h

revise get_moe_scores (#3164 )

2025-08-05 16:43:07 +08:00

open_shm_and_get_meta_signal.cc

…

per_token_quant_fp8.cu

Support 45t fp8 8 GPU (#3659 )

2025-08-28 10:52:53 +08:00

read_data_ipc.cu

…

read_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

read_temp_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

rebuild_padding.cu

[Code Simplification] remove cum_offsets (#3410 )

2025-08-18 20:21:25 +08:00

recover_decode_task.cu

support mtp in v1_scheduler mode (#3695 )

2025-09-04 17:39:59 +08:00

remote_cache_kv_ipc.cc

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

remote_cache_kv_ipc.h

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

save_output_msg_with_topk.cc

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

save_with_output_msg.cc

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

save_with_output_msg.h

…

save_with_output.cc

…

scaled_gemm_f8_i4_f16_gemm.cu

…

scaled_gemm_f8_i4_f16_weight_quantize.cu

fix scaled_gemm_f8_i4_f16_weight_quantize input (#3685 )

2025-08-29 11:04:04 +08:00

seqs2seqs.cu

…

set_data_ipc.cu

…

set_flags.cu

…

set_mask_value.cu

…

set_value_by_flags_and_idx.cu

rename fused_get_rope.cu (#3752 )

2025-09-03 10:54:34 +08:00

share_external_data.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step_reschedule.cu

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

step_system_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step.cu

…

stop_generation_multi_ends.cu

[stop sequence] support stop sequence (#3025 )

2025-07-29 14:17:37 +08:00

stop_generation.cu

…

swap_cache_batch.cu

…

swap_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

system2group.cu

…

text_image_gather_scatter.cu

[Feat] ernie4_5_vl_moe support CudaGraph (#3226 )

2025-09-10 13:11:57 +08:00

text_image_index_out.cu

[Feat] ernie4_5_vl_moe support CudaGraph (#3226 )

2025-09-10 13:11:57 +08:00

token_penalty_multi_scores.cu

[Feature] support bad_words (#3055 )

2025-07-30 09:31:29 +08:00

token_penalty_only_once.cu

…

token_transfer.hpp

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

transfer_output.cc

…

tune_cublaslt_gemm.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

unset_data_ipc.cu

[feat] support clearing prefix cache (cherry-picked from release/2.1)

2025-09-12 21:41:55 +08:00

update_inputs_beam.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

update_inputs_v1.cu

[Feature] Support block scheduler v1 for FD (#2928 )

2025-07-23 20:31:31 +08:00

update_inputs.cu

…

update_split_fuse_input.cu

fix typos (#3684 )

2025-09-01 17:50:17 +08:00