FastDeploy

append_attn

[Excutor] Fixed the issue of CUDA graph execution failure caused by different branches during decoding (#3223 )

2025-08-09 07:37:19 +08:00

common

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

custom_all_reduce

[Feature] support custom all-reduce (#2758 )

2025-07-09 16:00:27 +08:00

cutlass_extensions

Optimize the performance of moe_expert_ffn_wint2 (#2990 )

2025-07-28 10:32:43 +08:00

cutlass_kernels

optimize w4a8 decoding (#3050 )

2025-07-28 22:20:13 +08:00

flash_mask_attn

[New Feature] fa3 支持flash mask (#3184 )

2025-08-05 12:20:48 +08:00

fp8_gemm_with_cutlass

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

glog

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

int8_gemm_with_cutlass

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

mla_attn

[BugFix] Rename attention params of deepseekv3 (#2939 )

2025-07-22 14:01:30 +08:00

moe

Refactor moe_topk_select op to use apply_norm_weight as a template parameter (#3345 )

2025-08-13 08:44:16 +08:00

quantization

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

sample_kernels

[Iluvatar GPU] Optimze attention and moe performance (#3234 )

2025-08-08 10:51:24 +08:00

speculate_decoding

[Feature] support bad_words (#3055 )

2025-07-30 09:31:29 +08:00

w4afp8_gemm

[New Feature] Support W4Afp8 MoE GroupGemm (#3171 )

2025-08-06 10:34:05 +08:00

append_attention.cu

support qk norm (#3145 )

2025-08-05 16:46:14 +08:00

beam_search_softmax.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

cpp_extensions.cc

support qk norm (#3145 )

2025-08-05 16:46:14 +08:00

cuda_multiprocess.h

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

dequant_int8.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

enforce_generation.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

env.h

[Sync] Update to latest code (#2679 )

2025-07-03 15:43:53 +08:00

extract_text_token_output.cu

[Code Simplification] Refactor Post-processing in VL Model Forward Method (#2937 )

2025-08-01 17:28:07 +08:00

fused_get_rope.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

fused_hadamard_quant_fp8.cu

[Optimize] Optimize tensorwise fp8 performance (#2729 )

2025-07-07 20:06:28 +08:00

fused_rotary_position_encoding.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

gather_idx.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

get_data_ptr_ipc.cu

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

get_img_boundaries.cc

[Feature] Multimodal Scheduler V1 (#3019 )

2025-07-30 16:05:55 +08:00

get_mm_split_fuse.cc

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

get_output_ep.cc

[Feature][MTP] Support cacheKV transfer in per_chunk mode (#2890 )

2025-07-17 17:58:08 +08:00

get_output_msg_with_topk.cc

[Feature] Online Chat API Support Return logprobs (#2777 )

2025-07-10 16:33:40 +08:00

get_output.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

get_padding_offset_system.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

get_padding_offset.cu

[XPU] Remove padding_offsets from get_padding_offset.cu (#2911 )

2025-07-18 14:16:44 +08:00

get_position_ids_and_mask_encoder_batch.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

helper.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

helper.h

dcu adapter ernie45t (#2756 )

2025-07-09 18:56:27 +08:00

init_signal_layerwise.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

ipc_sent_key_value_cache_by_remote_ptr.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

merge_prefill_decode_output.cu

集中式支持fa3 (#3112 )

2025-08-01 18:03:36 +08:00

msg_utils.h

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

multi_head_latent_attention.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

ngram_mask.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

noaux_tc.cu

[Bugfix] Fix model accuracy in some ops (#3231 )

2025-08-08 17:30:37 +08:00

noauxtc_kernel.h

revise get_moe_scores (#3164 )

2025-08-05 16:43:07 +08:00

open_shm_and_get_meta_signal.cc

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

per_token_quant_fp8.cu

Add switch to apply fine-grained per token quant fp8 (#3192 )

2025-08-04 19:54:03 -07:00

read_data_ipc.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

read_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

read_temp_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

rebuild_padding.cu

Adapt for iluvatar gpu (#2684 )

2025-07-07 16:53:14 +08:00

recover_decode_task.cu

[Feature] Support block scheduler v1 for FD (#2928 )

2025-07-23 20:31:31 +08:00

remote_cache_kv_ipc.cc

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

remote_cache_kv_ipc.h

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

save_output_msg_with_topk.cc

[Feature] Online Chat API Support Return logprobs (#2777 )

2025-07-10 16:33:40 +08:00

save_with_output_msg.cc

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

save_with_output_msg.h

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

save_with_output.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

scaled_gemm_f8_i4_f16_gemm.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

scaled_gemm_f8_i4_f16_weight_quantize.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

seqs2seqs.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_data_ipc.cu

dcu adapter ernie45t (#2756 )

2025-07-09 18:56:27 +08:00

set_flags.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_mask_value.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_value_by_flags.cu

Adapt for iluvatar gpu (#2684 )

2025-07-07 16:53:14 +08:00

share_external_data.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step_reschedule.cu

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

step_system_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step.cu

Adapt for iluvatar gpu (#2684 )

2025-07-07 16:53:14 +08:00

stop_generation_multi_ends.cu

[stop sequence] support stop sequence (#3025 )

2025-07-29 14:17:37 +08:00

stop_generation.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

swap_cache_batch.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

swap_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

system2group.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

text_image_gather_scatter.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

text_image_index_out.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

token_penalty_multi_scores.cu

[Feature] support bad_words (#3055 )

2025-07-30 09:31:29 +08:00

token_penalty_only_once.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

token_transfer.hpp

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

transfer_output.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

tune_cublaslt_gemm.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

update_inputs_beam.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

update_inputs_v1.cu

[Feature] Support block scheduler v1 for FD (#2928 )

2025-07-23 20:31:31 +08:00

update_inputs.cu

Adapt for iluvatar gpu (#2684 )

2025-07-07 16:53:14 +08:00

update_split_fuse_input.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00