apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-07 17:41:52 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
Files
67990e0572b45167bdb6858ea32c55fd94f05441
FastDeploy/custom_ops/gpu_ops
History
lizexu123 67990e0572
Some checks failed
Deploy GitHub Pages / deploy (push) Has been cancelled
Details
[Feature] support min_p_sampling (#2872) 
* Fastdeploy support min_p

* add test_min_p

* fix

* min_p_sampling

* update

* delete vl_gpu_model_runner.py

* fix

* Align usage of min_p with vLLM

* fix

* modified unit test

* fix test_min_sampling

* pre-commit all files

* fix

* fix

* fix

* fix xpu_model_runner.py
2025-07-20 23:17:59 -07:00
..
append_attn
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
common
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
custom_all_reduce
[Feature] support custom all-reduce (#2758)
2025-07-09 16:00:27 +08:00
cutlass_extensions
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
cutlass_kernels
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
fp8_gemm_with_cutlass
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
glog
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
int8_gemm_with_cutlass
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
mla_attn
[Attention] remove cum_offsets from atten, and use cu_seqlens_q (#2870)
2025-07-16 20:10:57 +08:00
moe
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
quantization
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
sample_kernels
[Feature] support min_p_sampling (#2872)
2025-07-20 23:17:59 -07:00
speculate_decoding
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
append_attention.cu
[Inference, rename] remove padding_offsets from atten use batch_id_per_token (#2880)
2025-07-17 18:41:31 +08:00
beam_search_softmax.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
cpp_extensions.cc
remove cum_offsets from get_block_shape_and_split_kv_block (#2913)
2025-07-18 16:13:32 +08:00
cuda_multiprocess.h
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
dequant_int8.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
enforce_generation.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
env.h
[Sync] Update to latest code (#2679)
2025-07-03 15:43:53 +08:00
extract_text_token_output.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
fused_get_rope.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
fused_hadamard_quant_fp8.cu
[Optimize] Optimize tensorwise fp8 performance (#2729)
2025-07-07 20:06:28 +08:00
fused_rotary_position_encoding.cu
[Sync] Update to latest code (#2679)
2025-07-03 15:43:53 +08:00
gather_idx.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_data_ptr_ipc.cu
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
get_mm_split_fuse.cc
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
get_output_ep.cc
[Feature][MTP] Support cacheKV transfer in per_chunk mode (#2890)
2025-07-17 17:58:08 +08:00
get_output_msg_with_topk.cc
[Feature] Online Chat API Support Return logprobs (#2777)
2025-07-10 16:33:40 +08:00
get_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_padding_offset_system.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
get_padding_offset.cu
[XPU] Remove padding_offsets from get_padding_offset.cu (#2911)
2025-07-18 14:16:44 +08:00
get_position_ids_and_mask_encoder_batch.cu
[Sync] Update to latest code (#2679)
2025-07-03 15:43:53 +08:00
helper.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
helper.h
dcu adapter ernie45t (#2756)
2025-07-09 18:56:27 +08:00
init_signal_layerwise.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
ipc_sent_key_value_cache_by_remote_ptr.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
msg_utils.h
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
multi_head_latent_attention.cu
[Attention] remove cum_offsets from atten, and use cu_seqlens_q (#2870)
2025-07-16 20:10:57 +08:00
ngram_mask.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
noaux_tc.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
noauxtc_kernel.h
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
open_shm_and_get_meta_signal.cc
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
per_token_quant_fp8.cu
[Sync] Update to latest code (#2679)
2025-07-03 15:43:53 +08:00
read_data_ipc.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
read_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
read_temp_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
rebuild_padding.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
remote_cache_kv_ipc.cc
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
remote_cache_kv_ipc.h
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
save_output_msg_with_topk.cc
[Feature] Online Chat API Support Return logprobs (#2777)
2025-07-10 16:33:40 +08:00
save_with_output_msg.cc
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
save_with_output_msg.h
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
save_with_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
scaled_gemm_f8_i4_f16_gemm.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
scaled_gemm_f8_i4_f16_weight_quantize.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
seqs2seqs.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_data_ipc.cu
dcu adapter ernie45t (#2756)
2025-07-09 18:56:27 +08:00
set_flags.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_mask_value.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
set_value_by_flags.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
share_external_data.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step_reschedule.cu
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
step_system_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
stop_generation_multi_ends.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
stop_generation_multi_stop_seqs.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
stop_generation.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
swap_cache_batch.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
swap_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
system2group.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
text_image_gather_scatter.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
text_image_index_out.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
token_penalty_multi_scores.cu
[Feature] support prompt repetition_penalty (#2806)
2025-07-17 12:05:52 +08:00
token_penalty_only_once.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
token_transfer.hpp
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
transfer_output.cc
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00
tune_cublaslt_gemm.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
update_inputs_beam.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
update_inputs.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
update_split_fuse_input.cu
[LLM] First commit the llm deployment code
2025-06-09 19:20:15 +08:00