AIbin a7392a0ff9 【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886) ...

* support MLA chunk_size auto search & cuda_graph

2025-09-11 10:46:09 +08:00

..

__init__.py

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

dcu_model_runner.py

enable dcu ci (#3402)

2025-08-29 10:23:08 +08:00

dcu_worker.py

fix dcu_worker.py (#3734)

2025-09-03 10:57:42 +08:00

eplb.py

polish code with new pre-commit rule (#2923)

2025-07-19 23:19:27 +08:00

experts_manager.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

gcu_model_runner.py

[Excutor] Experiment Feature-Support Prefill in cudagraph (#3459)

2025-09-08 13:12:24 +08:00

gcu_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

gpu_model_runner.py

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886)

2025-09-11 10:46:09 +08:00

gpu_worker.py

[BugFix] Fix the abnormal memory usage caused by shape errors in the triton moe backend (#4026)

2025-09-09 20:05:54 -07:00

iluvatar_model_runner.py

[Iluvatar GPU] Optimze attention and moe performance (#3234)

2025-08-08 10:51:24 +08:00

iluvatar_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

metax_model_runner.py

[V1 Loader]support param create and load for wint2 and xpu backend (#3581)

2025-08-28 09:49:36 +08:00

metax_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

model_runner_base.py

Unify server-side and model-side Config (Part1) (#3018)

2025-07-28 10:51:52 +08:00

output.py

[Feature] General support for logprobs (#2974)

2025-07-31 20:25:56 +08:00

utils.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

worker_base.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

worker_process.py

[V1 Loader] Ernie kv cache quant support v1 loader (#3899)

2025-09-09 05:25:08 -07:00

xpu_model_runner.py

[XPU]Fixed the issue of performance degradation caused by enabling ENABLE_V1_KVCACHE_SCHEDULER (#3897)

2025-09-08 10:34:46 +08:00

xpu_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00