qw86972190 b52971749c Print KV Cache available memory and block memory usage in GB format (#4148)

2025-09-17 20:01:55 +08:00

..

__init__.py

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

dcu_model_runner.py

enable dcu ci (#3402)

2025-08-29 10:23:08 +08:00

dcu_worker.py

fix typos (#3840)

2025-09-12 11:04:38 +08:00

eplb.py

polish code with new pre-commit rule (#2923)

2025-07-19 23:19:27 +08:00

experts_manager.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

gcu_model_runner.py

[FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116)

2025-09-17 10:43:35 +08:00

gcu_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

gpu_model_runner.py

Reconstruct streaming data transfer with zmq (#3836)

2025-09-17 14:30:39 +08:00

gpu_worker.py

[BugFix] Fix the abnormal memory usage caused by shape errors in the triton moe backend (#4026)

2025-09-09 20:05:54 -07:00

iluvatar_model_runner.py

[Iluvatar GPU] Optimze attention and moe performance (#3234)

2025-08-08 10:51:24 +08:00

iluvatar_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

metax_model_runner.py

[FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116)

2025-09-17 10:43:35 +08:00

metax_worker.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

model_runner_base.py

[FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116)

2025-09-17 10:43:35 +08:00

output.py

[Feature] General support for logprobs (#2974)

2025-07-31 20:25:56 +08:00

utils.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

worker_base.py

fix typos (#3684)

2025-09-01 17:50:17 +08:00

worker_process.py

Print KV Cache available memory and block memory usage in GB format (#4148)

2025-09-17 20:01:55 +08:00

xpu_model_runner.py

[FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116)

2025-09-17 10:43:35 +08:00

xpu_worker.py

[xpu] support ep (#4067)

2025-09-15 13:53:11 +08:00