GoldPancake f7cad30a38 [Feature] Add speculative decoding simulation benchmark. (#2751) ...

* Add speculative decoding simulation benchmark

* Fix the name of the parameter

2025-07-09 12:08:43 +08:00

..

__init__.py

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

args_utils.py

[LLM] support multi node deploy (#2708)

2025-07-06 10:33:51 +08:00

config.py

[Feature] Add speculative decoding simulation benchmark. (#2751)

2025-07-09 12:08:43 +08:00

engine.py

[Feature] Add speculative decoding simulation benchmark. (#2751)

2025-07-09 12:08:43 +08:00

expert_service.py

[Bug fix] Add the missing pod_ip param to the launch_cache_manager function. (#2742)

2025-07-08 14:52:13 +08:00

kv_cache_interface.py

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

request.py

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

resource_manager.py

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

sampling_params.py

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00