Yonghua Li cead6b26fa [Metrics] Update time_to_first_token to include tokenization & queue time, and remove redundant metrics (#4993) ...

* [update] update time_to_first_tokens to include queue time, and remove first_token_latency and infer_latency

* [doc] update docs

* [ci] fix test

* [chore] delete redundant code

---------

Co-authored-by: Jiaxin Sui <95567040+plusNew001@users.noreply.github.com>

2025-11-26 14:42:17 +08:00

..

sched

[Metrics] Update time_to_first_token to include tokenization & queue time, and remove redundant metrics (#4993)

2025-11-26 14:42:17 +08:00

__init__.py

polish code with new pre-commit rule (#2923)

2025-07-19 23:19:27 +08:00

args_utils.py

[Feature] enable guided decoding ENABLE_V1_KVCACHE_SCHEDULER = 1 (#5140)

2025-11-26 10:22:35 +08:00

async_llm.py

[Feature] support eplb in api_server (#4782)

2025-11-24 20:22:29 +08:00

common_engine.py

[Metrics] Update time_to_first_token to include tokenization & queue time, and remove redundant metrics (#4993)

2025-11-26 14:42:17 +08:00

engine.py

[Feature] support eplb in api_server (#4782)

2025-11-24 20:22:29 +08:00

expert_service.py

[Optimization] Improve perf for fd response token with internal adapter (#4992)

2025-11-21 19:02:03 +08:00

kv_cache_interface.py

bug: fix list to List (#4818)

2025-11-06 16:13:12 +08:00

pooling_params.py

[feature] support reward api (#4518)

2025-10-29 00:20:28 +08:00

request.py

[Metrics] Update time_to_first_token to include tokenization & queue time, and remove redundant metrics (#4993)

2025-11-26 14:42:17 +08:00

resource_manager.py

[fix] fix requests & block metrics (#4404)

2025-10-15 11:49:24 +08:00

sampling_params.py

[Logprobs]Support prompt_logprobs and max_logprobs (#4897)

2025-11-12 19:29:48 +08:00

tasks.py

[feature] support reward api (#4518)

2025-10-29 00:20:28 +08:00