[metrics] Add serveral observability metrics (#3868)

* Add several observability metrics

* [wenxin-tools-584] 【可观测性】支持查看本节点的并发数、剩余block_size、排队请求数等信息

* adjust some metrics and md files

* trigger ci

* adjust ci file

* trigger ci

* trigger ci

---------

Co-authored-by: K11OntheBoat <your_email@example.com>
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
This commit is contained in:
qwes5s5
2025-09-08 14:13:13 +08:00
committed by GitHub
parent 3d0aaa5923
commit 17169a14f2
10 changed files with 244 additions and 6 deletions

View File

@@ -39,6 +39,7 @@ from fastdeploy.engine.expert_service import start_data_parallel_service
from fastdeploy.engine.request import Request
from fastdeploy.input.preprocess import InputPreprocessor
from fastdeploy.inter_communicator import EngineWorkerQueue, IPCSignal
from fastdeploy.metrics.metrics import main_process_metrics
from fastdeploy.utils import EngineError, console_logger, envs, llm_logger
@@ -101,6 +102,8 @@ class LLMEngine:
self.do_profile = 0
self._finalizer = weakref.finalize(self, self._exit_sub_services)
main_process_metrics.set_cache_config_info(obj=self.cfg.cache_config)
def start(self, api_server_pid=None):
"""
Initializes the engine and starts its sub-services.