apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

polish code with new pre-commit rule (#2923)

Browse Source
This commit is contained in:
Zero Rains
2025-07-19 23:19:27 +08:00
committed by GitHub
parent b8676d71a8
commit 25698d56d1
424 changed files with 14307 additions and 13518 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/zh/features/prefix_caching.md
View File

@@ -8,7 +8,6 @@ Prefix Caching（前缀缓存）是一种优化生成式模型推理效率的技
增量计算：对于后续请求，只需计算新增部分（如用户追加的输入）并复用缓存的中间结果，显著减少计算量。
## 服务化部署开启 Prefix Caching
启动服务增加以下参数 `enable-prefix-caching`，默认只开启一级缓存（GPU 缓存）。
@@ -37,4 +36,4 @@ python -m fastdeploy.entrypoints.openai.api_server \
FastDeploy 启动时设置 `enable_prefix_caching=True`，CPU Cache 根据机器内存选择开启 `swap_space`。
提供了测试示例 `demo/offline_prefix_caching_demo.py`。
提供了测试示例 `demo/offline_prefix_caching_demo.py`。