apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[docs] add cli uasge to docs (#4569)

Browse Source
This commit is contained in:
xiaolei373
2025-10-28 10:35:11 +08:00
committed by GitHub
parent 6426414a0f
commit 0b196d82f3
17 changed files with 1282 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/zh/cli/run-batch.md Normal file
View File

@@ -0,0 +1,36 @@
# run-batch：批处理
## 说明
批量运行推理任务。支持从本地文件或远程 URL 读取输入请求，并将推理结果输出到文件或上传至远程 URL
## 用法
```
fastdeploy run-batch --model MODEL --input-file INPUT --output-file OUTPUT [参数]
```
## 参数
|参数|说明|默认值|
|-|-|-|
|-i, --input-file|单个输入文件的路径或 URL。目前支持本地文件路径或 HTTP/HTTPS 协议。如果是文件路径，文件里每行一个请求；如果指定了 URL，文件应可通过 HTTP GET 访问。|None|
|-o, --output-file|单个输出文件的路径或 URL。目前支持本地文件路径或网络 URL（HTTP/HTTPS）。如果是输出文件路径，每行一个响应；如果指定了 URL，文件应可通过 HTTP PUT 上传。|None|
|--output-tmp-dir|在将输出文件上传到输出 URL 之前，用于存放输出文件的临时目录。|None|
|--model|模型路径|None|
更多参数说明见：[FastDeploy 参数文档](../parameters.md)
## 示例
```
fastdeploy run-batch -i Input.json -o Output.json --model baidu/ERNIE-4.5-0.3B-Paddle
```
## 输入文件格式示例（Input.json）
```
{"custom_id": "req-00001", "method": "POST", "url": "/v1/chat/completions", "body": {"messages": [{"role": "user", "content": "Tell me a fun fact. (id=1)"}], "temperature": 0.7, "max_tokens": 50}}
{"custom_id": "req-00002", "method": "POST", "url": "/v1/chat/completions", "body": {"messages": [{"role": "user", "content": "What's the weather like today? (id=2)"}], "temperature": 0.7, "max_tokens": 50}}
```
## 输出文件格式示例（Output.json）
```
{"id":"fastdeploy-84601f40de3e48aeb3fe4d2ca328c32e","custom_id":"req-00001","response":{"status_code":200,"request_id":"fastdeploy-batch-0c18b71f5349453eaf00ae04659a21a0","body":{"id":"chatcmpl-024e9267-3d44-4594-91da-b5033c856da9","object":"chat.completion","created":1761203881,"model":"/root/PaddlePaddle/ERNIE-4.5-0.3B-Paddle","choices":[{"index":0,"message":{"role":"assistant","content":"Here's a fun fact about a funny animal:\n\n**The Elephant in the Room**\n\nElephants are known for their ability to inflate themselves with air pressure. Imagine a giant elephant standing upright, its trunk filling","multimodal_content":null,"reasoning_content":null,"tool_calls":null,"prompt_token_ids":null,"completion_token_ids":null,"prompt_tokens":null,"completion_tokens":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"total_tokens":68,"completion_tokens":50,"prompt_tokens_details":{"cached_tokens":0}}}},"error":null}
{"id":"fastdeploy-04cdfbd5b51e43408971b16be4439888","custom_id":"req-00002","response":{"status_code":200,"request_id":"fastdeploy-batch-dd7a9bebd2964acba6713c5dcb4b4aa6","body":{"id":"chatcmpl-452e9d0b-6c04-4b6f-9a2c-7d961f2dc605","object":"chat.completion","created":1761203881,"model":"/root/PaddlePaddle/ERNIE-4.5-0.3B-Paddle","choices":[{"index":0,"message":{"role":"assistant","content":"根据您提供的查询语句“What's the weather like today? (id=2)”，我需要先了解您想要查询的内容。请问您想查询的是某个特定天气状况（如晴天、下雨、阴天等）还是当前具体","multimodal_content":null,"reasoning_content":null,"tool_calls":null,"prompt_token_ids":null,"completion_token_ids":null,"prompt_tokens":null,"completion_tokens":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":20,"total_tokens":70,"completion_tokens":50,"prompt_tokens_details":{"cached_tokens":0}}}},"error":null}
```