apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

remove load default_v1 since already been as default (#4980)

Browse Source
This commit is contained in:
JYChen
2025-11-12 16:49:48 +08:00
committed by GitHub
parent c45b3ccb52
commit a1218076dc
14 changed files with 10 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/get_started/quick_start_qwen.md
View File

@@ -17,9 +17,6 @@ For more information about how to install FastDeploy, refer to the [installation
## 1. Launch Service
After installing FastDeploy, execute the following command in the terminal to start the service. For the configuration method of the startup command, refer to [Parameter Description](../parameters.md)
> ⚠️ **Note:**
> When using HuggingFace models (torch format), you need to enable `--load-choices "default_v1"`.
```
export ENABLE_V1_KVCACHE_SCHEDULER=1
python -m fastdeploy.entrypoints.openai.api_server \
@@ -28,8 +25,7 @@ python -m fastdeploy.entrypoints.openai.api_server \
--metrics-port 8181 \
--engine-worker-queue-port 8182 \
--max-model-len 32768 \
--max-num-seqs 32 \
--load-choices "default_v1"
--max-num-seqs 32
```
> 💡 Note: In the path specified by ```--model```, if the subdirectory corresponding to the path does not exist in the current directory, it will try to query whether AIStudio has a preset model based on the specified model name (such as ```Qwen/QWEN3-0.6b```). If it exists, it will automatically start downloading. The default download path is: ```~/xx```. For instructions and configuration on automatic model download, see [Model Download](../supported_models.md).