apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

remove load default_v1 since already been as default (#4980)

Browse Source
This commit is contained in:
JYChen
2025-11-12 16:49:48 +08:00
committed by GitHub
parent c45b3ccb52
commit a1218076dc
14 changed files with 10 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/supported_models.md
View File

@@ -15,8 +15,6 @@ export FD_MODEL_SOURCE=AISTUDIO # "AISTUDIO", "MODELSCOPE" or "HUGGINGFACE"
export FD_MODEL_CACHE=/ssd1/download_models
```
> ⭐ **Note**: Models marked with an asterisk can directly use **HuggingFace Torch weights** and support **FP8/WINT8/WINT4** as well as **BF16**. When running inference, you need to enable **`--load-choices "default_v1"`**.
> Example launch Command using baidu/ERNIE-4.5-21B-A3B-PT:
```
python -m fastdeploy.entrypoints.openai.api_server \
@@ -25,8 +23,7 @@ python -m fastdeploy.entrypoints.openai.api_server \
--metrics-port 8181 \
--engine-worker-queue-port 8182 \
--max-model-len 32768 \
--max-num-seqs 32 \
--load-choices "default_v1"
--max-num-seqs 32
```
## Large Language Models