apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[v1 loader]code style (#4204)

Browse Source 
* code style

* update
This commit is contained in:
bukejiyu
2025-09-23 19:36:00 +08:00
committed by GitHub
parent 1a6283424e
commit 62d1c48363
18 changed files with 33 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/supported_models.md
View File

@@ -13,7 +13,7 @@ export FD_MODEL_SOURCE=AISTUDIO # "AISTUDIO", "MODELSCOPE" or "HUGGINGFACE"
export FD_MODEL_CACHE=/ssd1/download_models
```
> ⭐ **Note**: Models marked with an asterisk can directly use **HuggingFace Torch weights** and support **FP8/WINT8/WINT4** as well as **BF16**. When running inference, you need to enable **`--load_choices "default_v1"`**.
> ⭐ **Note**: Models marked with an asterisk can directly use **HuggingFace Torch weights** and support **FP8/WINT8/WINT4** as well as **BF16**. When running inference, you need to enable **`--load-choices "default_v1"`**.
> Example launch Command using baidu/ERNIE-4.5-21B-A3B-PT:
```
@@ -24,7 +24,7 @@ python -m fastdeploy.entrypoints.openai.api_server \
--engine-worker-queue-port 8182 \
--max-model-len 32768 \
--max-num-seqs 32 \
--load_choices "default_v1"
--load-choices "default_v1"
```
## Large Language Models