apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Feature] ThreadPoolExecutor async fill_token_bitmask (#5083)

Browse Source 
* ThreadPoolExecutor async fill_token_bitmask

* ThreadPoolExecutor async fill_token_bitmask logging

* fix test_guided_decoding

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add fill_bitmask_parallel_batch_size ENV

* FD_FILL_BITMASK_BATCH fastdeploy.envs

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Daci
2025-11-19 10:04:16 +08:00
committed by GitHub
parent 4a7739ec0b
commit eab8384da6
6 changed files with 76 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
fastdeploy/worker/worker_process.py
View File

@@ -726,7 +726,7 @@ def parse_args():
)
parser.add_argument(
"--disable_any_whitespace",
action="store_false",
action="store_true",
help="Disable any whitespace for guided decoding.",
)
parser.add_argument(