[Feature] ThreadPoolExecutor async fill_token_bitmask (#5083)

* ThreadPoolExecutor async fill_token_bitmask

* ThreadPoolExecutor async fill_token_bitmask logging

* fix test_guided_decoding

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add fill_bitmask_parallel_batch_size ENV

* FD_FILL_BITMASK_BATCH fastdeploy.envs

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Daci
2025-11-19 10:04:16 +08:00
committed by GitHub
parent 4a7739ec0b
commit eab8384da6
6 changed files with 76 additions and 16 deletions

View File

@@ -26,6 +26,7 @@ class TestGuidedDecoding(unittest.TestCase):
"""Setup for each test case."""
# 创建一个基本的FDConfig对象
self.fd_config = Mock()
self.fd_config.scheduler_config = Mock()
self.fd_config.scheduler_config.max_num_seqs = 5
# 创建GuidedDecoding对象