apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-05 16:48:03 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Cherry-pick] FIx bug for scheduler V1 (#3167)

Browse Source 
* [BUG FIX] Fix bug when preempted request rescheduled (#3080)

* Fix bug when preempted request rescheduled

* Fix bug when preempted request rescheduled

* Fix bug when preempted request rescheduled

* Fix bug for offline inference in scheduler v1 (#3117)
This commit is contained in:
chenjian
2025-08-04 17:08:12 +08:00
committed by GitHub
parent a592d17615
commit 5439fb6336
3 changed files with 89 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
fastdeploy/engine/engine.py
View File

@@ -499,6 +499,7 @@ class LLMEngine:
enable_thinking = kwargs.get("enable_thinking", None)
request = self.data_processor.process_request(request, self.cfg.max_model_len, enable_thinking=enable_thinking)
request.prompt_token_ids_len = len(request.prompt_token_ids)
request.need_prefill_tokens = request.prompt_token_ids_len
input_ids_len = request.prompt_token_ids_len
request.set(
"max_tokens",