apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

add cache queue port (#3904) (#3926)
Some checks failed
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details

Browse Source 
* add cache queue port

* add cache queue port

* add cache queue port
This commit is contained in:
Zhang Yulong
2025-09-06 00:00:12 +08:00
committed by GitHub
parent e2c764fd5a
commit 11b18e5ef0
22 changed files with 74 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/model_loader/utils.py
View File

@@ -78,6 +78,7 @@ def form_model_get_output_topp0(
load_choices,
engine_worker_queue_port,
prompts,
cache_queue_port,
result_queue,
):
try:
@@ -88,6 +89,7 @@ def form_model_get_output_topp0(
load_choices=load_choices,
quantization=quantization,
engine_worker_queue_port=engine_worker_queue_port,
cache_queue_port=cache_queue_port,
) as fd_model:
fd_outputs = fd_model.generate_topp0(prompts, max_tokens=max_tokens)
result_queue.put(fd_outputs)