apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-07 01:22:59 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Serving][backend]serving support multi stream and backend support external stream (#431)

Browse Source 
* serving support multi stream

* pybind add external stream

Co-authored-by: Jason <jiangjiajun@baidu.com>
This commit is contained in:
heliqi
2022-10-26 14:46:13 +08:00
committed by GitHub
parent 718698a32a
commit b064ddf7ed
10 changed files with 32 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
fastdeploy/backends/ort/ort_backend.cc
View File

@@ -63,6 +63,10 @@ void OrtBackend::BuildOption(const OrtBackendOption& option) {
} else {
OrtCUDAProviderOptions cuda_options;
cuda_options.device_id = option.gpu_id;
if(option.external_stream_) {
cuda_options.has_user_compute_stream = 1;
cuda_options.user_compute_stream = option.external_stream_;
}
session_options_.AppendExecutionProvider_CUDA(cuda_options);
}
}