apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-15 21:20:53 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

[Backend] Add fp16 support for ONNXRuntime-GPU (#1239)

Browse Source 
* add fp16 support for ort-gpu

* add enable_ort_fp16 option

* fix

* fix bugs

* use fp16 for test

* update code

* update code

* fix windows bug
This commit is contained in:
yeliang2258
2023-03-23 18:23:13 +08:00
committed by GitHub
parent b62912ff9c
commit cae341e6c5
9 changed files with 67 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
fastdeploy/runtime/backends/ort/ort_backend.h Normal file → Executable file
View File

@@ -74,6 +74,10 @@ class OrtBackend : public BaseBackend {
std::shared_ptr<Ort::IoBinding> binding_;
std::vector<OrtValueInfo> inputs_desc_;
std::vector<OrtValueInfo> outputs_desc_;
// the ONNX model file name,
// when ONNX is bigger than 2G, we will set this name
std::string model_file_name;
#ifndef NON_64_PLATFORM
Ort::CustomOpDomain custom_op_domain_ = Ort::CustomOpDomain("Paddle");
#endif