diff --git a/docs/docs_en/runtime/usage.md b/docs/docs_en/runtime/usage.md new file mode 100644 index 000000000..d66a0a6e6 --- /dev/null +++ b/docs/docs_en/runtime/usage.md @@ -0,0 +1,45 @@ +# FastDeploy Runtime User Guideline + +`Runtime`, the module for model inference in FastDeploy, currently integrates a variety of backends. It allows users to quickly complete inference in different model formats on different hardware, platforms and backends through a unified backend. This demo shows the inference on each hardware and backend. + +## CPU Inference + +Python demo + +```python +import fastdeploy as fd +import numpy as np +option = fd.RuntimeOption() +# Set model path +option.set_model_path("resnet50/inference.pdmodel", "resnet50/inference.pdiparams") +# Use OpenVINO backend +option.use_openvino_backend() +# Initialize runtime +runtime = fd.Runtime(option) +# Get input info +input_name = runtime.get_input_info(0).name +# Constructing data for inference +results = runtime.infer({input_name: np.random.rand(1, 3, 224, 224).astype("float32")}) +``` + +## GPU Inference + +```python +import fastdeploy as fd +import numpy as np +option = fd.RuntimeOption() +# Set model path +option.set_model_path("resnet50/inference.pdmodel", "resnet50/inference.pdiparams") +# Use the GPU (0th GPU card) +option.use_gpu(0) +# Use Paddle Inference backend +option.use_paddle_backend() +# Initialize runtime +runtime = fd.Runtime(option) +# Get input info +input_name = runtime.get_input_info(0).name +# Constructing data for inference +results = runtime.infer({input_name: np.random.rand(1, 3, 224, 224).astype("float32")}) +``` + +More Python/C++ inference demo, please refer to [FastDeploy/examples/runtime](../../examples/runtime)