apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-15 21:20:53 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

[Backend] Add Huawei Ascend NPU deploy using PaddleLite CANN. (#757)

Browse Source 
* Add Huawei Ascend NPU deploy through PaddleLite CANN

* Add NNAdapter interface for paddlelite

* Modify Huawei Ascend Cmake

* Update way for compiling Huawei Ascend NPU deployment

* remove UseLiteBackend in UseCANN

* Support compile python whlee

* Change names of nnadapter API

* Add nnadapter pybind and remove useless API

* Support Python deployment on Huawei Ascend NPU

* Add models suppor for ascend

* Add PPOCR rec reszie for ascend

* fix conflict for ascend

* Rename CANN to Ascend

* Rename CANN to Ascend

* Improve ascend

* fix ascend bug

* improve ascend docs

* improve ascend docs

* improve ascend docs

* Improve Ascend

* Improve Ascend

* Move ascend python demo

* Imporve ascend

* Improve ascend

* Improve ascend

* Improve ascend

* Improve ascend

* Imporve ascend

* Imporve ascend

* Improve ascend
This commit is contained in:
yunyaoXYY
2022-12-26 10:18:34 +08:00
committed by GitHub
parent 2d3d941372
commit d45382e3cc
42 changed files with 714 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/en/build_and_install/README.md
View File

@@ -28,9 +28,10 @@ English | [中文](../../cn/build_and_install/README.md)
| ENABLE_OPENVINO_BACKEND | Default OFF，whether to enable OpenVINO backend(CPU) |
| ENABLE_VISION | Default OFF，whether to enable vision models deployment module |
| ENABLE_TEXT | Default OFF，whether to enable text models deployment module |
| WITH_GPU | Default OFF, if build on GPU, this need to be ON |
| WITH_XPU | Default OFF，if deploy on KunlunXin XPU，this need to be ON |
| WITH_TIMVX | Default OFF，if deploy on RV1126/RV1109/A311D，this need to be ON |
| WITH_GPU | Default OFF, if build on GPU, this needs to be ON |
| WITH_XPU | Default OFF，if deploy on KunlunXin XPU，this needs to be ON |
| WITH_TIMVX | Default OFF，if deploy on RV1126/RV1109/A311D，this needs to be ON |
| WITH_ASCEND | Default OFF，if deploy on Huawei Ascend，this needs to be ON |
| CUDA_DIRECTORY | Default /usr/local/cuda, if build on GPU, this defines the path of CUDA(>=11.2) |
| TRT_DIRECTORY | If build with ENABLE_TRT_BACKEND=ON, this defines the path of TensorRT(>=8.4) |
| ORT_DIRECTORY | [Optional] If build with ENABLE_ORT_BACKEND=ON, this flag defines the path of ONNX Runtime, but if this flag is not set, it will download ONNX Runtime library automatically |

98
docs/en/build_and_install/huawei_ascend.md Normal file
View File

@@ -0,0 +1,98 @@
# How to build Huawei Ascend Deployment Environment
Based on the Paddle-Lite backend, FastDeploy supports model inference on Huawei's Ascend NPU.
For more detailed information, please refer to: [Paddle Lite Deployment Example](https://github.com/PaddlePaddle/Paddle-Lite/blob/develop/docs/demo_guides/huawei_ascend_npu.md).
This document describes how to compile C++ and Python FastDeploy source code under ARM Linux OS environment to generate prediction libraries for Huawei Sunrise NPU as the target hardware.
For more compilation options, please refer to the [FastDeploy compilation options description](./README.md)
## Huawei Ascend Environment Preparation
- Atlas 300I Pro, see detailes at [Spec Sheet](https://e.huawei.com/cn/products/cloud-computing-dc/atlas/atlas-300i-pro)
- Install the driver and firmware package (Driver and Firmware) for the Atlas 300I Pro
- Download the matching driver and firmware package at:
- https://www.hiascend.com/hardware/firmware-drivers?tag=community（Community Edition）
- https://www.hiascend.com/hardware/firmware-drivers?tag=commercial（Commercial version）
- driver：Atlas-300i-pro-npu-driver_5.1.rc2_linux-aarch64.run
- firmware：Atlas-300i-pro-npu-firmware_5.1.rc2.run
- Installing drivers and firmware packages:
```shell
$ chmod +x *.run
$ ./Atlas-300i-pro-npu-driver_5.1.rc2_linux-aarch64.run --full
$ ./Atlas-300i-pro-npu-firmware_5.1.rc2.run --full
$ reboot
# Check the driver information to confirm successful installation
$ npu-smi info
```
- More system and detailed information is available in the [Ascend Hardware Product Documentation](https://www.hiascend.com/document?tag=hardware)
## Compilation environment construction
### Host environment requirements
- os: ARM-Linux
- gcc, g++, git, make, wget, python, pip, python-dev, patchelf
- cmake (version 3.10 or above recommended)
### Using Docker development environment
In order to ensure consistency with the FastDeploy verified build environment, it is recommended to use the Docker development environment for configuration.
```shell
# Download Dockerfile
$ wget https://bj.bcebos.com/fastdeploy/test/Ascend_ubuntu18.04_aarch64_5.1.rc2.Dockerfile
# Create docker images
$ docker build --network=host -f Ascend_ubuntu18.04_aarch64_5.1.rc2.Dockerfile -t Paddle Lite/ascend_aarch64:cann_5.1.rc2 .
# Create container
$ docker run -itd --privileged --name=ascend-aarch64 --net=host -v $PWD:/Work -w /Work --device=/dev/davinci0 --device=/dev/davinci_manager --device=/dev/hisi_hdc --device /dev/devmm_svm -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/:/usr/local/Ascend/driver/ Paddle Lite/ascend_aarch64:cann_5.1.rc2 /bin/bash
# Enter the container
$ docker exec -it ascend-aarch64 /bin/bash
# Verify that the Ascend environment for the container is created successfully
$ npu-smi info
```
Once the above steps are successful, the user can start compiling FastDeploy directly from within docker.
Note:
- If you want to use another CANN version in Docker, please update the CANN download path in the Dockerfile file, and update the corresponding driver and firmware. The current default in Dockerfile is [CANN 5.1.RC2](https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%205.1.RC2/Ascend-cann-toolkit_5.1.RC2_linux-aarch64.run).
- If users do not want to use docker, you can refer to [Compile Environment Preparation for ARM Linux Environments](https://github.com/PaddlePaddle/Paddle-Lite/blob/develop/docs/source_compile/arm_linux_compile_arm_linux.rst) provided by Paddle Lite and configure your own compilation environment, and then download and install the proper CANN packages to complete the configuration.
## C++ FastDeploy library compilation based on Paddle Lite
After setting up the compilation environment, the compilation command is as follows.
```bash
# Download the latest source code
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd FastDeploy
mkdir build && cd build
# CMake configuration with Ascend
cmake -DWITH_ASCEND=ON \
-DCMAKE_INSTALL_PREFIX=fastdeploy-ascend \
-DENABLE_VISION=ON \
..
# Build FastDeploy Ascend C++ SDK
make -j8
make install
```
When the compilation is complete, the fastdeploy-ascend directory is created in the current build directory, indicating that the FastDeploy library has been compiled.
## Compiling Python FastDeploy Libraries Based on Paddle Lite
```bash
# Download the latest source code
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd FastDeploy/python
export WITH_ASCEND=ON
export ENABLE_VISION=ON
python setup.py build
python setup.py bdist_wheel
#After the compilation is complete, please install the whl package in the dist folder of the current directory.
```
Deploying PaddleClas Classification Model on Huawei Ascend NPU using C++ please refer to: [PaddleClas Huawei Ascend NPU C++ Deployment Example](../../../examples/vision/classification/paddleclas/ascend/cpp/README.md)
Deploying PaddleClas classification model on Huawei Ascend NPU using Python please refer to: [PaddleClas Huawei Ascend NPU Python Deployment Example](../../../examples/vision/classification/paddleclas/ascend/python/README.md)

21
docs/en/faq/use_sdk_on_ascend.md Normal file
View File

@@ -0,0 +1,21 @@
[简体中文](../../cn/faq/use_sdk_on_linux.md) | English
# # Linux deployment with C++ on Huawei Ascend
After the deployment example is compiled, we need to import some environment variables to initialize the deployment environment before running the program, because we need to use the Huawei Ascend toolkit.
Users can use the following script (located in the directory of the compiled FastDeploy library) to initialize the Huawei Ascend deployment environment.
```
# The path to our default Ascend Toolkit is as follows,
# HUAWEI_ASCEND_TOOLKIT_HOME="/usr/local/Ascend/ascend-toolkit/latest"
# HUAWEI_ASCEND_DRIVER_PATH="/usr/local/Ascend/driver"
# If the user's installation directory is different from this, you need to export it manually first.
# export HUAWEI_ASCEND_TOOLKIT_HOME="Your_ascend_toolkit_path"
# export HUAWEI_ASCEND_DRIVER_PATH="Your_ascend_driver_path"
source fastdeploy-ascend/fastdeploy_init.sh
```
Note that this command only takes effect in the current command environment after execution (switching to a new terminal window, or closing the window and reopening it will not work), if you need to keep it in effect on the system, add these environment variables to the `~/.bashrc` file.