diff --git a/docs/get_started/installation/Enflame_gcu.md b/docs/get_started/installation/Enflame_gcu.md index 46d7f0d84..e443a7ce3 100644 --- a/docs/get_started/installation/Enflame_gcu.md +++ b/docs/get_started/installation/Enflame_gcu.md @@ -53,21 +53,24 @@ After driver installation, **re-enter the Docker container**: docker start paddle-gcu-llm docker exec -it paddle-gcu-llm bash ``` -5. Install PaddlePaddle +5. Install PaddlePaddle & PaddleCustomDevice
```bash # PaddlePaddle Deep Learning Framework provides fundamental computing capabilities -python -m pip install paddlepaddle==3.1.0a0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ -``` -6. Install PaddleCustomDevice
-```bash +python -m pip install paddlepaddle==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ + # PaddleCustomDevice implements custom hardware backend for PaddlePaddle, providing GCU operator implementations -python -m pip install paddle-custom-gcu==3.1.0 -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ +python -m pip install paddle-custom-gcu==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ # For source compilation, refer to: https://github.com/PaddlePaddle/PaddleCustomDevice/blob/develop/backends/gcu/README_cn.md ``` -7. Install FastDeploy and dependencies +For latest paddle verion on iluvatar. Refer to [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/) + +6. Install FastDeploy and dependencies ```bash python -m pip install fastdeploy -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels -# For source compilation, refer to the following steps +``` + +You can build FastDeploy from source if you need the ```latest version```. +```bash git clone https://github.com/PaddlePaddle/FastDeploy cd FastDeploy python -m pip install -r requirements.txt --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels diff --git a/docs/get_started/installation/iluvatar_gpu.md b/docs/get_started/installation/iluvatar_gpu.md index 658d9605f..9b4c96f00 100644 --- a/docs/get_started/installation/iluvatar_gpu.md +++ b/docs/get_started/installation/iluvatar_gpu.md @@ -18,7 +18,7 @@ docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-ixuca:latest ``` ## Container Preparation -1. Start Container +### Start Container ```bash docker run -itd --name paddle_infer -v /usr/src:/usr/src -v /lib/modules:/lib/modules -v /dev:/dev -v /home/paddle:/home/paddle --privileged --cap-add=ALL --pid=host ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-ixuca:latest @@ -27,13 +27,26 @@ docker exec -it paddle_infer bash /home/paddle contains the model files, *.whl packages, and scripts. -1. Install packages +### Install paddle ```bash -pip3 install paddlepaddle==3.1.0a0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ -pip3 install paddle-iluvatar-gpu==3.1.0 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ +pip3 install paddlepaddle==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ +pip3 install paddle-iluvatar-gpu==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ +``` +For latest paddle verion on iluvatar. Refer to [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/) + +### Install or build FastDeploy +```bash pip3 install fastdeploy_iluvatar_gpu==2.1.0.dev0 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels ``` +You can build FastDeploy from source if you need the ```latest version```. +```bash +git clone https://github.com/PaddlePaddle/FastDeploy +cd FastDeploy +pip install -r requirements_iluvatar.txt +export LD_PRELOAD=/usr/local/corex/lib64/libcuda.so.1 +bash build.sh +``` ## Prepare the inference demo script diff --git a/docs/zh/get_started/installation/Enflame_gcu.md b/docs/zh/get_started/installation/Enflame_gcu.md index b71a97a8a..cc1042e75 100644 --- a/docs/zh/get_started/installation/Enflame_gcu.md +++ b/docs/zh/get_started/installation/Enflame_gcu.md @@ -52,21 +52,24 @@ bash TopsRider_i3x_*_deb_amd64.run --driver --no-auto-load -y docker start paddle-gcu-llm docker exec -it paddle-gcu-llm bash ``` -5. 安装 PaddlePaddle
+5. 安装 PaddlePaddle & PaddleCustomDevice
```bash # PaddlePaddle『飞桨』深度学习框架,提供运算基础能力 -python -m pip install paddlepaddle==3.1.0a0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ -``` -6. 安装 PaddleCustomDevice
-```bash +python -m pip install paddlepaddle==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ + # PaddleCustomDevice是PaddlePaddle『飞桨』深度学习框架的自定义硬件接入实现,提供GCU的算子实现 -python -m pip install paddle-custom-gcu==3.1.0 -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ +python -m pip install paddle-custom-gcu==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ # 如想源码编译安装,请参考https://github.com/PaddlePaddle/PaddleCustomDevice/blob/develop/backends/gcu/README_cn.md ``` -7. 安装 FastDeploy
+获取Paddle的最新安装版本: [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/) + +6. 安装 FastDeploy
```bash python -m pip install fastdeploy -i https://www.paddlepaddle.org.cn/packages/stable/gcu/ --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels -# 如想源码编译安装,请参考如下步骤 +``` + +可以按如下步骤编译FastDeploy,得到```最新版本```. +```bash git clone https://github.com/PaddlePaddle/FastDeploy cd FastDeploy python -m pip install -r requirements.txt --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels diff --git a/docs/zh/get_started/installation/iluvatar_gpu.md b/docs/zh/get_started/installation/iluvatar_gpu.md index c01ffe93a..1c66a609f 100644 --- a/docs/zh/get_started/installation/iluvatar_gpu.md +++ b/docs/zh/get_started/installation/iluvatar_gpu.md @@ -1,5 +1,5 @@ # 如何在天数机器上运行 ERNIE-4.5-300B-A47B-BF16 & ERNIE-4.5-21B-A3B -当前版本软件只是作为天数芯片 + Fastdeploy 推理大模型的一个演示 demo,跑最新ERNIE4.5模型可能存在问题,后续进行修复和性能优化,给客户提供一个更稳定的版本。 +该软件的当前版本仅作为Iluvatar CoreX与大型模型的Fastdeploy推理框架相结合的演示。在GSM8K数据集上运行最新的ERNIE4.5 300B模型大约需要6.3小时。 ## 准备机器 首先您需要准备以下配置的机器 @@ -18,7 +18,7 @@ docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-ixuca:latest ``` ## 准备容器 -1. 启动容器 +### 启动容器 ```bash docker run -itd --name paddle_infer -v /usr/src:/usr/src -v /lib/modules:/lib/modules -v /dev:/dev -v /home/paddle:/home/paddle --privileged --cap-add=ALL --pid=host ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-ixuca:latest @@ -27,12 +27,25 @@ docker exec -it paddle_infer bash /home/paddle 为模型文件、whl包、脚本所在目录 -1. 安装whl包 +### 安装paddle ```bash -pip3 install paddlepaddle==3.1.0a0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ -pip3 install paddle-iluvatar-gpu==3.1.0 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ -pip3 install fastdeploy_iluvatar_gpu -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels +pip3 install paddlepaddle==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ +pip3 install paddle-iluvatar-gpu==3.1.1 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ +``` +获取Paddle的最新安装版本: [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/) + +### 安装fastdeploy +```bash +pip3 install fastdeploy_iluvatar_gpu==2.1.0.dev0 -i https://www.paddlepaddle.org.cn/packages/stable/ixuca/ --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simplels +``` +可以按如下步骤编译FastDeploy,,得到```最新版本```. +```bash +git clone https://github.com/PaddlePaddle/FastDeploy +cd FastDeploy +pip install -r requirements_iluvatar.txt +export LD_PRELOAD=/usr/local/corex/lib64/libcuda.so.1 +bash build.sh ``` ## 准备推理demo脚本