apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[LLM] Add output module and polish docs

Browse Source
This commit is contained in:
jiangjiajun
2025-06-09 20:26:53 +08:00
parent 684703fd72
commit fb18f3092d
8 changed files with 548 additions and 364 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
README.md
View File

@@ -1,5 +1,16 @@
# FastDeploy 2.0: 大模型推理部署
<p align="center">
<a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/FastDeploy?color=ffa"></a>
<a href=""><img src="https://img.shields.io/badge/python-3.10+-aff.svg"></a>
<a href=""><img src="https://img.shields.io/badge/os-linux-pink.svg"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/graphs/contributors"><img src="https://img.shields.io/github/contributors/PaddlePaddle/FastDeploy?color=9ea"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/commits"><img src="https://img.shields.io/github/commit-activity/m/PaddlePaddle/FastDeploy?color=3af"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/issues"><img src="https://img.shields.io/github/issues/PaddlePaddle/FastDeploy?color=9cc"></a>
<a href="https://github.com/PaddlePaddle/FastDeploy/stargazers"><img src="https://img.shields.io/github/stars/PaddlePaddle/FastDeploy?color=ccf"></a>
</p>
FastDeploy升级2.0版本支持多种大模型推理（当前仅支持Qwen2，更多模型即将更新支持)，其推理部署功能涵盖：
- 一行命令即可快速实现模型的服务化部署，并支持流式生成
@@ -9,6 +20,8 @@ FastDeploy升级2.0版本支持多种大模型推理（当前仅支持Qwen2，
- 提供 Weight only int8/int4 无损压缩方案
- 支持 Prometheus Metrics 指标
> 注意: 老版本FastDeploy对于小模型的支持，请checkout [release/1.1.0分支](https://github.com/PaddlePaddle/FastDeploy/tree/release/1.1.0)。
## 环境依赖
- A800/H800/H100
- Python>=3.10
@@ -18,10 +31,9 @@ FastDeploy升级2.0版本支持多种大模型推理（当前仅支持Qwen2，
## 安装
推荐使用Docker环境
推荐使用Docker安装
```
docker pull
iregistry.baidu-int.com/paddlecloud/base-images:paddlecloud-ubuntu24.04-gcc12.3-cuda12.8-cudnn9.7-openmpi4.1.5-bccl2.15.5.4-ofed24.10-hadoop2.2.4.2-afsshell1.9.3.4095-250227
docker pull iregistry.baidu-int.com/paddlepaddle/fastdeploy:2.0.0-alpha
```
### 源码安装
@@ -33,9 +45,7 @@ python -m pip install --pre paddlepaddle-gpu -i https://www.paddlepaddle.org.cn/
2. 安装FastDeploy
```
# git clone FastDeploy仓库
cd FastDeploy
# 一键编译+安装本机可用的sm架构，whl包产物在dist/
bash build.sh
```