mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-08 01:50:27 +08:00
translate text/readme and its links (#914)
* Create README_CN.md * Create README_EN.md * Update README.md * Update and rename README.md to README_CN.md * Create README_EN.md * Create README_EN.md * Create README_EN.md * Update and rename README.md to README_CN.md * Update README_EN.md * Update and rename README.md to README_CN.md * Create README_EN.md * Create README_CN.md * Create README_EN.md * Update README.md * Update README_EN.md * Update README.md * Create README_EN.md * Update README_EN.md * Update README.md * Create README_EN.md * Update README.md * Create README_EN.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Rename README_EN.md to README_.md * Rename README_.md to README.md * Update README_CN.md * Rename README_EN.md to README.md * Update README_CN.md * Update README_CN.md * Rename README_EN.md to README.md * Update README_CN.md * Update README.md * Rename README_EN.md to README_2EN.md * Update and rename README_EN.md to README.md * Rename README_2EN.md to README_EN.md * Update and rename README_EN.md to README_CN.md * Update README.md * Update and rename README_EN.md to README_CN.md * Update README.md * Update README.md * Update README.md * Update and rename README_EN.md to README_CN.md * Update README.md * Update README_EN.md
This commit is contained in:
HCQ14
@@ -1,39 +1 @@
|
||||
# ERNIE 3.0 模型部署
|
||||
|
||||
## 模型详细说明
|
||||
- [PaddleNLP ERNIE 3.0模型说明](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
## 支持的模型列表
|
||||
|
||||
| 模型 | 结构 | 语言 |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `ERNIE 3.0-Base`| 12-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Medium`| 6-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Mini`| 6-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Micro`| 4-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Nano `| 4-layers, 312-hidden, 12-heads | 中文 |
|
||||
|
||||
## 支持的NLP任务列表
|
||||
|
||||
| 任务 Task | 是否支持 |
|
||||
| :--------------- | ------- |
|
||||
| 文本分类 | ✅ |
|
||||
| 序列标注 | ❌ |
|
||||
|
||||
## 导出部署模型
|
||||
|
||||
在部署前，需要先将训练好的ERNIE模型导出成部署模型，导出步骤可参考文档[导出模型](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0).
|
||||
|
||||
## 下载微调模型
|
||||
|
||||
### 分类任务
|
||||
|
||||
为了方便开发者的测试，下面提供了在文本分类[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上微调的ERNIE 3.0-Medium模型，开发者可直接下载体验。
|
||||
|
||||
- [ERNIE 3.0 Medium AFQMC](https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz)
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [Serving部署](serving)
|
||||
README_EN.md
|
||||
|
||||
41
examples/text/ernie-3.0/README_CN.md
Normal file
41
examples/text/ernie-3.0/README_CN.md
Normal file
@@ -0,0 +1,41 @@
|
||||
[English](README_EN.md) | 简体中文
|
||||
|
||||
# ERNIE 3.0 模型部署
|
||||
|
||||
## 模型详细说明
|
||||
- [PaddleNLP ERNIE 3.0模型说明](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
## 支持的模型列表
|
||||
|
||||
| 模型 | 结构 | 语言 |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `ERNIE 3.0-Base`| 12-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Medium`| 6-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Mini`| 6-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Micro`| 4-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `ERNIE 3.0-Nano `| 4-layers, 312-hidden, 12-heads | 中文 |
|
||||
|
||||
## 支持的NLP任务列表
|
||||
|
||||
| 任务 Task | 是否支持 |
|
||||
| :--------------- | ------- |
|
||||
| 文本分类 | ✅ |
|
||||
| 序列标注 | ❌ |
|
||||
|
||||
## 导出部署模型
|
||||
|
||||
在部署前，需要先将训练好的ERNIE模型导出成部署模型，导出步骤可参考文档[导出模型](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0).
|
||||
|
||||
## 下载微调模型
|
||||
|
||||
### 分类任务
|
||||
|
||||
为了方便开发者的测试，下面提供了在文本分类[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上微调的ERNIE 3.0-Medium模型，开发者可直接下载体验。
|
||||
|
||||
- [ERNIE 3.0 Medium AFQMC](https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz)
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [Serving部署](serving)
|
||||
41
examples/text/ernie-3.0/README_EN.md
Normal file
41
examples/text/ernie-3.0/README_EN.md
Normal file
@@ -0,0 +1,41 @@
|
||||
[简体中文](README_CN.md)| English
|
||||
|
||||
# ERNIE 3.0 Model Deployment
|
||||
|
||||
## Model Description
|
||||
- [PaddleNLP ERNIE 3.0 Model Description](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
## A list of Supported Models
|
||||
|
||||
| Model | Structure | Language |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `ERNIE 3.0-Base`| 12-layers, 768-hidden, 12-heads | Chinese |
|
||||
| `ERNIE 3.0-Medium`| 6-layers, 768-hidden, 12-heads | Chinese |
|
||||
| `ERNIE 3.0-Mini`| 6-layers, 384-hidden, 12-heads | Chinese |
|
||||
| `ERNIE 3.0-Micro`| 4-layers, 384-hidden, 12-heads | Chinese |
|
||||
| `ERNIE 3.0-Nano `| 4-layers, 312-hidden, 12-heads | Chinese |
|
||||
|
||||
## A list of Supported NLP Tasks
|
||||
|
||||
| Task | Yes or No |
|
||||
| :--------------- | ------- |
|
||||
| text classification | ✅ |
|
||||
| sequence labeling | ❌ |
|
||||
|
||||
## Export Deployment Models
|
||||
|
||||
Before deployment, it is required to export trained ERNIE models into deployment models. The export steps can be found in the document [Export Model](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0).
|
||||
|
||||
## Download Fine-tuning Models
|
||||
|
||||
### Classification Task
|
||||
|
||||
For developers' testing, the fine-tuned ERNIE 3.0-Medium Model on text classification [AFQMC dataset] (https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip) is provided below. Developers can download it directly.
|
||||
|
||||
- [ERNIE 3.0 Medium AFQMC](https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz)
|
||||
|
||||
## Detailed Deployment Documents
|
||||
|
||||
- [Python Deployment](python)
|
||||
- [C++ Deployment](cpp)
|
||||
- [Serving Deployment](serving)
|
||||
@@ -1,42 +1,42 @@
|
||||
# ERNIE 3.0 模型C++部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
# Example of ERNIE 3.0 models C++ Deployment
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
Before deployment, two steps require confirmation.
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 1. Environment of software and hardware should meet the requirements. Please refer to[FastDeploy Environment Requirements](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Based on the develop environment, download the precompiled deployment library and samples code. Please refer to [FastDeploy Precompiled Library](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
本目录下提供`seq_cls_infer.cc`快速完成在CPU/GPU的文本分类任务的C++部署示例。
|
||||
This directory provides deployment examples that seq_cls_inferve.py fast finish text classification tasks on CPU/GPU.
|
||||
|
||||
|
||||
## 文本分类任务
|
||||
## Text Classification Tasks
|
||||
|
||||
### 快速开始
|
||||
### A Quick Start
|
||||
|
||||
以下示例展示如何基于FastDeploy库完成ERNIE 3.0 Medium模型在CLUE Benchmark的[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上进行文本分类任务的C++预测部署。支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)
|
||||
The following example shows how to employ FastDeploy library to complete C++ predictive deployment of ERNIE 3.0 Medium model on [AFQMC dataset](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip) of CLUE Benchmark for text classification tasks.FastDeploy version 0.7.0 or above is required to support this model(x.x.x>=0.7.0) )
|
||||
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
# Download FastDeploy precompiled library. Uses can choose proper version in the `FastDeploy Precompiled Library`mentioned above.
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载AFQMC数据集的微调后的ERNIE 3.0模型以及词表
|
||||
# Download the fine-tuned ERNIE 3.0 models for the AFQMC dataset and the word lists.
|
||||
wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz
|
||||
tar xvfz ernie-3.0-medium-zh-afqmc.tgz
|
||||
|
||||
# CPU 推理
|
||||
# CPU Inference
|
||||
./seq_cls_infer_demo --device cpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
# GPU 推理
|
||||
# GPU Inference
|
||||
./seq_cls_infer_demo --device gpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
```
|
||||
|
||||
运行完成后返回的结果如下：
|
||||
The result returned after running is as follows：
|
||||
```bash
|
||||
[INFO] /paddle/FastDeploy/examples/text/ernie-3.0/cpp/seq_cls_infer.cc(93)::CreateRuntimeOption model_path = ernie-3.0-medium-zh-afqmc/infer.pdmodel, param_path = ernie-3.0-medium-zh-afqmc/infer.pdiparams
|
||||
[INFO] fastdeploy/runtime.cc(469)::Init Runtime initialized with Backend::ORT in Device::CPU.
|
||||
@@ -46,23 +46,23 @@ Batch id: 1, example id: 0, sentence 1: 花呗支持高铁票支付吗, sentence
|
||||
|
||||
|
||||
|
||||
### 参数说明
|
||||
### Parameter Description
|
||||
|
||||
`seq_cls_infer_demo` 除了以上示例的命令行参数，还支持更多命令行参数的设置。以下为各命令行参数的说明。
|
||||
`seq_cls_infer_demo` supports more command-line arguments in addition to the preceding example. The following is a description of every command-line argument:
|
||||
|
||||
| 参数 |参数说明 |
|
||||
| Parameter | Parameter Description |
|
||||
|----------|--------------|
|
||||
|--model_dir | 指定部署模型的目录， |
|
||||
|--batch_size |最大可测的 batch size，默认为 1|
|
||||
|--max_length |最大序列长度，默认为 128|
|
||||
|--device | 运行的设备，可选范围: ['cpu', 'gpu']，默认为'cpu' |
|
||||
|--backend | 支持的推理后端，可选范围: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，默认为'onnx_runtime' |
|
||||
|--use_fp16 | 是否使用FP16模式进行推理。使用tensorrt和paddle_tensorrt后端时可开启，默认为False |
|
||||
|--model_dir | Specify the directory where the model is to be deployed |
|
||||
|--batch_size |Maximum measurable batch size, default 1|
|
||||
|--max_length |Maximum sequence length, default 128|
|
||||
|--device | Running devices, optional range: ['cpu', 'gpu'], default 'cpu' |
|
||||
|--backend | Supported inference backend, optional range: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，default'onnx_runtime' |
|
||||
|--use_fp16 | Whether to use FP16 mode for inference. Enabled when using tensorrt and paddle_tensorrt backend, and default is False |
|
||||
|
||||
## 相关文档
|
||||
## Related Documents
|
||||
|
||||
[ERNIE 3.0模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
[ERNIE 3.0 Model Detailed Instruction](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
[ERNIE 3.0 Model Export Method](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型Python部署方法](../python/README.md)
|
||||
[ERNIE 3.0 Model Python Deployment Method](../python/README.md)
|
||||
|
||||
69
examples/text/ernie-3.0/cpp/README_CN.md
Normal file
69
examples/text/ernie-3.0/cpp/README_CN.md
Normal file
@@ -0,0 +1,69 @@
|
||||
[English](README.md) | 简体中文
|
||||
# ERNIE 3.0 模型C++部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
本目录下提供`seq_cls_infer.cc`快速完成在CPU/GPU的文本分类任务的C++部署示例。
|
||||
|
||||
|
||||
## 文本分类任务
|
||||
|
||||
### 快速开始
|
||||
|
||||
以下示例展示如何基于FastDeploy库完成ERNIE 3.0 Medium模型在CLUE Benchmark的[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上进行文本分类任务的C++预测部署。支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)
|
||||
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载AFQMC数据集的微调后的ERNIE 3.0模型以及词表
|
||||
wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz
|
||||
tar xvfz ernie-3.0-medium-zh-afqmc.tgz
|
||||
|
||||
# CPU 推理
|
||||
./seq_cls_infer_demo --device cpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
# GPU 推理
|
||||
./seq_cls_infer_demo --device gpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
```
|
||||
|
||||
运行完成后返回的结果如下：
|
||||
```bash
|
||||
[INFO] /paddle/FastDeploy/examples/text/ernie-3.0/cpp/seq_cls_infer.cc(93)::CreateRuntimeOption model_path = ernie-3.0-medium-zh-afqmc/infer.pdmodel, param_path = ernie-3.0-medium-zh-afqmc/infer.pdiparams
|
||||
[INFO] fastdeploy/runtime.cc(469)::Init Runtime initialized with Backend::ORT in Device::CPU.
|
||||
Batch id: 0, example id: 0, sentence 1: 花呗收款额度限制, sentence 2: 收钱码，对花呗支付的金额有限制吗, label: 1, confidence: 0.581852
|
||||
Batch id: 1, example id: 0, sentence 1: 花呗支持高铁票支付吗, sentence 2: 为什么友付宝不支持花呗付款, label: 0, confidence: 0.997921
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 参数说明
|
||||
|
||||
`seq_cls_infer_demo` 除了以上示例的命令行参数，还支持更多命令行参数的设置。以下为各命令行参数的说明。
|
||||
|
||||
| 参数 |参数说明 |
|
||||
|----------|--------------|
|
||||
|--model_dir | 指定部署模型的目录， |
|
||||
|--batch_size |最大可测的 batch size，默认为 1|
|
||||
|--max_length |最大序列长度，默认为 128|
|
||||
|--device | 运行的设备，可选范围: ['cpu', 'gpu']，默认为'cpu' |
|
||||
|--backend | 支持的推理后端，可选范围: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，默认为'onnx_runtime' |
|
||||
|--use_fp16 | 是否使用FP16模式进行推理。使用tensorrt和paddle_tensorrt后端时可开启，默认为False |
|
||||
|
||||
## 相关文档
|
||||
|
||||
[ERNIE 3.0模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型Python部署方法](../python/README.md)
|
||||
@@ -1,46 +1,48 @@
|
||||
# ERNIE 3.0 模型Python部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
# Example of ERNIE 3.0 Models Python Deployment
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
Before deployment, two steps require confirmation.
|
||||
|
||||
本目录下提供`seq_cls_infer.py`快速完成在CPU/GPU的文本分类任务的部署示例。
|
||||
- 1. Environment of software and hardware should meet the requirements. Please refer to [FastDeploy Environment Requirements](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl package should be installed. Please refer to [FastDeploy Python Installation](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
## 依赖安装
|
||||
This directory provides deployment examples that seq_cls_inferve.py fast finish text classification tasks on CPU/GPU.
|
||||
|
||||
本项目提供的Python版本的预测器Predictor基于PaddleNLP提供的AutoTokenizer进行分词，并利用fast_tokenizer加速分词, 执行以下命令进行安装。
|
||||
## Dependency Installation
|
||||
|
||||
The Python Predictor in this project uses AutoTokenizer provided by PaddleNLP to conduct word segmentation and fast_tokenizer to speed up word segmentation. Run the following command to install it.
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
|
||||
## 文本分类任务
|
||||
## Text Classification Tasks
|
||||
|
||||
### 快速开始
|
||||
### A Quick Start
|
||||
|
||||
以下示例展示如何基于FastDeploy库完成ERNIE 3.0 Medium模型在CLUE Benchmark 的[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上进行文本分类任务的Python预测部署。
|
||||
The following example shows how to employ FastDeploy library to complete Python predictive deployment of ERNIE 3.0 Medium model on [AFQMC Dataset](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)of CLUE Benchmark for text classification tasks.
|
||||
|
||||
```bash
|
||||
|
||||
# 下载部署示例代码
|
||||
# Download the deployment example code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/text/ernie-3.0/python
|
||||
|
||||
# 下载AFQMC数据集的微调后的ERNIE 3.0模型
|
||||
# Download fine-tuned ERNIE 3.0 models for AFQMC dataset
|
||||
wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz
|
||||
tar xvfz ernie-3.0-medium-zh-afqmc.tgz
|
||||
|
||||
# CPU 推理
|
||||
# CPU Inference
|
||||
python seq_cls_infer.py --device cpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
# GPU 推理
|
||||
# GPU Inference
|
||||
python seq_cls_infer.py --device gpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
```
|
||||
The result returned after running is as follows:
|
||||
|
||||
运行完成后返回的结果如下：
|
||||
|
||||
```bash
|
||||
[INFO] fastdeploy/runtime.cc(469)::Init Runtime initialized with Backend::ORT in Device::CPU.
|
||||
@@ -48,24 +50,24 @@ Batch id:0, example id:0, sentence1:花呗收款额度限制, sentence2:收钱
|
||||
Batch id:1, example id:0, sentence1:花呗支持高铁票支付吗, sentence2:为什么友付宝不支持花呗付款, label:0, similarity:0.9979
|
||||
```
|
||||
|
||||
### 参数说明
|
||||
### Parameter Description
|
||||
|
||||
`seq_cls_infer.py` 除了以上示例的命令行参数，还支持更多命令行参数的设置。以下为各命令行参数的说明。
|
||||
`seq_cls_infer.py`supports more command-line arguments in addition to the preceding example. The following is a description of every command-line argument
|
||||
|
||||
| 参数 |参数说明 |
|
||||
| Parameter | Parameter Description |
|
||||
|----------|--------------|
|
||||
|--model_dir | 指定部署模型的目录， |
|
||||
|--batch_size |最大可测的 batch size，默认为 1|
|
||||
|--max_length |最大序列长度，默认为 128|
|
||||
|--device | 运行的设备，可选范围: ['cpu', 'gpu']，默认为'cpu' |
|
||||
|--backend | 支持的推理后端，可选范围: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，默认为'onnx_runtime' |
|
||||
|--use_fp16 | 是否使用FP16模式进行推理。使用tensorrt和paddle_tensorrt后端时可开启，默认为False |
|
||||
|--use_fast| 是否使用FastTokenizer加速分词阶段。默认为True|
|
||||
|--model_dir | Specify the directory where the model is to be deployed |
|
||||
|--batch_size |Maximum measurable batch size, default 1|
|
||||
|--max_length |Maximum sequence length, default 128|
|
||||
|--device | Running devices, optional range: ['cpu', 'gpu'], default 'cpu' |
|
||||
|--backend | Supported inference backend, optional range: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，default 'onnx_runtime' |
|
||||
|--use_fp16 | Whether to use FP16 mode for inference.Enabled when using tensorrt and paddle tensorrt backend, and default is False |
|
||||
|--use_fast| Whether to use FastTokenizer to speed up the tokenization, and default is True|
|
||||
|
||||
## 相关文档
|
||||
## Related Documents
|
||||
|
||||
[ERNIE 3.0模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
[ERNIE 3.0 Model detailed introduction](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
[ERNIE 3.0 Model Export Method](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型C++部署方法](../cpp/README.md)
|
||||
[ERNIE 3.0 Model C++ Deployment Method](../cpp/README.md)
|
||||
|
||||
73
examples/text/ernie-3.0/python/README_CN.md
Normal file
73
examples/text/ernie-3.0/python/README_CN.md
Normal file
@@ -0,0 +1,73 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# ERNIE 3.0 模型Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
本目录下提供`seq_cls_infer.py`快速完成在CPU/GPU的文本分类任务的部署示例。
|
||||
|
||||
## 依赖安装
|
||||
|
||||
本项目提供的Python版本的预测器Predictor基于PaddleNLP提供的AutoTokenizer进行分词，并利用fast_tokenizer加速分词, 执行以下命令进行安装。
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
|
||||
## 文本分类任务
|
||||
|
||||
### 快速开始
|
||||
|
||||
以下示例展示如何基于FastDeploy库完成ERNIE 3.0 Medium模型在CLUE Benchmark 的[AFQMC数据集](https://bj.bcebos.com/paddlenlp/datasets/afqmc_public.zip)上进行文本分类任务的Python预测部署。
|
||||
|
||||
```bash
|
||||
|
||||
# 下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/text/ernie-3.0/python
|
||||
|
||||
# 下载AFQMC数据集的微调后的ERNIE 3.0模型
|
||||
wget https://bj.bcebos.com/fastdeploy/models/ernie-3.0/ernie-3.0-medium-zh-afqmc.tgz
|
||||
tar xvfz ernie-3.0-medium-zh-afqmc.tgz
|
||||
|
||||
# CPU 推理
|
||||
python seq_cls_infer.py --device cpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
# GPU 推理
|
||||
python seq_cls_infer.py --device gpu --model_dir ernie-3.0-medium-zh-afqmc
|
||||
|
||||
```
|
||||
|
||||
运行完成后返回的结果如下：
|
||||
|
||||
```bash
|
||||
[INFO] fastdeploy/runtime.cc(469)::Init Runtime initialized with Backend::ORT in Device::CPU.
|
||||
Batch id:0, example id:0, sentence1:花呗收款额度限制, sentence2:收钱码，对花呗支付的金额有限制吗, label:1, similarity:0.5819
|
||||
Batch id:1, example id:0, sentence1:花呗支持高铁票支付吗, sentence2:为什么友付宝不支持花呗付款, label:0, similarity:0.9979
|
||||
```
|
||||
|
||||
### 参数说明
|
||||
|
||||
`seq_cls_infer.py` 除了以上示例的命令行参数，还支持更多命令行参数的设置。以下为各命令行参数的说明。
|
||||
|
||||
| 参数 |参数说明 |
|
||||
|----------|--------------|
|
||||
|--model_dir | 指定部署模型的目录， |
|
||||
|--batch_size |最大可测的 batch size，默认为 1|
|
||||
|--max_length |最大序列长度，默认为 128|
|
||||
|--device | 运行的设备，可选范围: ['cpu', 'gpu']，默认为'cpu' |
|
||||
|--backend | 支持的推理后端，可选范围: ['onnx_runtime', 'paddle', 'openvino', 'tensorrt', 'paddle_tensorrt']，默认为'onnx_runtime' |
|
||||
|--use_fp16 | 是否使用FP16模式进行推理。使用tensorrt和paddle_tensorrt后端时可开启，默认为False |
|
||||
|--use_fast| 是否使用FastTokenizer加速分词阶段。默认为True|
|
||||
|
||||
## 相关文档
|
||||
|
||||
[ERNIE 3.0模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/ernie-3.0)
|
||||
|
||||
[ERNIE 3.0模型C++部署方法](../cpp/README.md)
|
||||
@@ -1,31 +1,33 @@
|
||||
# ERNIE 3.0 服务化部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
在服务化部署前，需确认
|
||||
# Example of ERNIE 3.0 Serving Deployment
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
Before serving deployment, you need to confirm
|
||||
|
||||
## 准备模型
|
||||
- 1. Refer to [FastDeploy Serving Deployment](../../../../../serving/README_CN.md) for hardware and software environment requirements and image pull commands of serving images.
|
||||
|
||||
下载ERNIE 3.0的新闻分类模型、序列标注模型(如果有已训练好的模型，跳过此步骤):
|
||||
## Prepare Models
|
||||
|
||||
Download the news classification model and the sequence labeling model of ERNIE 3.0 (if you have trained models, skip this step):
|
||||
```bash
|
||||
# 下载并解压新闻分类模型
|
||||
# Download and decompress the news classification model
|
||||
wget https://paddlenlp.bj.bcebos.com/models/transformers/ernie_3.0/tnews_pruned_infer_model.zip
|
||||
unzip tnews_pruned_infer_model.zip
|
||||
|
||||
# 将下载的模型移动到分类任务的模型仓库目录
|
||||
# Move the download model to the model repository directory of classification tasks.
|
||||
mv tnews_pruned_infer_model/float32.pdmodel models/ernie_seqcls_model/1/model.pdmodel
|
||||
mv tnews_pruned_infer_model/float32.pdiparams models/ernie_seqcls_model/1/model.pdiparams
|
||||
|
||||
# 下载并解压序列标注模型
|
||||
# Download and decompress the sequence labelling model
|
||||
wget https://paddlenlp.bj.bcebos.com/models/transformers/ernie_3.0/msra_ner_pruned_infer_model.zip
|
||||
unzip msra_ner_pruned_infer_model.zip
|
||||
|
||||
# 将下载的模型移动到序列标注任务的模型仓库目录
|
||||
# Move the download model to the model repository directory of sequence labeling task.
|
||||
mv msra_ner_pruned_infer_model/float32.pdmodel models/ernie_tokencls_model/1/model.pdmodel
|
||||
mv msra_ner_pruned_infer_model/float32.pdiparams models/ernie_tokencls_model/1/model.pdiparams
|
||||
```
|
||||
|
||||
模型下载移动好之后，分类任务的models目录结构如下:
|
||||
After download and move, the models directory of the classification tasks is as follows:
|
||||
```
|
||||
models
|
||||
├── ernie_seqcls # 分类任务的pipeline
|
||||
@@ -45,20 +47,20 @@ models
|
||||
└── config.pbtxt
|
||||
```
|
||||
|
||||
## 拉取并运行镜像
|
||||
## Pull and Run Images
|
||||
```bash
|
||||
# x.y.z为镜像版本号，需参照serving文档替换为数字
|
||||
# GPU镜像
|
||||
# x.y.z represent image versions. Please refer to the serving document to replace them with numbers
|
||||
# GPU Image
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
# CPU Image
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行
|
||||
# Running
|
||||
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 bash
|
||||
```
|
||||
|
||||
## 部署模型
|
||||
serving目录包含启动pipeline服务的配置和发送预测请求的代码，包括：
|
||||
## Deployment Models
|
||||
The serving directory contains the configuration to start the pipeline service and the code to send the prediction request, including
|
||||
|
||||
```
|
||||
models # 服务化启动需要的模型仓库，包含模型和服务配置文件
|
||||
@@ -66,12 +68,13 @@ seq_cls_rpc_client.py # 新闻分类任务发送pipeline预测请求的脚
|
||||
token_cls_rpc_client.py # 序列标注任务发送pipeline预测请求的脚本
|
||||
```
|
||||
|
||||
*注意*:启动服务时，Server的每个python后端进程默认申请`64M`内存，默认启动的docker无法启动多个python后端节点。有两个解决方案：
|
||||
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash`
|
||||
- 2.启动服务时设置python后端的`shm-default-byte-size`参数, 设置python后端的默认内存为10M： `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760`
|
||||
*Attention*:Attention: When starting the service, each python backend process of Server requests 64M memory by default, and the docker started by default cannot start more than one python backend node. There are two solutions:
|
||||
|
||||
### 分类任务
|
||||
在容器内执行下面命令启动服务:
|
||||
- 1.Set the `shm-size` parameter when starting the container, for example, `docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash`
|
||||
- 2.Set the `shm-default-byte-size` parameter of python backend when starting the service. Set the default memory of python backend to 10M： `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760`
|
||||
|
||||
### Classification Task
|
||||
Execute the following command in the container to start the service:
|
||||
```
|
||||
# 默认启动models下所有模型
|
||||
fastdeployserver --model-repository=/models
|
||||
@@ -101,8 +104,8 @@ I0601 07:15:15.923604 8059 http_server.cc:2815] Started HTTPService at 0.0.0.0:8
|
||||
I0601 07:15:15.964984 8059 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
### 序列标注任务
|
||||
在容器内执行下面命令启动序列标注服务:
|
||||
### Sequence Labelling Task
|
||||
Execute the following command in the container to start the sequence labelling service:
|
||||
```
|
||||
fastdeployserver --model-repository=/models --model-control-mode=explicit --load-model=ernie_tokencls --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
@@ -128,20 +131,20 @@ I0601 07:15:15.923604 8059 http_server.cc:2815] Started HTTPService at 0.0.0.0:8
|
||||
I0601 07:15:15.964984 8059 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
## 客户端请求
|
||||
客户端请求可以在本地执行脚本请求；也可以在容器中执行。
|
||||
## Client Requests
|
||||
Client requests can execute script requests locally and in the container.
|
||||
|
||||
本地执行脚本需要先安装依赖:
|
||||
Dependencies should be installed to execute the script locally:
|
||||
```
|
||||
pip install grpcio
|
||||
pip install tritonclient[all]
|
||||
|
||||
# 如果bash无法识别括号，可以使用如下指令安装:
|
||||
# If bash cannot recognize the brackets, you can use the following command to install dependencies:
|
||||
pip install tritonclient\[all\]
|
||||
```
|
||||
|
||||
### 分类任务
|
||||
注意执行客户端请求时关闭代理，并根据实际情况修改main函数中的ip地址(启动服务所在的机器)
|
||||
### Classification Task
|
||||
Attention: The proxy need turning off when executing client requests. The ip address in the main function (the machine where you start services) should be modified as appropriate.
|
||||
```
|
||||
python seq_cls_grpc_client.py
|
||||
```
|
||||
@@ -152,8 +155,8 @@ python seq_cls_grpc_client.py
|
||||
acc: 0.5731
|
||||
```
|
||||
|
||||
### 序列标注任务
|
||||
注意执行客户端请求时关闭代理，并根据实际情况修改main函数中的ip地址(启动服务所在的机器)
|
||||
### Sequence Labeling Task
|
||||
Attention: The proxy need turning off when executing client requests. The ip address in the main function (the machine where you start services) should be modified as appropriate.
|
||||
```
|
||||
python token_cls_grpc_client.py
|
||||
```
|
||||
@@ -171,5 +174,4 @@ entity: 华夏 label: LOC pos: [14, 15]
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前分类任务(ernie_seqcls_model/config.pbtxt)默认配置在CPU上运行OpenVINO引擎; 序列标注任务默认配置在GPU上运行Paddle引擎。如果要在CPU/GPU或其他推理引擎上运行, 需要修改配置，详情请参考[配置文档](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
The current classification task (ernie_seqcls_model/config.pbtxt) is by default configured to run the OpenVINO engine on CPU; the sequence labelling task is by default configured to run the Paddle engine on GPU. If you want to run on CPU/GPU or other inference engines, you should modify the configuration. please refer to the [configuration document.](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
177
examples/text/ernie-3.0/serving/README_CN.md
Normal file
177
examples/text/ernie-3.0/serving/README_CN.md
Normal file
@@ -0,0 +1,177 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# ERNIE 3.0 服务化部署示例
|
||||
|
||||
在服务化部署前，需确认
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
|
||||
## 准备模型
|
||||
|
||||
下载ERNIE 3.0的新闻分类模型、序列标注模型(如果有已训练好的模型，跳过此步骤):
|
||||
```bash
|
||||
# 下载并解压新闻分类模型
|
||||
wget https://paddlenlp.bj.bcebos.com/models/transformers/ernie_3.0/tnews_pruned_infer_model.zip
|
||||
unzip tnews_pruned_infer_model.zip
|
||||
|
||||
# 将下载的模型移动到分类任务的模型仓库目录
|
||||
mv tnews_pruned_infer_model/float32.pdmodel models/ernie_seqcls_model/1/model.pdmodel
|
||||
mv tnews_pruned_infer_model/float32.pdiparams models/ernie_seqcls_model/1/model.pdiparams
|
||||
|
||||
# 下载并解压序列标注模型
|
||||
wget https://paddlenlp.bj.bcebos.com/models/transformers/ernie_3.0/msra_ner_pruned_infer_model.zip
|
||||
unzip msra_ner_pruned_infer_model.zip
|
||||
|
||||
# 将下载的模型移动到序列标注任务的模型仓库目录
|
||||
mv msra_ner_pruned_infer_model/float32.pdmodel models/ernie_tokencls_model/1/model.pdmodel
|
||||
mv msra_ner_pruned_infer_model/float32.pdiparams models/ernie_tokencls_model/1/model.pdiparams
|
||||
```
|
||||
|
||||
模型下载移动好之后，分类任务的models目录结构如下:
|
||||
```
|
||||
models
|
||||
├── ernie_seqcls # 分类任务的pipeline
|
||||
│ ├── 1
|
||||
│ └── config.pbtxt # 通过这个文件组合前后处理和模型推理
|
||||
├── ernie_seqcls_model # 分类任务的模型推理
|
||||
│ ├── 1
|
||||
│ │ └── model.onnx
|
||||
│ └── config.pbtxt
|
||||
├── ernie_seqcls_postprocess # 分类任务后处理
|
||||
│ ├── 1
|
||||
│ │ └── model.py
|
||||
│ └── config.pbtxt
|
||||
└── ernie_tokenizer # 预处理分词
|
||||
├── 1
|
||||
│ └── model.py
|
||||
└── config.pbtxt
|
||||
```
|
||||
|
||||
## 拉取并运行镜像
|
||||
```bash
|
||||
# x.y.z为镜像版本号，需参照serving文档替换为数字
|
||||
# GPU镜像
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行
|
||||
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 bash
|
||||
```
|
||||
|
||||
## 部署模型
|
||||
serving目录包含启动pipeline服务的配置和发送预测请求的代码，包括：
|
||||
|
||||
```
|
||||
models # 服务化启动需要的模型仓库，包含模型和服务配置文件
|
||||
seq_cls_rpc_client.py # 新闻分类任务发送pipeline预测请求的脚本
|
||||
token_cls_rpc_client.py # 序列标注任务发送pipeline预测请求的脚本
|
||||
```
|
||||
|
||||
*注意*:启动服务时，Server的每个python后端进程默认申请`64M`内存，默认启动的docker无法启动多个python后端节点。有两个解决方案：
|
||||
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash`
|
||||
- 2.启动服务时设置python后端的`shm-default-byte-size`参数, 设置python后端的默认内存为10M： `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760`
|
||||
|
||||
### 分类任务
|
||||
在容器内执行下面命令启动服务:
|
||||
```
|
||||
# 默认启动models下所有模型
|
||||
fastdeployserver --model-repository=/models
|
||||
|
||||
# 可通过参数只启动分类任务
|
||||
fastdeployserver --model-repository=/models --model-control-mode=explicit --load-model=ernie_seqcls
|
||||
```
|
||||
输出打印如下:
|
||||
```
|
||||
I1019 09:41:15.375496 2823 model_repository_manager.cc:1183] successfully loaded 'ernie_tokenizer' version 1
|
||||
I1019 09:41:15.375987 2823 model_repository_manager.cc:1022] loading: ernie_seqcls:1
|
||||
I1019 09:41:15.477147 2823 model_repository_manager.cc:1183] successfully loaded 'ernie_seqcls' version 1
|
||||
I1019 09:41:15.477325 2823 server.cc:522]
|
||||
...
|
||||
I0613 08:59:20.577820 10021 server.cc:592]
|
||||
+----------------------------+---------+--------+
|
||||
| Model | Version | Status |
|
||||
+----------------------------+---------+--------+
|
||||
| ernie_seqcls | 1 | READY |
|
||||
| ernie_seqcls_model | 1 | READY |
|
||||
| ernie_seqcls_postprocess | 1 | READY |
|
||||
| ernie_tokenizer | 1 | READY |
|
||||
+----------------------------+---------+--------+
|
||||
...
|
||||
I0601 07:15:15.923270 8059 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0601 07:15:15.923604 8059 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0601 07:15:15.964984 8059 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
### 序列标注任务
|
||||
在容器内执行下面命令启动序列标注服务:
|
||||
```
|
||||
fastdeployserver --model-repository=/models --model-control-mode=explicit --load-model=ernie_tokencls --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
输出打印如下:
|
||||
```
|
||||
I1019 09:41:15.375496 2823 model_repository_manager.cc:1183] successfully loaded 'ernie_tokenizer' version 1
|
||||
I1019 09:41:15.375987 2823 model_repository_manager.cc:1022] loading: ernie_seqcls:1
|
||||
I1019 09:41:15.477147 2823 model_repository_manager.cc:1183] successfully loaded 'ernie_seqcls' version 1
|
||||
I1019 09:41:15.477325 2823 server.cc:522]
|
||||
...
|
||||
I0613 08:59:20.577820 10021 server.cc:592]
|
||||
+----------------------------+---------+--------+
|
||||
| Model | Version | Status |
|
||||
+----------------------------+---------+--------+
|
||||
| ernie_tokencls | 1 | READY |
|
||||
| ernie_tokencls_model | 1 | READY |
|
||||
| ernie_tokencls_postprocess | 1 | READY |
|
||||
| ernie_tokenizer | 1 | READY |
|
||||
+----------------------------+---------+--------+
|
||||
...
|
||||
I0601 07:15:15.923270 8059 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0601 07:15:15.923604 8059 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0601 07:15:15.964984 8059 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
## 客户端请求
|
||||
客户端请求可以在本地执行脚本请求；也可以在容器中执行。
|
||||
|
||||
本地执行脚本需要先安装依赖:
|
||||
```
|
||||
pip install grpcio
|
||||
pip install tritonclient[all]
|
||||
|
||||
# 如果bash无法识别括号，可以使用如下指令安装:
|
||||
pip install tritonclient\[all\]
|
||||
```
|
||||
|
||||
### 分类任务
|
||||
注意执行客户端请求时关闭代理，并根据实际情况修改main函数中的ip地址(启动服务所在的机器)
|
||||
```
|
||||
python seq_cls_grpc_client.py
|
||||
```
|
||||
输出打印如下:
|
||||
```
|
||||
{'label': array([5, 9]), 'confidence': array([0.6425664 , 0.66534853], dtype=float32)}
|
||||
{'label': array([4]), 'confidence': array([0.53198355], dtype=float32)}
|
||||
acc: 0.5731
|
||||
```
|
||||
|
||||
### 序列标注任务
|
||||
注意执行客户端请求时关闭代理，并根据实际情况修改main函数中的ip地址(启动服务所在的机器)
|
||||
```
|
||||
python token_cls_grpc_client.py
|
||||
```
|
||||
输出打印如下:
|
||||
```
|
||||
input data: 北京的涮肉，重庆的火锅，成都的小吃都是极具特色的美食。
|
||||
The model detects all entities:
|
||||
entity: 北京 label: LOC pos: [0, 1]
|
||||
entity: 重庆 label: LOC pos: [6, 7]
|
||||
entity: 成都 label: LOC pos: [12, 13]
|
||||
input data: 原产玛雅故国的玉米，早已成为华夏大地主要粮食作物之一。
|
||||
The model detects all entities:
|
||||
entity: 玛雅 label: LOC pos: [2, 3]
|
||||
entity: 华夏 label: LOC pos: [14, 15]
|
||||
```
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前分类任务(ernie_seqcls_model/config.pbtxt)默认配置在CPU上运行OpenVINO引擎; 序列标注任务默认配置在GPU上运行Paddle引擎。如果要在CPU/GPU或其他推理引擎上运行, 需要修改配置，详情请参考[配置文档](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
@@ -1,40 +1 @@
|
||||
# 通用信息抽取 UIE模型部署
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleNLP 通用信息抽取UIE](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie)
|
||||
|
||||
## 支持的模型列表
|
||||
|
||||
| 模型 | 结构 | 语言 |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `uie-base`| 12-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `uie-medium`| 6-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `uie-mini`| 6-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `uie-micro`| 4-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `uie-nano`| 4-layers, 312-hidden, 12-heads | 中文 |
|
||||
|
||||
|
||||
## 导出部署模型
|
||||
|
||||
在部署前，需要先将UIE模型导出成部署模型，导出步骤参考文档[导出模型](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie#47-%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了UIE导出的各模型，开发者可直接下载使用。
|
||||
|
||||
其中精度指标F1来源于PaddleNLP中对各模型在互联网自建数据集进行0-shot的实验，详情各参考PaddleNLP中的说明。
|
||||
|
||||
| 模型 | 参数大小 | F1 值|
|
||||
|:---------------------------------------------------------------- |:----- |:----- |
|
||||
|[uie-base](https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz)| 416M | 78.33 |
|
||||
|[uie-medium](https://bj.bcebos.com/fastdeploy/models/uie/uie-medium.tgz)| 265M | 78.32 |
|
||||
|[uie-mini](https://bj.bcebos.com/fastdeploy/models/uie/uie-mini.tgz)| 95M | 72.09 |
|
||||
|[uie-micro](https://bj.bcebos.com/fastdeploy/models/uie/uie-micro.tgz)| 83M | 66.00 |
|
||||
|[uie-nano](https://bj.bcebos.com/fastdeploy/models/uie/uie-nano.tgz)| 64M | 62.86 |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [服务化部署](serving)
|
||||
README_EN.md
|
||||
|
||||
42
examples/text/uie/README_CN.md
Normal file
42
examples/text/uie/README_CN.md
Normal file
@@ -0,0 +1,42 @@
|
||||
[English](README_EN.md) | 简体中文
|
||||
|
||||
# 通用信息抽取 UIE模型部署
|
||||
|
||||
## 模型版本说明
|
||||
|
||||
- [PaddleNLP 通用信息抽取UIE](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie)
|
||||
|
||||
## 支持的模型列表
|
||||
|
||||
| 模型 | 结构 | 语言 |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `uie-base`| 12-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `uie-medium`| 6-layers, 768-hidden, 12-heads | 中文 |
|
||||
| `uie-mini`| 6-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `uie-micro`| 4-layers, 384-hidden, 12-heads | 中文 |
|
||||
| `uie-nano`| 4-layers, 312-hidden, 12-heads | 中文 |
|
||||
|
||||
|
||||
## 导出部署模型
|
||||
|
||||
在部署前，需要先将UIE模型导出成部署模型，导出步骤参考文档[导出模型](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie#47-%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)
|
||||
|
||||
## 下载预训练模型
|
||||
|
||||
为了方便开发者的测试，下面提供了UIE导出的各模型，开发者可直接下载使用。
|
||||
|
||||
其中精度指标F1来源于PaddleNLP中对各模型在互联网自建数据集进行0-shot的实验，详情各参考PaddleNLP中的说明。
|
||||
|
||||
| 模型 | 参数大小 | F1 值|
|
||||
|:---------------------------------------------------------------- |:----- |:----- |
|
||||
|[uie-base](https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz)| 416M | 78.33 |
|
||||
|[uie-medium](https://bj.bcebos.com/fastdeploy/models/uie/uie-medium.tgz)| 265M | 78.32 |
|
||||
|[uie-mini](https://bj.bcebos.com/fastdeploy/models/uie/uie-mini.tgz)| 95M | 72.09 |
|
||||
|[uie-micro](https://bj.bcebos.com/fastdeploy/models/uie/uie-micro.tgz)| 83M | 66.00 |
|
||||
|[uie-nano](https://bj.bcebos.com/fastdeploy/models/uie/uie-nano.tgz)| 64M | 62.86 |
|
||||
|
||||
## 详细部署文档
|
||||
|
||||
- [Python部署](python)
|
||||
- [C++部署](cpp)
|
||||
- [服务化部署](serving)
|
||||
42
examples/text/uie/README_EN.md
Normal file
42
examples/text/uie/README_EN.md
Normal file
@@ -0,0 +1,42 @@
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
# Universal Information Extraction UIE Model Deployment
|
||||
|
||||
## Introduction to the Model Version
|
||||
|
||||
- [PaddleNLP 通用信息抽取UIE](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie)
|
||||
|
||||
## A List of Supported Models
|
||||
|
||||
| Model | Structure | Language |
|
||||
| :---: | :--------: | :--------: |
|
||||
| `uie-base`| 12-layers, 768-hidden, 12-heads | Chinese |
|
||||
| `uie-medium`| 6-layers, 768-hidden, 12-heads | Chinese |
|
||||
| `uie-mini`| 6-layers, 384-hidden, 12-heads | Chinese |
|
||||
| `uie-micro`| 4-layers, 384-hidden, 12-heads | Chinese |
|
||||
| `uie-nano`| 4-layers, 312-hidden, 12-heads | Chinese |
|
||||
|
||||
|
||||
## Export Deployment Models
|
||||
|
||||
Before deployment, you need to export the UIE model into the deployment model. Please refer to [导出模型](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.4/model_zoo/uie#47-%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)
|
||||
|
||||
## Download Pre-trained Models
|
||||
|
||||
Models exported by UIE are provided below for developers' testing. Developers can directly download them.
|
||||
|
||||
The accuracy metric F1 is derived from the 0-shot experiments conducted in PaddleNLP on each model in the Internet self-built dataset. Please refer to instructions in PaddleNLP for more details.
|
||||
|
||||
| Model | Parameter Size | F1 Value|
|
||||
|:---------------------------------------------------------------- |:----- |:----- |
|
||||
|[uie-base](https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz)| 416M | 78.33 |
|
||||
|[uie-medium](https://bj.bcebos.com/fastdeploy/models/uie/uie-medium.tgz)| 265M | 78.32 |
|
||||
|[uie-mini](https://bj.bcebos.com/fastdeploy/models/uie/uie-mini.tgz)| 95M | 72.09 |
|
||||
|[uie-micro](https://bj.bcebos.com/fastdeploy/models/uie/uie-micro.tgz)| 83M | 66.00 |
|
||||
|[uie-nano](https://bj.bcebos.com/fastdeploy/models/uie/uie-nano.tgz)| 64M | 62.86 |
|
||||
|
||||
## Detailed Deployment Documents
|
||||
|
||||
- [Python Deployment](python)
|
||||
- [C++ Deployment](cpp)
|
||||
- [Serve Deployment](serving)
|
||||
@@ -1,40 +1,42 @@
|
||||
# 通用信息抽取 UIE C++部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
本目录下提供`infer.cc`快速完成[UIE模型](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/uie)在CPU/GPU的示例。
|
||||
# Universal Information Extraction UIE C++ Deployment Example
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
This directory provides `infer.cc` quickly complete the example on CPU/GPU by [UIE Model](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/uie)
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
Before deployment, two steps need to be confirmed.
|
||||
|
||||
## 快速开始
|
||||
以Linux上uie-base模型推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)。
|
||||
- 1. The software and hardware environment meets the requirements. Please refer to [FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. Download precompiled deployment library and samples code based on the develop environment. Please refer to [FastDeploy预编译库](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
## A Quick Start
|
||||
Take uie-base model inference on Linux as an example, execute the following command in this directory to complete the compilation test. FastDeploy version 0.7.0 or above is required to support this model (x.x.x>=0.7.0).
|
||||
|
||||
```
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
# Download FastDeploy precompiled library. Users can choose proper versions in the `FastDeploy预编译库` mentioned above.
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载uie-base模型以及词表
|
||||
# Download the uie-base model and vocabulary
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
|
||||
# CPU 推理
|
||||
# CPU Inference
|
||||
./infer_demo uie-base 0
|
||||
|
||||
# GPU 推理
|
||||
# GPU Inference
|
||||
./infer_demo uie-base 1
|
||||
|
||||
# 使用OpenVINO推理
|
||||
# Use OpenVINO for inference
|
||||
./infer_demo uie-base 1 2
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示(仅截取NER任务的输出)。
|
||||
The results after running are as follows (only the output of the NER task is captured).
|
||||
```bash
|
||||
[INFO] fastdeploy/fastdeploy_runtime.cc(264)::Init Runtime initialized with Backend::PDINFER in device Device::CPU.
|
||||
After init predictor
|
||||
@@ -58,11 +60,11 @@ The result:
|
||||
end: 31
|
||||
```
|
||||
|
||||
## UIE模型各抽取任务使用方式
|
||||
## The way to use the UIE model in each extraction task
|
||||
|
||||
在UIE模型中，schema代表要抽取的结构化信息，所以UIE模型可通过设置不同的schema支持不同信息抽取任务。
|
||||
In the UIE model, schema represents the structured information to be extracted, so the UIE model can support different information extraction tasks by setting different schemas.
|
||||
|
||||
### 初始化UIEModel
|
||||
### Initialize UIEModel
|
||||
|
||||
```c++
|
||||
std::string model_dir = "uie-base";
|
||||
@@ -80,9 +82,9 @@ auto predictor =
|
||||
{"时间", "选手", "赛事名称"}, option);
|
||||
```
|
||||
|
||||
### 实体抽取
|
||||
### Entity Extraction
|
||||
|
||||
初始化阶段将schema设置为```["时间", "选手", "赛事名称"]```，可对输入的文本抽取时间、选手以及赛事名称三个信息。
|
||||
The initialization stage sets the schema```["time", "player", "event name"]``` to extract the time, player and event name from the input text.
|
||||
|
||||
```c++
|
||||
// Named Entity Recognition
|
||||
@@ -113,7 +115,7 @@ results.clear();
|
||||
// end: 31
|
||||
```
|
||||
|
||||
例如抽取的目标实体类型是"肿瘤的大小"、"肿瘤的个数"、"肝癌级别"和"脉管内癌栓分级", 则可编写如下语句：
|
||||
For example, if the target entity types are "肿瘤的大小", "肿瘤的个数", "肝癌级别" and "脉管内癌栓分级", the following statements can be executed.
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(
|
||||
@@ -154,11 +156,11 @@ results.clear();
|
||||
|
||||
```
|
||||
|
||||
### 关系抽取
|
||||
### Relation Extraction
|
||||
|
||||
关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
|
||||
Relation Extraction (RE) refers to identifying entities from text and extracting semantic relationships between them to obtain triadic information, i.e. <subject, predicate, object>.
|
||||
|
||||
例如以"竞赛名称"作为抽取主体，抽取关系类型为"主办方"、"承办方"和"已举办次数", 则可编写如下语句：
|
||||
For example, if we take "contest name" as the extracted entity, and the relations are "主办方", "承办方" and "已举办次数", then we can write the following statements.
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(
|
||||
@@ -214,11 +216,11 @@ results.clear();
|
||||
// end: 72
|
||||
```
|
||||
|
||||
### 事件抽取
|
||||
### Event Extraction
|
||||
|
||||
事件抽取 (Event Extraction, 简称EE)，是指从自然语言文本中抽取预定义的事件触发词(Trigger)和事件论元(Argument)，组合为相应的事件结构化信息。
|
||||
Event Extraction (EE) refers to extracting predefined Trigger and Argument from natural language texts and combining them into structured event information.
|
||||
|
||||
例如抽取的目标是"地震"事件的"地震强度"、"时间"、"震中位置"和"震源深度"这些信息，则可编写如下代码：
|
||||
For example, if the targets are"地震强度", "时间", "震中位置" and "引源深度" for the event "地震", we can execute the following codes.
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode(
|
||||
@@ -264,11 +266,11 @@ results.clear();
|
||||
// end: 22
|
||||
```
|
||||
|
||||
### 评论观点抽取
|
||||
### Opinion Extraction
|
||||
|
||||
评论观点抽取，是指抽取文本中包含的评价维度、观点词。
|
||||
opinion extraction refers to the extraction of evaluation dimensions and opinions contained in the text.
|
||||
|
||||
例如抽取的目标是文本中包含的评价维度及其对应的观点词和情感倾向，可编写以下代码：
|
||||
For example, if the extraction target is the evaluation dimensions and their corresponding opinions and sentiment tendencies. We can execute the following codes：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode(
|
||||
@@ -319,9 +321,10 @@ results.clear();
|
||||
// end: 22
|
||||
```
|
||||
|
||||
### 情感分类
|
||||
### Sentiment Classification
|
||||
|
||||
Sentence-level sentiment classification, i.e., determining a sentence has a "positive" sentiment or "negative" sentiment. We can execute the following codes:
|
||||
|
||||
句子级情感倾向分类，即判断句子的情感倾向是“正向”还是“负向”，可编写以下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(SchemaNode("情感倾向[正向，负向]"));
|
||||
@@ -336,9 +339,10 @@ results.clear();
|
||||
// probability: 0.999002
|
||||
```
|
||||
|
||||
### 跨任务抽取
|
||||
### Cross-task Extraction
|
||||
|
||||
or example, in a legal scenario where both entity extraction and relation extraction need to be performed. We can execute the following codes:
|
||||
|
||||
例如在法律场景同时对文本进行实体抽取和关系抽取，可编写以下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode("法院", {}),
|
||||
@@ -384,24 +388,24 @@ results.clear();
|
||||
// end: 46
|
||||
```
|
||||
|
||||
## UIEModel C++接口
|
||||
## UIEModel C++ Interface
|
||||
|
||||
### SchemaNode 结构
|
||||
表示UIE模型目标模式的结构。
|
||||
### SchemaNode Structure
|
||||
Represent the structure of UIE model target mode.
|
||||
|
||||
```c++
|
||||
SchemaNode(const std::string& name,
|
||||
const std::vector<SchemaNode>& children = {});
|
||||
```
|
||||
**参数**
|
||||
**Parameter**
|
||||
|
||||
> * **name**(str): 需要抽取的信息。
|
||||
> * **children**(str): 当前节点需抽取信息关联的子信息。
|
||||
> * **name**(str): information requiring extraction.
|
||||
> * **children**(str): the current node needs to extract the sub-information associated with the original information.
|
||||
|
||||
### UIEModel 结构
|
||||
用于信息抽取任务的UIE模型结构。
|
||||
### UIEModel Structure
|
||||
The UIE model structure for information extraction task.
|
||||
|
||||
#### 初始化函数
|
||||
#### Initialized Function
|
||||
```c++
|
||||
UIEModel(
|
||||
const std::string& model_file, const std::string& params_file,
|
||||
@@ -429,21 +433,21 @@ UIEModel(
|
||||
SchemaLanguage schema_language = SchemaLanguage::ZH);
|
||||
```
|
||||
|
||||
UIE模型加载和初始化，其中model_file, params_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)。
|
||||
UIEModel loading and initialization. Among them, model_file, params_file are Paddle inference documents exported by trained models. Please refer to [模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)。
|
||||
|
||||
**参数**
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **vocab_file**(str): 词表文件路径
|
||||
> * **position_prob**(str): 位置概率，模型将输出位置概率大于`position_prob`的位置，默认为0.5
|
||||
> * **max_length**(int): 输入文本的最大长度。输入文本下标超过`max_length`的部分将被截断。默认为128
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): 抽取任务的目标模式。
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
> * **schema_language** (SchemaLanguage): Schema 语言，默认为ZH（中文），目前支持的语言种类包括：ZH（中文），EN（英文）。
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **vocab_file**(str): Vocabulary file
|
||||
> * **position_prob**(str): Position probability. The model will output positions with probability greater than `position_prob`, default is 0.5
|
||||
> * **max_length**(int): Maximized length of input text. Input text subscript exceeding `max_length` will be truncated. Default is 128
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): Target information for extraction tasks
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration, the default is None, i.e., the default configuration
|
||||
> * **model_format**(ModelFormat): Model format, and default is Paddle format
|
||||
> * **schema_language** (SchemaLanguage): Schema language, and default is ZH（Chinese）. Currently supported language：ZH（Chinese），EN（English）
|
||||
|
||||
#### SetSchema函数
|
||||
#### SetSchema Function
|
||||
|
||||
```c++
|
||||
void SetSchema(const std::vector<std::string>& schema);
|
||||
@@ -451,21 +455,22 @@ void SetSchema(const std::vector<SchemaNode>& schema);
|
||||
void SetSchema(const SchemaNode& schema);
|
||||
```
|
||||
|
||||
**参数**
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): 输入数据，待抽取文本模式。
|
||||
**Parameter**
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): Input data, in a text pattern to be extracted.
|
||||
|
||||
#### Predict函数
|
||||
#### Predict Function
|
||||
|
||||
```c++
|
||||
void Predict(
|
||||
const std::vector<std::string>& texts,
|
||||
std::vector<std::unordered_map<std::string, std::vector<UIEResult>>>* results);
|
||||
```
|
||||
**参数**
|
||||
**Parameter**
|
||||
|
||||
> * **texts**(list(str)): 文本列表
|
||||
> * **results**(list(dict())): UIE模型抽取结果。
|
||||
## 相关文档
|
||||
> * **texts**(list(str)): text list
|
||||
> * **results**(list(dict())): UIE model extraction results
|
||||
|
||||
## Related Documents
|
||||
|
||||
[UIE模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md)
|
||||
|
||||
|
||||
475
examples/text/uie/cpp/README_CN.md
Normal file
475
examples/text/uie/cpp/README_CN.md
Normal file
@@ -0,0 +1,475 @@
|
||||
[English](README.md) | 简体中文
|
||||
# 通用信息抽取 UIE C++部署示例
|
||||
|
||||
本目录下提供`infer.cc`快速完成[UIE模型](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/uie)在CPU/GPU的示例。
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. 根据开发环境，下载预编译部署库和samples代码，参考[FastDeploy预编译库](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
## 快速开始
|
||||
以Linux上uie-base模型推理为例，在本目录执行如下命令即可完成编译测试，支持此模型需保证FastDeploy版本0.7.0以上(x.x.x>=0.7.0)。
|
||||
|
||||
```
|
||||
mkdir build
|
||||
cd build
|
||||
# 下载FastDeploy预编译库，用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
|
||||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
|
||||
tar xvf fastdeploy-linux-x64-x.x.x.tgz
|
||||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
|
||||
make -j
|
||||
|
||||
# 下载uie-base模型以及词表
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
|
||||
# CPU 推理
|
||||
./infer_demo uie-base 0
|
||||
|
||||
# GPU 推理
|
||||
./infer_demo uie-base 1
|
||||
|
||||
# 使用OpenVINO推理
|
||||
./infer_demo uie-base 1 2
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示(仅截取NER任务的输出)。
|
||||
```bash
|
||||
[INFO] fastdeploy/fastdeploy_runtime.cc(264)::Init Runtime initialized with Backend::PDINFER in device Device::CPU.
|
||||
After init predictor
|
||||
The result:
|
||||
赛事名称:
|
||||
text: 北京冬奥会自由式滑雪女子大跳台决赛
|
||||
probability: 0.850309
|
||||
start: 6
|
||||
end: 23
|
||||
|
||||
时间:
|
||||
text: 2月8日上午
|
||||
probability: 0.985738
|
||||
start: 0
|
||||
end: 6
|
||||
|
||||
选手:
|
||||
text: 谷爱凌
|
||||
probability: 0.898155
|
||||
start: 28
|
||||
end: 31
|
||||
```
|
||||
|
||||
## UIE模型各抽取任务使用方式
|
||||
|
||||
在UIE模型中，schema代表要抽取的结构化信息，所以UIE模型可通过设置不同的schema支持不同信息抽取任务。
|
||||
|
||||
### 初始化UIEModel
|
||||
|
||||
```c++
|
||||
std::string model_dir = "uie-base";
|
||||
std::string model_path = model_dir + sep + "inference.pdmodel";
|
||||
std::string param_path = model_dir + sep + "inference.pdiparams";
|
||||
std::string vocab_path = model_dir + sep + "vocab.txt";
|
||||
using fastdeploy::text::SchemaNode;
|
||||
using fastdeploy::text::UIEResult;
|
||||
// 定义uie result对象
|
||||
std::vector<std::unordered_map<std::string, std::vector<UIEResult>>> results;
|
||||
|
||||
// 初始化UIE模型
|
||||
auto predictor =
|
||||
fastdeploy::text::UIEModel(model_path, param_path, vocab_path, 0.5, 128,
|
||||
{"时间", "选手", "赛事名称"}, option);
|
||||
```
|
||||
|
||||
### 实体抽取
|
||||
|
||||
初始化阶段将schema设置为```["时间", "选手", "赛事名称"]```，可对输入的文本抽取时间、选手以及赛事名称三个信息。
|
||||
|
||||
```c++
|
||||
// Named Entity Recognition
|
||||
predictor.Predict({"2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷"
|
||||
"爱凌以188.25分获得金牌！"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 赛事名称:
|
||||
// text: 北京冬奥会自由式滑雪女子大跳台决赛
|
||||
// probability: 0.850309
|
||||
// start: 6
|
||||
// end: 23
|
||||
//
|
||||
// 时间:
|
||||
// text: 2月8日上午
|
||||
// probability: 0.985738
|
||||
// start: 0
|
||||
// end: 6
|
||||
//
|
||||
// 选手:
|
||||
// text: 谷爱凌
|
||||
// probability: 0.898155
|
||||
// start: 28
|
||||
// end: 31
|
||||
```
|
||||
|
||||
例如抽取的目标实体类型是"肿瘤的大小"、"肿瘤的个数"、"肝癌级别"和"脉管内癌栓分级", 则可编写如下语句：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(
|
||||
{"肿瘤的大小", "肿瘤的个数", "肝癌级别", "脉管内癌栓分级"});
|
||||
predictor.Predict({"（右肝肿瘤）肝细胞性肝癌（II-"
|
||||
"III级，梁索型和假腺管型），肿瘤包膜不完整，紧邻肝被膜，侵"
|
||||
"及周围肝组织，未见脉管内癌栓（MVI分级：M0级）及卫星子灶形"
|
||||
"成。（肿物1个，大小4.2×4.0×2.8cm）。"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 脉管内癌栓分级:
|
||||
// text: M0级
|
||||
// probability: 0.908329
|
||||
// start: 67
|
||||
// end: 70
|
||||
//
|
||||
// 肝癌级别:
|
||||
// text: II-III级
|
||||
// probability: 0.924327
|
||||
// start: 13
|
||||
// end: 20
|
||||
//
|
||||
// 肿瘤的大小:
|
||||
// text: 4.2×4.0×2.8cm
|
||||
// probability: 0.834113
|
||||
// start: 87
|
||||
// end: 100
|
||||
//
|
||||
// 肿瘤的个数:
|
||||
// text: 1个
|
||||
// probability: 0.753841
|
||||
// start: 82
|
||||
// end: 84
|
||||
|
||||
```
|
||||
|
||||
### 关系抽取
|
||||
|
||||
关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
|
||||
|
||||
例如以"竞赛名称"作为抽取主体，抽取关系类型为"主办方"、"承办方"和"已举办次数", 则可编写如下语句：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(
|
||||
{SchemaNode("竞赛名称", {SchemaNode("主办方"), SchemaNode("承办方"),
|
||||
SchemaNode("已举办次数")})});
|
||||
predictor.Predict(
|
||||
{"2022语言与智能技术竞赛由中国中文信息学会和中国计算机学会联合主办，百度"
|
||||
"公司、中国中文信息学会评测工作委员会和中国计算机学会自然语言处理专委会"
|
||||
"承办，已连续举办4届，成为全球最热门的中文NLP赛事之一。"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 竞赛名称:
|
||||
// text: 2022语言与智能技术竞赛
|
||||
// probability: 0.78254
|
||||
// start: 0
|
||||
// end: 13
|
||||
// relation:
|
||||
// 已举办次数:
|
||||
// text: 4届
|
||||
// probability: 0.46713
|
||||
// start: 80
|
||||
// end: 82
|
||||
//
|
||||
// 主办方:
|
||||
// text: 中国中文信息学会
|
||||
// probability: 0.842172
|
||||
// start: 14
|
||||
// end: 22
|
||||
//
|
||||
// text: 中国计算机学会
|
||||
// probability: 0.758081
|
||||
// start: 23
|
||||
// end: 30
|
||||
//
|
||||
// 承办方:
|
||||
// text: 百度公司
|
||||
// probability: 0.829271
|
||||
// start: 35
|
||||
// end: 39
|
||||
//
|
||||
// text: 中国中文信息学会评测工作委员会
|
||||
// probability: 0.70005
|
||||
// start: 40
|
||||
// end: 55
|
||||
//
|
||||
// text: 中国计算机学会自然语言处理专委会
|
||||
// probability: 0.619348
|
||||
// start: 56
|
||||
// end: 72
|
||||
```
|
||||
|
||||
### 事件抽取
|
||||
|
||||
事件抽取 (Event Extraction, 简称EE)，是指从自然语言文本中抽取预定义的事件触发词(Trigger)和事件论元(Argument)，组合为相应的事件结构化信息。
|
||||
|
||||
例如抽取的目标是"地震"事件的"地震强度"、"时间"、"震中位置"和"震源深度"这些信息，则可编写如下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode(
|
||||
"地震触发词", {SchemaNode("地震强度"), SchemaNode("时间"),
|
||||
SchemaNode("震中位置"), SchemaNode("震源深度")})});
|
||||
predictor.Predict(
|
||||
{"中国地震台网正式测定：5月16日06时08分在云南临沧市凤庆县(北纬24."
|
||||
"34度，东经99.98度)发生3.5级地震，震源深度10千米。"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 地震触发词:
|
||||
// text: 地震
|
||||
// probability: 0.997743
|
||||
// start: 56
|
||||
// end: 58
|
||||
// relation:
|
||||
// 震源深度:
|
||||
// text: 10千米
|
||||
// probability: 0.993797
|
||||
// start: 63
|
||||
// end: 67
|
||||
//
|
||||
// 震中位置:
|
||||
// text: 云南临沧市凤庆县(北纬24.34度，东经99.98度)
|
||||
// probability: 0.787402
|
||||
// start: 23
|
||||
// end: 50
|
||||
//
|
||||
// 地震强度:
|
||||
// text: 3.5级
|
||||
// probability: 0.99808
|
||||
// start: 52
|
||||
// end: 56
|
||||
//
|
||||
// 时间:
|
||||
// text: 5月16日06时08分
|
||||
// probability: 0.98533
|
||||
// start: 11
|
||||
// end: 22
|
||||
```
|
||||
|
||||
### 评论观点抽取
|
||||
|
||||
评论观点抽取，是指抽取文本中包含的评价维度、观点词。
|
||||
|
||||
例如抽取的目标是文本中包含的评价维度及其对应的观点词和情感倾向，可编写以下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode(
|
||||
"评价维度",
|
||||
// NOTE(zhoushunjie): It's necessary to explicitly use
|
||||
// std::vector to convert initializer list of SchemaNode whose size is
|
||||
// two. If not to do so, an ambiguous compliation error will occur in
|
||||
// mac x64 platform.
|
||||
std::vector<SchemaNode>{SchemaNode("观点词"),
|
||||
SchemaNode("情感倾向[正向，负向]")})});
|
||||
predictor.Predict(
|
||||
{"店面干净，很清静，服务员服务热情，性价比很高，发现收银台有排队"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 评价维度:
|
||||
// text: 店面
|
||||
// probability: 0.969685
|
||||
// start: 0
|
||||
// end: 2
|
||||
// relation:
|
||||
// 情感倾向[正向，负向]:
|
||||
// text: 正向
|
||||
// probability: 0.998215
|
||||
//
|
||||
// 观点词:
|
||||
// text: 干净
|
||||
// probability: 0.994532
|
||||
// start: 2
|
||||
// end: 4
|
||||
//
|
||||
// text: 性价比
|
||||
// probability: 0.981704
|
||||
// start: 17
|
||||
// end: 20
|
||||
// relation:
|
||||
// 情感倾向[正向，负向]:
|
||||
// text: 正向
|
||||
// probability: 0.996614
|
||||
//
|
||||
// 观点词:
|
||||
// text: 高
|
||||
// probability: 0.957397
|
||||
// start: 21
|
||||
// end: 22
|
||||
```
|
||||
|
||||
### 情感分类
|
||||
|
||||
句子级情感倾向分类，即判断句子的情感倾向是“正向”还是“负向”，可编写以下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema(SchemaNode("情感倾向[正向，负向]"));
|
||||
predictor.Predict({"这个产品用起来真的很流畅，我非常喜欢"}, &results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 情感倾向[正向，负向]:
|
||||
// text: 正向
|
||||
// probability: 0.999002
|
||||
```
|
||||
|
||||
### 跨任务抽取
|
||||
|
||||
例如在法律场景同时对文本进行实体抽取和关系抽取，可编写以下代码：
|
||||
|
||||
```c++
|
||||
predictor.SetSchema({SchemaNode("法院", {}),
|
||||
SchemaNode("原告", {SchemaNode("委托代理人")}),
|
||||
SchemaNode("被告", {SchemaNode("委托代理人")})});
|
||||
predictor.Predict({"北京市海淀区人民法院\n民事判决书\n(199x)"
|
||||
"建初字第xxx号\n原告：张三。\n委托代理人李四，北京市 "
|
||||
"A律师事务所律师。\n被告：B公司，法定代表人王五，开发公司"
|
||||
"总经理。\n委托代理人赵六，北京市 C律师事务所律师。"},
|
||||
&results);
|
||||
std::cout << results << std::endl;
|
||||
results.clear();
|
||||
// 示例输出
|
||||
// The result:
|
||||
// 被告:
|
||||
// text: B公司
|
||||
// probability: 0.843735
|
||||
// start: 64
|
||||
// end: 67
|
||||
// relation:
|
||||
// 委托代理人:
|
||||
// text: 赵六
|
||||
// probability: 0.726712
|
||||
// start: 90
|
||||
// end: 92
|
||||
//
|
||||
// 法院:
|
||||
// text: 北京市海淀区人民法院
|
||||
// probability: 0.922107
|
||||
// start: 0
|
||||
// end: 10
|
||||
//
|
||||
// 原告:
|
||||
// text: 张三
|
||||
// probability: 0.994981
|
||||
// start: 35
|
||||
// end: 37
|
||||
// relation:
|
||||
// 委托代理人:
|
||||
// text: 李四
|
||||
// probability: 0.795686
|
||||
// start: 44
|
||||
// end: 46
|
||||
```
|
||||
|
||||
## UIEModel C++接口
|
||||
|
||||
### SchemaNode 结构
|
||||
表示UIE模型目标模式的结构。
|
||||
|
||||
```c++
|
||||
SchemaNode(const std::string& name,
|
||||
const std::vector<SchemaNode>& children = {});
|
||||
```
|
||||
**参数**
|
||||
|
||||
> * **name**(str): 需要抽取的信息。
|
||||
> * **children**(str): 当前节点需抽取信息关联的子信息。
|
||||
|
||||
### UIEModel 结构
|
||||
用于信息抽取任务的UIE模型结构。
|
||||
|
||||
#### 初始化函数
|
||||
```c++
|
||||
UIEModel(
|
||||
const std::string& model_file, const std::string& params_file,
|
||||
const std::string& vocab_file, float position_prob, size_t max_length,
|
||||
const std::vector<std::string>& schema,
|
||||
const fastdeploy::RuntimeOption& custom_option =
|
||||
fastdeploy::RuntimeOption(),
|
||||
const fastdeploy::ModelFormat& model_format = fastdeploy::ModelFormat::PADDLE,
|
||||
SchemaLanguage schema_language = SchemaLanguage::ZH);
|
||||
UIEModel(
|
||||
const std::string& model_file, const std::string& params_file,
|
||||
const std::string& vocab_file, float position_prob, size_t max_length,
|
||||
const SchemaNode& schema, const fastdeploy::RuntimeOption& custom_option =
|
||||
fastdeploy::RuntimeOption(),
|
||||
const fastdeploy::ModelFormat& model_format = fastdeploy::ModelFormat::PADDLE,
|
||||
SchemaLanguage schema_language = SchemaLanguage::ZH);
|
||||
UIEModel(
|
||||
const std::string& model_file, const std::string& params_file,
|
||||
const std::string& vocab_file, float position_prob, size_t max_length,
|
||||
const std::vector<SchemaNode>& schema,
|
||||
const fastdeploy::RuntimeOption& custom_option =
|
||||
fastdeploy::RuntimeOption(),
|
||||
const fastdeploy::ModelFormat& model_format =
|
||||
fastdeploy::ModelFormat::PADDLE,
|
||||
SchemaLanguage schema_language = SchemaLanguage::ZH);
|
||||
```
|
||||
|
||||
UIE模型加载和初始化，其中model_file, params_file为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)。
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **vocab_file**(str): 词表文件路径
|
||||
> * **position_prob**(str): 位置概率，模型将输出位置概率大于`position_prob`的位置，默认为0.5
|
||||
> * **max_length**(int): 输入文本的最大长度。输入文本下标超过`max_length`的部分将被截断。默认为128
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): 抽取任务的目标模式。
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
> * **schema_language** (SchemaLanguage): Schema 语言，默认为ZH（中文），目前支持的语言种类包括：ZH（中文），EN（英文）。
|
||||
|
||||
#### SetSchema函数
|
||||
|
||||
```c++
|
||||
void SetSchema(const std::vector<std::string>& schema);
|
||||
void SetSchema(const std::vector<SchemaNode>& schema);
|
||||
void SetSchema(const SchemaNode& schema);
|
||||
```
|
||||
|
||||
**参数**
|
||||
> * **schema**(list(SchemaNode) | SchemaNode | list(str)): 输入数据，待抽取文本模式。
|
||||
|
||||
#### Predict函数
|
||||
|
||||
```c++
|
||||
void Predict(
|
||||
const std::vector<std::string>& texts,
|
||||
std::vector<std::unordered_map<std::string, std::vector<UIEResult>>>* results);
|
||||
```
|
||||
**参数**
|
||||
|
||||
> * **texts**(list(str)): 文本列表
|
||||
> * **results**(list(dict())): UIE模型抽取结果。
|
||||
## 相关文档
|
||||
|
||||
[UIE模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md)
|
||||
|
||||
[UIE模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)
|
||||
|
||||
[UIE Python部署方法](../python/README.md)
|
||||
@@ -1,32 +1,34 @@
|
||||
# 通用信息抽取 UIE Python部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
# Universal Information Extraction UIE Python Deployment Example
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
Before deployment, two steps need to be confirmed.
|
||||
|
||||
本目录下提供`infer.py`快速完成UIE模型在CPU/GPU，以及CPU上通过OpenVINO加速CPU端部署示例。执行如下脚本即可完成。
|
||||
- 1. The software and hardware environment meets the requirements. Please refer to [FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl pacakage needs installation. Please refer to [FastDeploy Python安装](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
## 快速开始
|
||||
This directory provides an example that `infer.py` quickly complete CPU deployment conducted by the UIE model with OpenVINO acceleration on CPU/GPU and CPU.
|
||||
|
||||
## A Quick Start
|
||||
```bash
|
||||
|
||||
#下载部署示例代码
|
||||
# Download deployment sample code
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/text/uie/python
|
||||
|
||||
# 下载UIE模型文件和词表，以uie-base模型为例
|
||||
# Download the UIE model file and word list. Taking the uie-base model as an example.
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
# CPU推理
|
||||
# CPU Inference
|
||||
python infer.py --model_dir uie-base --device cpu
|
||||
# GPU推理
|
||||
# GPU Inference
|
||||
python infer.py --model_dir uie-base --device gpu
|
||||
# 使用OpenVINO推理
|
||||
# Use OpenVINO for inference
|
||||
python infer.py --model_dir uie-base --device cpu --backend openvino --cpu_num_threads 8
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示(仅截取NER任务的输出)。
|
||||
The results after running are as follows(only the output of the NER task is captured).
|
||||
```bash
|
||||
1. Named Entity Recognition Task
|
||||
The extraction schema: ['时间', '选手', '赛事名称']
|
||||
@@ -63,11 +65,11 @@ The extraction schema: ['肿瘤的大小', '肿瘤的个数', '肝癌级别', '
|
||||
......
|
||||
```
|
||||
|
||||
## UIE模型各抽取任务使用方式
|
||||
## The way to use the UIE model in each extraction task
|
||||
|
||||
在UIE模型中，schema代表要抽取的结构化信息，所以UIE模型可通过设置不同的schema支持不同信息抽取任务。
|
||||
In the UIE model, schema represents the structured information to be extracted, so the UIE model can support different information extraction tasks by setting different schemas.
|
||||
|
||||
### 初始化UIEModel
|
||||
### Initialize UIEModel
|
||||
|
||||
```python
|
||||
import fastdeploy
|
||||
@@ -91,9 +93,9 @@ uie = UIEModel(
|
||||
runtime_option=runtime_option)
|
||||
```
|
||||
|
||||
### 实体抽取
|
||||
### Entity Extraction
|
||||
|
||||
初始化阶段将schema设置为```["时间", "选手", "赛事名称"]```，可对输入的文本抽取时间、选手以及赛事名称三个信息。
|
||||
The initialization stage sets the schema```["time", "player", "event name"]``` to extract the time, player and event name from the input text.
|
||||
|
||||
```python
|
||||
>>> from pprint import pprint
|
||||
@@ -117,7 +119,7 @@ uie = UIEModel(
|
||||
|
||||
```
|
||||
|
||||
例如抽取的目标实体类型是"肿瘤的大小"、"肿瘤的个数"、"肝癌级别"和"脉管内癌栓分级", 则可执行如下语句：
|
||||
For example, if the target entity types are "肿瘤的大小", "肿瘤的个数", "肝癌级别" and "脉管内癌栓分级", the following statements can be executed.
|
||||
|
||||
```python
|
||||
>>> schema = ["肿瘤的大小", "肿瘤的个数", "肝癌级别", "脉管内癌栓分级"]
|
||||
@@ -150,12 +152,11 @@ uie = UIEModel(
|
||||
```
|
||||
|
||||
|
||||
### 关系抽取
|
||||
### Relation Extraction
|
||||
|
||||
关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
|
||||
|
||||
例如以"竞赛名称"作为抽取主体，抽取关系类型为"主办方"、"承办方"和"已举办次数", 则可执行如下语句：
|
||||
Relation Extraction (RE) refers to identifying entities from text and extracting semantic relationships between them to obtain triadic information, i.e. <subject, predicate, object>.
|
||||
|
||||
For example, if we take "contest name" as the extracted entity, and the relations are "主办方", "承办方" and "已举办次数", then we can execute the following statements.
|
||||
```python
|
||||
>>> schema = {"竞赛名称": ["主办方", "承办方", "已举办次数"]}
|
||||
>>> uie.set_schema(schema)
|
||||
@@ -198,11 +199,11 @@ uie = UIEModel(
|
||||
# 'text': '2022语言与智能技术竞赛'}}]
|
||||
```
|
||||
|
||||
### 事件抽取
|
||||
### Event Extraction
|
||||
|
||||
事件抽取 (Event Extraction, 简称EE)，是指从自然语言文本中抽取预定义的事件触发词(Trigger)和事件论元(Argument)，组合为相应的事件结构化信息。
|
||||
Event Extraction (EE) refers to extracting predefined Trigger and Argument from natural language texts and combining them into structured event information.
|
||||
|
||||
例如抽取的目标是"地震"事件的"地震强度"、"时间"、"震中位置"和"震源深度"这些信息，则可执行如下代码：
|
||||
For example, if the targets are"地震强度", "时间", "震中位置" and "引源深度" for the event "地震", we can execute the following codes.
|
||||
|
||||
```python
|
||||
>>> schema = {"地震触发词": ["地震强度", "时间", "震中位置", "震源深度"]}
|
||||
@@ -238,11 +239,11 @@ uie = UIEModel(
|
||||
# 'text': '地震'}}]
|
||||
```
|
||||
|
||||
### 评论观点抽取
|
||||
### Opinion Extraction
|
||||
|
||||
评论观点抽取，是指抽取文本中包含的评价维度、观点词。
|
||||
opinion extraction refers to the extraction of evaluation dimensions and opinions contained in the text.
|
||||
|
||||
例如抽取的目标是文本中包含的评价维度及其对应的观点词和情感倾向，可执行以下代码：
|
||||
For example, if the extraction target is the evaluation dimensions and their corresponding opinions and sentiment tendencies. We can execute the following codes：
|
||||
|
||||
```python
|
||||
>>> schema = {"评价维度": ["观点词", "情感倾向[正向，负向]"]}
|
||||
@@ -266,9 +267,9 @@ uie = UIEModel(
|
||||
# 'text': '性价比'}}]
|
||||
```
|
||||
|
||||
### 情感分类
|
||||
### Sentiment Classification
|
||||
|
||||
句子级情感倾向分类，即判断句子的情感倾向是“正向”还是“负向”，可执行以下代码：
|
||||
Sentence-level sentiment classification, i.e., determining a sentence has a "positive" sentiment or "negative" sentiment. We can execute the following codes:
|
||||
|
||||
```python
|
||||
>>> schema = ["情感倾向[正向，负向]"]
|
||||
@@ -283,9 +284,9 @@ uie = UIEModel(
|
||||
# 'text': '正向'}}]
|
||||
```
|
||||
|
||||
### 跨任务抽取
|
||||
### Cross-task Extraction
|
||||
|
||||
例如在法律场景同时对文本进行实体抽取和关系抽取，可执行以下代码：
|
||||
For example, in a legal scenario where both entity extraction and relation extraction need to be performed. We can execute the following codes.
|
||||
|
||||
```python
|
||||
>>> schema = ["法院", {"原告": "委托代理人"}, {"被告": "委托代理人"}]
|
||||
@@ -320,7 +321,7 @@ uie = UIEModel(
|
||||
# 'text': 'B公司'}}]
|
||||
```
|
||||
|
||||
## UIEModel Python接口
|
||||
## UIEModel Python Interface
|
||||
|
||||
```python
|
||||
fd.text.uie.UIEModel(model_file,
|
||||
@@ -334,50 +335,50 @@ fd.text.uie.UIEModel(model_file,
|
||||
schema_language=SchemaLanguage.ZH)
|
||||
```
|
||||
|
||||
UIEModel模型加载和初始化，其中`model_file`, `params_file`为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)，`vocab_file`为词表文件，UIE模型的词表可在[UIE配置文件](https://github.com/PaddlePaddle/PaddleNLP/blob/5401f01af85f1c73d8017c6b3476242fce1e6d52/model_zoo/uie/utils.py)中下载相应的UIE模型的vocab_file。
|
||||
UIEModel loading and initialization. Among them, `model_file`, `params_file` are Paddle inference documents exported by trained models. Please refer to [模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2).`vocab_file`refers to the vocabulary file. The vocabulary of the UIE model UIE can be downloaded in [UIE配置文件](https://github.com/PaddlePaddle/PaddleNLP/blob/5401f01af85f1c73d8017c6b3476242fce1e6d52/model_zoo/uie/utils.py)
|
||||
|
||||
**参数**
|
||||
**Parameter**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **vocab_file**(str): 词表文件
|
||||
> * **position_prob**(str): 位置概率，模型将输出位置概率大于`position_prob`的位置，默认为0.5
|
||||
> * **max_length**(int): 输入文本的最大长度。输入文本下标超过`max_length`的部分将被截断。默认为128
|
||||
> * **schema**(list|dict): 抽取任务的目标信息。
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
> * **schema_language**(SchemaLanguage): Schema语言。默认为ZH（中文），目前支持的语言种类包括：ZH（中文），EN（英文）。
|
||||
> * **model_file**(str): Model file path
|
||||
> * **params_file**(str): Parameter file path
|
||||
> * **vocab_file**(str): Vocabulary file
|
||||
> * **position_prob**(str): Position probability. The model will output positions with probability greater than `position_prob`, default is 0.5
|
||||
> * **max_length**(int): Maximized length of input text. Input text subscript exceeding `max_length` will be truncated. Default is 128
|
||||
> * **schema**(list|dict): Target information for extraction tasks
|
||||
> * **runtime_option**(RuntimeOption): Backend inference configuration, the default is None, i.e., the default configuration
|
||||
> * **model_format**(ModelFormat): Model format, and default is Paddle format
|
||||
> * **schema_language**(SchemaLanguage): Schema language, and default is ZH（Chinese）. Currently supported language：ZH（Chinese），EN（English）
|
||||
|
||||
### set_schema函数
|
||||
### set_schema Function
|
||||
|
||||
> ```python
|
||||
> set_schema(schema)
|
||||
> ```
|
||||
> 设置UIE模型的schema接口。
|
||||
> Set schema interface of the UIE model.
|
||||
>
|
||||
> **参数**
|
||||
> > * **schema**(list|dict): 输入数据，待抽取文本列表。
|
||||
> **Parameter**
|
||||
> > * **schema**(list|dict): Enter the data to be extracted from the text.
|
||||
>
|
||||
> **返回**
|
||||
> 空。
|
||||
> **Return**
|
||||
> Blank.
|
||||
|
||||
### predict函数
|
||||
### predict Function
|
||||
|
||||
> ```python
|
||||
> UIEModel.predict(texts, return_dict=False)
|
||||
> ```
|
||||
>
|
||||
> 模型预测接口，输入文本列表直接输出抽取结果。
|
||||
> Model prediction interface where input text list directly output extraction results.
|
||||
>
|
||||
> **参数**
|
||||
> **Parameter**
|
||||
>
|
||||
> > * **texts**(list(str)): 输入数据，待抽取文本列表。
|
||||
> > * **return_dict**(bool): 是否以字典形式输出UIE结果，默认为False。
|
||||
> **返回**
|
||||
> > * **texts**(list(str)): Enter the data to be extracted from the text.
|
||||
> > * **return_dict**(bool): Whether to output UIE results in the form of dictionary, and default is False。
|
||||
> **Return**
|
||||
>
|
||||
> > 返回`dict(str, list(fastdeploy.text.C.UIEResult))`。
|
||||
> > Return`dict(str, list(fastdeploy.text.C.UIEResult))`。
|
||||
|
||||
## 相关文档
|
||||
## Related Documents
|
||||
|
||||
[UIE模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md)
|
||||
|
||||
|
||||
388
examples/text/uie/python/README_CN.md
Normal file
388
examples/text/uie/python/README_CN.md
Normal file
@@ -0,0 +1,388 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# 通用信息抽取 UIE Python部署示例
|
||||
|
||||
在部署前，需确认以下两个步骤
|
||||
|
||||
- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
- 2. FastDeploy Python whl包安装，参考[FastDeploy Python安装](../../../../docs/cn/build_and_install/download_prebuilt_libraries.md)
|
||||
|
||||
本目录下提供`infer.py`快速完成UIE模型在CPU/GPU，以及CPU上通过OpenVINO加速CPU端部署示例。执行如下脚本即可完成。
|
||||
|
||||
## 快速开始
|
||||
```bash
|
||||
|
||||
#下载部署示例代码
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy.git
|
||||
cd FastDeploy/examples/text/uie/python
|
||||
|
||||
# 下载UIE模型文件和词表，以uie-base模型为例
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
# CPU推理
|
||||
python infer.py --model_dir uie-base --device cpu
|
||||
# GPU推理
|
||||
python infer.py --model_dir uie-base --device gpu
|
||||
# 使用OpenVINO推理
|
||||
python infer.py --model_dir uie-base --device cpu --backend openvino --cpu_num_threads 8
|
||||
```
|
||||
|
||||
运行完成后返回结果如下所示(仅截取NER任务的输出)。
|
||||
```bash
|
||||
1. Named Entity Recognition Task
|
||||
The extraction schema: ['时间', '选手', '赛事名称']
|
||||
[{'时间': {'end': 6,
|
||||
'probability': 0.9857379794120789,
|
||||
'start': 0,
|
||||
'text': '2月8日上午'},
|
||||
'赛事名称': {'end': 23,
|
||||
'probability': 0.8503087162971497,
|
||||
'start': 6,
|
||||
'text': '北京冬奥会自由式滑雪女子大跳台决赛'},
|
||||
'选手': {'end': 31,
|
||||
'probability': 0.8981553912162781,
|
||||
'start': 28,
|
||||
'text': '谷爱凌'}}]
|
||||
|
||||
The extraction schema: ['肿瘤的大小', '肿瘤的个数', '肝癌级别', '脉管内癌栓分级']
|
||||
[{'肝癌级别': {'end': 20,
|
||||
'probability': 0.9243271350860596,
|
||||
'start': 13,
|
||||
'text': 'II-III级'},
|
||||
'肿瘤的个数': {'end': 84,
|
||||
'probability': 0.7538408041000366,
|
||||
'start': 82,
|
||||
'text': '1个'},
|
||||
'肿瘤的大小': {'end': 100,
|
||||
'probability': 0.8341134190559387,
|
||||
'start': 87,
|
||||
'text': '4.2×4.0×2.8cm'},
|
||||
'脉管内癌栓分级': {'end': 70,
|
||||
'probability': 0.9083293080329895,
|
||||
'start': 67,
|
||||
'text': 'M0级'}}]
|
||||
......
|
||||
```
|
||||
|
||||
## UIE模型各抽取任务使用方式
|
||||
|
||||
在UIE模型中，schema代表要抽取的结构化信息，所以UIE模型可通过设置不同的schema支持不同信息抽取任务。
|
||||
|
||||
### 初始化UIEModel
|
||||
|
||||
```python
|
||||
import fastdeploy
|
||||
from fastdeploy.text import UIEModel
|
||||
model_dir = "uie-base"
|
||||
model_path = os.path.join(model_dir, "inference.pdmodel")
|
||||
param_path = os.path.join(model_dir, "inference.pdiparams")
|
||||
vocab_path = os.path.join(model_dir, "vocab.txt")
|
||||
|
||||
runtime_option = fastdeploy.RuntimeOption()
|
||||
schema = ["时间", "选手", "赛事名称"]
|
||||
|
||||
# 初始化UIE模型
|
||||
uie = UIEModel(
|
||||
model_path,
|
||||
param_path,
|
||||
vocab_path,
|
||||
position_prob=0.5,
|
||||
max_length=128,
|
||||
schema=schema,
|
||||
runtime_option=runtime_option)
|
||||
```
|
||||
|
||||
### 实体抽取
|
||||
|
||||
初始化阶段将schema设置为```["时间", "选手", "赛事名称"]```，可对输入的文本抽取时间、选手以及赛事名称三个信息。
|
||||
|
||||
```python
|
||||
>>> from pprint import pprint
|
||||
>>> results = uie.predict(
|
||||
["2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！"], return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'时间': {'end': 6,
|
||||
# 'probability': 0.9857379794120789,
|
||||
# 'start': 0,
|
||||
# 'text': '2月8日上午'},
|
||||
# '赛事名称': {'end': 23,
|
||||
# 'probability': 0.8503087162971497,
|
||||
# 'start': 6,
|
||||
# 'text': '北京冬奥会自由式滑雪女子大跳台决赛'},
|
||||
# '选手': {'end': 31,
|
||||
# 'probability': 0.8981553912162781,
|
||||
# 'start': 28,
|
||||
# 'text': '谷爱凌'}}]
|
||||
|
||||
```
|
||||
|
||||
例如抽取的目标实体类型是"肿瘤的大小"、"肿瘤的个数"、"肝癌级别"和"脉管内癌栓分级", 则可执行如下语句：
|
||||
|
||||
```python
|
||||
>>> schema = ["肿瘤的大小", "肿瘤的个数", "肝癌级别", "脉管内癌栓分级"]
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(
|
||||
[
|
||||
"（右肝肿瘤）肝细胞性肝癌（II-III级，梁索型和假腺管型），肿瘤包膜不完整，紧邻肝被膜，侵及周围肝组织，"
|
||||
"未见脉管内癌栓（MVI分级：M0级）及卫星子灶形成。（肿物1个，大小4.2×4.0×2.8cm）。"
|
||||
],
|
||||
return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'肝癌级别': {'end': 20,
|
||||
# 'probability': 0.9243271350860596,
|
||||
# 'start': 13,
|
||||
# 'text': 'II-III级'},
|
||||
# '肿瘤的个数': {'end': 84,
|
||||
# 'probability': 0.7538408041000366,
|
||||
# 'start': 82,
|
||||
# 'text': '1个'},
|
||||
# '肿瘤的大小': {'end': 100,
|
||||
# 'probability': 0.8341134190559387,
|
||||
# 'start': 87,
|
||||
# 'text': '4.2×4.0×2.8cm'},
|
||||
# '脉管内癌栓分级': {'end': 70,
|
||||
# 'probability': 0.9083293080329895,
|
||||
# 'start': 67,
|
||||
# 'text': 'M0级'}}]
|
||||
```
|
||||
|
||||
|
||||
### 关系抽取
|
||||
|
||||
关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
|
||||
|
||||
例如以"竞赛名称"作为抽取主体，抽取关系类型为"主办方"、"承办方"和"已举办次数", 则可执行如下语句：
|
||||
|
||||
```python
|
||||
>>> schema = {"竞赛名称": ["主办方", "承办方", "已举办次数"]}
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(
|
||||
[
|
||||
"2022语言与智能技术竞赛由中国中文信息学会和中国计算机学会联合主办，百度公司、中国中文信息学会评测工作"
|
||||
"委员会和中国计算机学会自然语言处理专委会承办，已连续举办4届，成为全球最热门的中文NLP赛事之一。"
|
||||
],
|
||||
return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'竞赛名称': {'end': 13,
|
||||
# 'probability': 0.7825401425361633,
|
||||
# 'relation': {'主办方': [{'end': 22,
|
||||
# 'probability': 0.8421716690063477,
|
||||
# 'start': 14,
|
||||
# 'text': '中国中文信息学会'},
|
||||
# {'end': 30,
|
||||
# 'probability': 0.7580805420875549,
|
||||
# 'start': 23,
|
||||
# 'text': '中国计算机学会'}],
|
||||
# '已举办次数': [{'end': 82,
|
||||
# 'probability': 0.4671304225921631,
|
||||
# 'start': 80,
|
||||
# 'text': '4届'}],
|
||||
# '承办方': [{'end': 39,
|
||||
# 'probability': 0.8292709589004517,
|
||||
# 'start': 35,
|
||||
# 'text': '百度公司'},
|
||||
# {'end': 55,
|
||||
# 'probability': 0.7000502943992615,
|
||||
# 'start': 40,
|
||||
# 'text': '中国中文信息学会评测工作委员会'},
|
||||
# {'end': 72,
|
||||
# 'probability': 0.6193484663963318,
|
||||
# 'start': 56,
|
||||
# 'text': '中国计算机学会自然语言处理专委会'}]},
|
||||
# 'start': 0,
|
||||
# 'text': '2022语言与智能技术竞赛'}}]
|
||||
```
|
||||
|
||||
### 事件抽取
|
||||
|
||||
事件抽取 (Event Extraction, 简称EE)，是指从自然语言文本中抽取预定义的事件触发词(Trigger)和事件论元(Argument)，组合为相应的事件结构化信息。
|
||||
|
||||
例如抽取的目标是"地震"事件的"地震强度"、"时间"、"震中位置"和"震源深度"这些信息，则可执行如下代码：
|
||||
|
||||
```python
|
||||
>>> schema = {"地震触发词": ["地震强度", "时间", "震中位置", "震源深度"]}
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(
|
||||
[
|
||||
"中国地震台网正式测定：5月16日06时08分在云南临沧市凤庆县(北纬24.34度，东经99.98度)发生3.5级地震，"
|
||||
"震源深度10千米。"
|
||||
],
|
||||
return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'地震触发词': {'end': 58,
|
||||
# 'probability': 0.9977425932884216,
|
||||
# 'relation': {'地震强度': [{'end': 56,
|
||||
# 'probability': 0.9980800747871399,
|
||||
# 'start': 52,
|
||||
# 'text': '3.5级'}],
|
||||
# '时间': [{'end': 22,
|
||||
# 'probability': 0.9853301644325256,
|
||||
# 'start': 11,
|
||||
# 'text': '5月16日06时08分'}],
|
||||
# '震中位置': [{'end': 50,
|
||||
# 'probability': 0.7874020934104919,
|
||||
# 'start': 23,
|
||||
# 'text': '云南临沧市凤庆县(北纬24.34度，东经99.98度)'}],
|
||||
# '震源深度': [{'end': 67,
|
||||
# 'probability': 0.9937973618507385,
|
||||
# 'start': 63,
|
||||
# 'text': '10千米'}]},
|
||||
# 'start': 56,
|
||||
# 'text': '地震'}}]
|
||||
```
|
||||
|
||||
### 评论观点抽取
|
||||
|
||||
评论观点抽取，是指抽取文本中包含的评价维度、观点词。
|
||||
|
||||
例如抽取的目标是文本中包含的评价维度及其对应的观点词和情感倾向，可执行以下代码：
|
||||
|
||||
```python
|
||||
>>> schema = {"评价维度": ["观点词", "情感倾向[正向，负向]"]}
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(
|
||||
["店面干净，很清静，服务员服务热情，性价比很高，发现收银台有排队"], return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'评价维度': {'end': 20,
|
||||
# 'probability': 0.9817039966583252,
|
||||
# 'relation': {'情感倾向[正向，负向]': [{'end': 0,
|
||||
# 'probability': 0.9966142177581787,
|
||||
# 'start': 0,
|
||||
# 'text': '正向'}],
|
||||
# '观点词': [{'end': 22,
|
||||
# 'probability': 0.9573966264724731,
|
||||
# 'start': 21,
|
||||
# 'text': '高'}]},
|
||||
# 'start': 17,
|
||||
# 'text': '性价比'}}]
|
||||
```
|
||||
|
||||
### 情感分类
|
||||
|
||||
句子级情感倾向分类，即判断句子的情感倾向是“正向”还是“负向”，可执行以下代码：
|
||||
|
||||
```python
|
||||
>>> schema = ["情感倾向[正向，负向]"]
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(["这个产品用起来真的很流畅，我非常喜欢"], return_dict=True)
|
||||
>>> pprint(results)
|
||||
|
||||
# 示例输出
|
||||
# [{'情感倾向[正向，负向]': {'end': 0,
|
||||
# 'probability': 0.9990023970603943,
|
||||
# 'start': 0,
|
||||
# 'text': '正向'}}]
|
||||
```
|
||||
|
||||
### 跨任务抽取
|
||||
|
||||
例如在法律场景同时对文本进行实体抽取和关系抽取，可执行以下代码：
|
||||
|
||||
```python
|
||||
>>> schema = ["法院", {"原告": "委托代理人"}, {"被告": "委托代理人"}]
|
||||
>>> uie.set_schema(schema)
|
||||
>>> results = uie.predict(
|
||||
[
|
||||
"北京市海淀区人民法院\n民事判决书\n(199x)建初字第xxx号\n原告：张三。\n委托代理人李四，北京市 A律师"
|
||||
"事务所律师。\n被告：B公司，法定代表人王五，开发公司总经理。\n委托代理人赵六，北京市 C律师事务所律师。"
|
||||
],
|
||||
return_dict=True)
|
||||
>>> pprint(results)
|
||||
# 示例输出
|
||||
# [{'原告': {'end': 37,
|
||||
# 'probability': 0.9949813485145569,
|
||||
# 'relation': {'委托代理人': [{'end': 46,
|
||||
# 'probability': 0.7956855297088623,
|
||||
# 'start': 44,
|
||||
# 'text': '李四'}]},
|
||||
# 'start': 35,
|
||||
# 'text': '张三'},
|
||||
# '法院': {'end': 10,
|
||||
# 'probability': 0.9221072793006897,
|
||||
# 'start': 0,
|
||||
# 'text': '北京市海淀区人民法院'},
|
||||
# '被告': {'end': 67,
|
||||
# 'probability': 0.8437348008155823,
|
||||
# 'relation': {'委托代理人': [{'end': 92,
|
||||
# 'probability': 0.7267124652862549,
|
||||
# 'start': 90,
|
||||
# 'text': '赵六'}]},
|
||||
# 'start': 64,
|
||||
# 'text': 'B公司'}}]
|
||||
```
|
||||
|
||||
## UIEModel Python接口
|
||||
|
||||
```python
|
||||
fd.text.uie.UIEModel(model_file,
|
||||
params_file,
|
||||
vocab_file,
|
||||
position_prob=0.5,
|
||||
max_length=128,
|
||||
schema=[],
|
||||
runtime_option=None,
|
||||
model_format=ModelFormat.PADDLE,
|
||||
schema_language=SchemaLanguage.ZH)
|
||||
```
|
||||
|
||||
UIEModel模型加载和初始化，其中`model_file`, `params_file`为训练模型导出的Paddle inference文件，具体请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)，`vocab_file`为词表文件，UIE模型的词表可在[UIE配置文件](https://github.com/PaddlePaddle/PaddleNLP/blob/5401f01af85f1c73d8017c6b3476242fce1e6d52/model_zoo/uie/utils.py)中下载相应的UIE模型的vocab_file。
|
||||
|
||||
**参数**
|
||||
|
||||
> * **model_file**(str): 模型文件路径
|
||||
> * **params_file**(str): 参数文件路径
|
||||
> * **vocab_file**(str): 词表文件
|
||||
> * **position_prob**(str): 位置概率，模型将输出位置概率大于`position_prob`的位置，默认为0.5
|
||||
> * **max_length**(int): 输入文本的最大长度。输入文本下标超过`max_length`的部分将被截断。默认为128
|
||||
> * **schema**(list|dict): 抽取任务的目标信息。
|
||||
> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
|
||||
> * **model_format**(ModelFormat): 模型格式，默认为Paddle格式
|
||||
> * **schema_language**(SchemaLanguage): Schema语言。默认为ZH（中文），目前支持的语言种类包括：ZH（中文），EN（英文）。
|
||||
|
||||
### set_schema函数
|
||||
|
||||
> ```python
|
||||
> set_schema(schema)
|
||||
> ```
|
||||
> 设置UIE模型的schema接口。
|
||||
>
|
||||
> **参数**
|
||||
> > * **schema**(list|dict): 输入数据，待抽取文本列表。
|
||||
>
|
||||
> **返回**
|
||||
> 空。
|
||||
|
||||
### predict函数
|
||||
|
||||
> ```python
|
||||
> UIEModel.predict(texts, return_dict=False)
|
||||
> ```
|
||||
>
|
||||
> 模型预测接口，输入文本列表直接输出抽取结果。
|
||||
>
|
||||
> **参数**
|
||||
>
|
||||
> > * **texts**(list(str)): 输入数据，待抽取文本列表。
|
||||
> > * **return_dict**(bool): 是否以字典形式输出UIE结果，默认为False。
|
||||
> **返回**
|
||||
>
|
||||
> > 返回`dict(str, list(fastdeploy.text.C.UIEResult))`。
|
||||
|
||||
## 相关文档
|
||||
|
||||
[UIE模型详细介绍](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md)
|
||||
|
||||
[UIE模型导出方法](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/uie/README.md#%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2)
|
||||
|
||||
[UIE C++部署方法](../cpp/README.md)
|
||||
@@ -1,22 +1,25 @@
|
||||
# UIE 服务化部署示例
|
||||
English | [简体中文](README_CN.md)
|
||||
|
||||
在服务化部署前，需确认
|
||||
# Example of UIE Serving Deployment
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
Before serving deployment, you need to confirm:
|
||||
|
||||
## 准备模型
|
||||
- 1. You can refer to [FastDeploy服务化部署](../../../../../serving/README_CN.md) for hardware and software environment requirements and image pull commands for serving images.
|
||||
|
||||
下载UIE-Base模型(如果有已训练好的模型，跳过此步骤):
|
||||
## Prepare models
|
||||
|
||||
Download the UIE-Base model (if you have trained models, skip this step):
|
||||
```bash
|
||||
# 下载UIE模型文件和词表，以uie-base模型为例
|
||||
# Download UIE model documents and vocabulary. Taking the uie-base model as an example
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
# 将下载的模型移动到模型仓库目录
|
||||
# Move the model to the model repository directory
|
||||
mv uie-base/* models/uie/1/
|
||||
```
|
||||
|
||||
模型下载移动好之后，目录结构如下:
|
||||
After download and move, the models directory is as follows:
|
||||
|
||||
```
|
||||
models
|
||||
└── uie
|
||||
@@ -28,24 +31,24 @@ models
|
||||
└── config.pbtxt
|
||||
```
|
||||
|
||||
## 拉取并运行镜像
|
||||
## Pull and run images.
|
||||
```bash
|
||||
# x.y.z为镜像版本号，需参照serving文档替换为数字
|
||||
# GPU镜像
|
||||
# x.y.z represent image versions. You can refer to the serving documents to replace them with numbers
|
||||
# GPU Image
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
# CPU Image
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /uie_serving 目录
|
||||
# Run the container. The container name is fd_serving, and the current directory is mounted as the container's /uie_serving directory
|
||||
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
# Start the service (it will have scheduling privileges for all GPU cards without setting the CUDA_VISIBLE_DEVICES environment variable)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/uie_serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改grpc_client.py中的请求端口号
|
||||
>> **Attention**: When appearing "Address already in use", please use `--grpc-port`specified port numbers to start the service. Meanwhile you should change the requesting port numbers in grpc_client.py
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
When starting the service, the following output will be displayed:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
@@ -54,22 +57,22 @@ I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
客户端请求可以在本地执行脚本请求；也可以在容器中执行。
|
||||
## Client Requests
|
||||
Client requests can execute script requests locally and in the container.
|
||||
|
||||
本地执行脚本需要先安装依赖:
|
||||
Dependencies should be installed to execute the script locally:
|
||||
```
|
||||
pip install grpcio
|
||||
pip install tritonclient[all]
|
||||
|
||||
# 如果bash无法识别括号，可以使用如下指令安装:
|
||||
# If bash cannot recognize the brackets, you can use the following command to install dependencies:
|
||||
pip install tritonclient\[all\]
|
||||
|
||||
# 发送请求
|
||||
# Send Requests
|
||||
python3 grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回结果并打印输出:
|
||||
When the request is sent successfully, the result is returned and printed out:
|
||||
```
|
||||
1. Named Entity Recognition Task--------------
|
||||
The extraction schema: ['时间', '选手', '赛事名称']
|
||||
@@ -138,6 +141,6 @@ results:
|
||||
```
|
||||
|
||||
|
||||
## 配置修改
|
||||
## Configuration Modification
|
||||
|
||||
当前默认配置在GPU上运行Paddle引擎,如果要在CPU/GPU或其他推理引擎上运行, 需要修改配置，详情请参考[配置文档](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
The current configuration is by default to run the paddle engine on CPU. If you want to run on CPU/GPU or other inference engines, modifying the configuration is needed.Please refer to [配置文档](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
|
||||
145
examples/text/uie/serving/README_CN.md
Normal file
145
examples/text/uie/serving/README_CN.md
Normal file
@@ -0,0 +1,145 @@
|
||||
[English](README.md) | 简体中文
|
||||
|
||||
# UIE 服务化部署示例
|
||||
|
||||
在服务化部署前，需确认
|
||||
|
||||
- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)
|
||||
|
||||
## 准备模型
|
||||
|
||||
下载UIE-Base模型(如果有已训练好的模型，跳过此步骤):
|
||||
```bash
|
||||
# 下载UIE模型文件和词表，以uie-base模型为例
|
||||
wget https://bj.bcebos.com/fastdeploy/models/uie/uie-base.tgz
|
||||
tar -xvfz uie-base.tgz
|
||||
|
||||
# 将下载的模型移动到模型仓库目录
|
||||
mv uie-base/* models/uie/1/
|
||||
```
|
||||
|
||||
模型下载移动好之后，目录结构如下:
|
||||
```
|
||||
models
|
||||
└── uie
|
||||
├── 1
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── model.py
|
||||
│ └── vocab.txt
|
||||
└── config.pbtxt
|
||||
```
|
||||
|
||||
## 拉取并运行镜像
|
||||
```bash
|
||||
# x.y.z为镜像版本号，需参照serving文档替换为数字
|
||||
# GPU镜像
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
|
||||
# CPU镜像
|
||||
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10
|
||||
|
||||
# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /uie_serving 目录
|
||||
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
|
||||
|
||||
# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量，会拥有所有GPU卡的调度权限)
|
||||
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/uie_serving/models --backend-config=python,shm-default-byte-size=10485760
|
||||
```
|
||||
|
||||
>> **注意**: 当出现"Address already in use", 请使用`--grpc-port`指定端口号来启动服务，同时更改grpc_client.py中的请求端口号
|
||||
|
||||
服务启动成功后， 会有以下输出:
|
||||
```
|
||||
......
|
||||
I0928 04:51:15.784517 206 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001
|
||||
I0928 04:51:15.785177 206 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000
|
||||
I0928 04:51:15.826578 206 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002
|
||||
```
|
||||
|
||||
|
||||
## 客户端请求
|
||||
客户端请求可以在本地执行脚本请求；也可以在容器中执行。
|
||||
|
||||
本地执行脚本需要先安装依赖:
|
||||
```
|
||||
pip install grpcio
|
||||
pip install tritonclient[all]
|
||||
|
||||
# 如果bash无法识别括号，可以使用如下指令安装:
|
||||
pip install tritonclient\[all\]
|
||||
|
||||
# 发送请求
|
||||
python3 grpc_client.py
|
||||
```
|
||||
|
||||
发送请求成功后，会返回结果并打印输出:
|
||||
```
|
||||
1. Named Entity Recognition Task--------------
|
||||
The extraction schema: ['时间', '选手', '赛事名称']
|
||||
text= ['2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！']
|
||||
results:
|
||||
{'时间': {'end': 6,
|
||||
'probability': 0.9857379794120789,
|
||||
'start': 0,
|
||||
'text': '2月8日上午'},
|
||||
'赛事名称': {'end': 23,
|
||||
'probability': 0.8503087162971497,
|
||||
'start': 6,
|
||||
'text': '北京冬奥会自由式滑雪女子大跳台决赛'},
|
||||
'选手': {'end': 31,
|
||||
'probability': 0.8981545567512512,
|
||||
'start': 28,
|
||||
'text': '谷爱凌'}}
|
||||
================================================
|
||||
text= ['2月7日北京冬奥会短道速滑男子1000米决赛中任子威获得冠军！']
|
||||
results:
|
||||
{'时间': {'end': 4,
|
||||
'probability': 0.9921242594718933,
|
||||
'start': 0,
|
||||
'text': '2月7日'},
|
||||
'赛事名称': {'end': 22,
|
||||
'probability': 0.8171929121017456,
|
||||
'start': 4,
|
||||
'text': '北京冬奥会短道速滑男子1000米决赛'},
|
||||
'选手': {'end': 26,
|
||||
'probability': 0.9821093678474426,
|
||||
'start': 23,
|
||||
'text': '任子威'}}
|
||||
|
||||
2. Relation Extraction Task
|
||||
The extraction schema: {'竞赛名称': ['主办方', '承办方', '已举办次数']}
|
||||
text= ['2022语言与智能技术竞赛由中国中文信息学会和中国计算机学会联合主办，百度公司、中国中文信息学会评测工作委员会和中国计算机学会自然语言处理专委会承办，已连续举办4届，成为全球最热门的中文NLP赛事之一。']
|
||||
results:
|
||||
{'竞赛名称': {'end': 13,
|
||||
'probability': 0.7825395464897156,
|
||||
'relation': {'主办方': [{'end': 22,
|
||||
'probability': 0.8421710729598999,
|
||||
'start': 14,
|
||||
'text': '中国中文信息学会'},
|
||||
{'end': 30,
|
||||
'probability': 0.7580801248550415,
|
||||
'start': 23,
|
||||
'text': '中国计算机学会'}],
|
||||
'已举办次数': [{'end': 82,
|
||||
'probability': 0.4671308398246765,
|
||||
'start': 80,
|
||||
'text': '4届'}],
|
||||
'承办方': [{'end': 39,
|
||||
'probability': 0.8292703628540039,
|
||||
'start': 35,
|
||||
'text': '百度公司'},
|
||||
{'end': 55,
|
||||
'probability': 0.7000497579574585,
|
||||
'start': 40,
|
||||
'text': '中国中文信息学会评测工作委员会'},
|
||||
{'end': 72,
|
||||
'probability': 0.6193480491638184,
|
||||
'start': 56,
|
||||
'text': '中国计算机学会自然语言处理专委会'}]},
|
||||
'start': 0,
|
||||
'text': '2022语言与智能技术竞赛'}}
|
||||
```
|
||||
|
||||
|
||||
## 配置修改
|
||||
|
||||
当前默认配置在GPU上运行Paddle引擎,如果要在CPU/GPU或其他推理引擎上运行, 需要修改配置，详情请参考[配置文档](../../../../serving/docs/zh_CN/model_configuration.md)
|
||||
Reference in New Issue
Block a user