English | [简体中文](README_CN.md) # PP-OCR Serving Deployment Example Before the serving deployment, please confirm - 1. Refer to [FastDeploy Serving Deployment](../../../../../serving/README.md) for software and hardware environment requirements and image pull commands ## Introduction This document describes how to build an OCR text recognition service with FastDeploy. The server must be started in docker, while the client does not need to be in a docker container. **The models in the path ($PWD) contain the model configuration and code (the server will load the models and code to start the service), which needs to be mapped to docker.** OCR consists of det (detection), cls (classification) and rec (recognition) models. The diagram of the serving deployment is shown below, where `pp_ocr` connects to `det_preprocess`、`det_runtime` and `det_postprocess`,`cls_pp` connects to `cls_runtime` and `cls_postprocess`,`rec_pp` connects to `rec_runtime` and `rec_postprocess`. In particular, `cls_pp` and `rec_pp` services are called multiple times in `det_postprocess` to realize the classification and identification of the detection results (multiple boxes), and finally return the identification results to users.



## Usage ### 1. Server #### 1.1 Docker ```bash # Download the repository code git clone https://github.com/PaddlePaddle/FastDeploy.git cd FastDeploy/examples/vision/ocr/PP-OCRv3/serving/ # Dpwnload model, image, and dictionary files wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_infer.tar tar xvf ch_PP-OCRv3_det_infer.tar && mv ch_PP-OCRv3_det_infer 1 mv 1/inference.pdiparams 1/model.pdiparams && mv 1/inference.pdmodel 1/model.pdmodel mv 1 models/det_runtime/ && rm -rf ch_PP-OCRv3_det_infer.tar wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar tar xvf ch_ppocr_mobile_v2.0_cls_infer.tar && mv ch_ppocr_mobile_v2.0_cls_infer 1 mv 1/inference.pdiparams 1/model.pdiparams && mv 1/inference.pdmodel 1/model.pdmodel mv 1 models/cls_runtime/ && rm -rf ch_ppocr_mobile_v2.0_cls_infer.tar wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_infer.tar tar xvf ch_PP-OCRv3_rec_infer.tar && mv ch_PP-OCRv3_rec_infer 1 mv 1/inference.pdiparams 1/model.pdiparams && mv 1/inference.pdmodel 1/model.pdmodel mv 1 models/rec_runtime/ && rm -rf ch_PP-OCRv3_rec_infer.tar mkdir models/pp_ocr/1 && mkdir models/rec_pp/1 && mkdir models/cls_pp/1 wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/ppocr/utils/ppocr_keys_v1.txt mv ppocr_keys_v1.txt models/rec_postprocess/1/ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg # x.y.z represent the image version. Refer to serving document to replace them with numbers docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash docker exec -it -u root fastdeploy bash ``` #### 1.2 Installation (in docker) ```bash ldconfig apt-get install libgl1 ``` #### 1.3 Start the server (in docker) ```bash fastdeployserver --model-repository=/ocr_serving/models ``` Parameter: - `model-repository`(required): The storage path of the entire model streaming_pp_tts. - `http-port`(optional): Port number for the HTTP service. Default: `8000`. This port is not used in this example. - `grpc-port`(optional): Port number for the GRPC service. Default: `8001`. - `metrics-port`(optional): Port number for the serer metric. Default: `8002`. This port is not used in this example. ### 2. Client #### 2.1 Installation ```bash pip3 install tritonclient[all] ``` #### 2.2 Send Requests ```bash python3 client.py ``` ## Configuration Change The current default configuration runs on GPU. If you want to run it on CPU or other inference engines, please modify the configuration in `models/runtime/config.pbtxt`. Refer to [Configuration Document](../../../../../serving/docs/EN/model_configuration-en.md) for more information. ## Use VisualDL for serving deployment visualization You can use VisualDL for [serving deployment visualization](../../../../../serving/docs/EN/vdl_management-en.md) , the above model preparation, deployment, configuration modification and client request operations can all be performed based on VisualDL. The serving deployment of PP-OCR by VisualDL only needs the following three steps: ```text 1. Load the model repository: ./vision/ocr/PP-OCRv3/serving 2. Download the model resource file: click the det_runtime model, click the version number 1 to add the pre-training model, and select the text recognition model ch_PP-OCRv3_det to download. click the cls_runtime model, click the version number 1 to add the pre-training model, and select the text recognition model ch_ppocr_mobile_v2.0_cls to download. click the rec_runtime model, click the version number 1 to add the pre-training model, and select the text recognition model ch_PP-OCRv3_rec to download. click the rec_postprocess model, click the version number 1 to add the pre-training model, and select the text recognition model ch_PP-OCRv3_rec to download. 3. Start the service: Click the "launch server" button and input the launch parameters. ```