mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-05 08:37:06 +08:00
[Other] Refactor js submodule (#415)
* Refactor js submodule * Remove change-log * Update ocr module * Update ocr-detection module * Update ocr-detection module * Remove change-log
This commit is contained in:
chenqianhe
11
.gitignore
vendored
11
.gitignore
vendored
@@ -25,3 +25,14 @@ python/scripts/process_libraries.py
|
||||
.vs
|
||||
.idea
|
||||
.DS_Store
|
||||
miniprogram_npm
|
||||
node_modules
|
||||
.DS_Store
|
||||
dist
|
||||
etc
|
||||
lib
|
||||
dist-ssr
|
||||
coverage
|
||||
*.local
|
||||
yalc.*
|
||||
.yalc
|
||||
|
||||
@@ -12,14 +12,14 @@
|
||||
|
||||
## Web Demo使用
|
||||
|
||||
在浏览器中直接运行官方demo参考[文档](./web_demo/README.md)
|
||||
在浏览器中直接运行官方demo参考[文档](./WebDemo.md)
|
||||
|
||||
|demo名称|web demo目录|可视化|
|
||||
|-|-|-|
|
||||
|目标检测|[ScrewDetection、FaceDetection](./web_demo/demo/src/pages/cv/detection/)| <img src="https://user-images.githubusercontent.com/26592129/196874536-b7fa2c0a-d71f-4271-8c40-f9088bfad3c9.png" height="200px">|
|
||||
|人像分割背景替换|[HumanSeg](./web_demo//demo/src/pages/cv/segmentation/HumanSeg)|<img src="https://user-images.githubusercontent.com/26592129/196874452-4ef2e770-fbb3-4a35-954b-f871716d6669.png" height="200px">|
|
||||
|物体识别|[GestureRecognition、ItemIdentification](./web_demo//demo/src/pages/cv/recognition/)|<img src="https://user-images.githubusercontent.com/26592129/196874416-454e6bb0-4ebd-4b51-a88a-8c40614290ae.png" height="200px">|
|
||||
|PP-OCRv3|[TextDetection、TextRecognition](./web_demo//demo/src/pages/cv/ocr/)|<img src="https://user-images.githubusercontent.com/26592129/196874354-1b5eecb0-f273-403c-aa6c-4463bf6d78db.png" height="200px">|
|
||||
|目标检测|[ScrewDetection、FaceDetection](./web_demo/src/pages/cv/detection/)| <img src="https://user-images.githubusercontent.com/26592129/196874536-b7fa2c0a-d71f-4271-8c40-f9088bfad3c9.png" height="200px">|
|
||||
|人像分割背景替换|[HumanSeg](./web_demo/src/pages/cv/segmentation/HumanSeg)|<img src="https://user-images.githubusercontent.com/26592129/196874452-4ef2e770-fbb3-4a35-954b-f871716d6669.png" height="200px">|
|
||||
|物体识别|[GestureRecognition、ItemIdentification](./web_demo/src/pages/cv/recognition/)|<img src="https://user-images.githubusercontent.com/26592129/196874416-454e6bb0-4ebd-4b51-a88a-8c40614290ae.png" height="200px">|
|
||||
|PP-OCRv3|[TextDetection、TextRecognition](./web_demo/src/pages/cv/ocr/)|<img src="https://user-images.githubusercontent.com/26592129/196874354-1b5eecb0-f273-403c-aa6c-4463bf6d78db.png" height="200px">|
|
||||
|
||||
|
||||
## 微信小程序Demo使用
|
||||
@@ -37,5 +37,5 @@
|
||||
|
||||
## Contributor
|
||||
|
||||
感谢飞桨开发者专家(PPDE)陈千鹤(github: [chenqianhe](https://github.com/chenqianhe))贡献的小程序、Web demo，项目[链接](https://github.com/chenqianhe/Paddle.js)。
|
||||
感谢飞桨开发者专家(PPDE) 陈千鹤(github: [chenqianhe](https://github.com/chenqianhe))贡献的Web demo, 小程序。
|
||||
|
||||
|
||||
172
examples/application/js/WebDemo.md
Normal file
172
examples/application/js/WebDemo.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# Web Demo介绍
|
||||
|
||||
- [简介](#0)
|
||||
- [1. 快速开始](#1)
|
||||
- [2. npm包调用](#2)
|
||||
- [3. 模型替换](#3)
|
||||
- [4. 自定义前后处理参数](#4)
|
||||
- [5. 其他](#5)
|
||||
|
||||
<a name="0"></a>
|
||||
## 简介
|
||||
|
||||
本项目基于[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)在浏览器中实现目标检测，人像分割，OCR，物品分类等计算机视觉任务。
|
||||
|
||||
|
||||
|demo名称|web demo组件|源码目录|npm包|
|
||||
|-|-|-|-|
|
||||
|人脸检测|[FaceDetection](./web_demo/src/pages/cv/detection/FaceDetection/)| [facedetect](./package/packages/paddlejs-models/facedetect)|[@paddle-js-models/facedetect](https://www.npmjs.com/package/@paddle-js-models/facedetect)|
|
||||
|螺丝钉检测|[ScrewDetection](./web_demo/src/pages/cv/detection/ScrewDetection)| [detect](./package/packages/paddlejs-models/detect)|[@paddle-js-models/detect](https://www.npmjs.com/package/@paddle-js-models/detect)|
|
||||
|人像分割背景替换|[HumanSeg](./web_demo/src/pages/cv/segmentation/HumanSeg)|[humanseg](./package/packages/paddlejs-models/humanseg)|[@paddle-js-models/humanseg](https://www.npmjs.com/package/@paddle-js-models/humanseg)|
|
||||
|手势识别AI猜丁壳|[GestureRecognition](./web_demo/src/pages/cv/recognition/GestureRecognition)|[gesture](./package/packages/paddlejs-models/gesture)|[@paddle-js-models/gesture](https://www.npmjs.com/package/@paddle-js-models/gesture)|
|
||||
|1000种物品识别|[ItemIdentification](./web_demo/src/pages/cv/recognition/ItemIdentification)|[mobilenet](./package/packages/paddlejs-models/mobilenet)|[@paddle-js-models/mobilenet](https://www.npmjs.com/package/@paddle-js-models/mobilenet)|
|
||||
|文本检测|[TextDetection](./web_demo/src/pages/cv/ocr/TextDetection)|[ocrdetection](./package/packages/paddlejs-models/ocrdetection)|[@paddle-js-models/ocrdet](https://www.npmjs.com/package/@paddle-js-models/ocrdet)|
|
||||
|文本识别|[TextRecognition](./web_demo/src/pages/cv/ocr/TextRecognition)|[ocr](./package/packages/paddlejs-models/ocr)|[@paddle-js-models/ocr](https://www.npmjs.com/package/@paddle-js-models/ocr)|
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. 快速开始
|
||||
|
||||
本节介绍如何在浏览器中直接运行官方demo。
|
||||
|
||||
**1. 安装Node.js**
|
||||
|
||||
从`Node.js`官网https://nodejs.org/en/download/ 下载适合自己平台的`Node.js`安装包并安装。
|
||||
|
||||
**2. 安装demo依赖并启动**
|
||||
在`./web_demo`目录下执行如下指令：
|
||||
|
||||
```
|
||||
# 安装依赖
|
||||
npm install
|
||||
# 启动demo
|
||||
npm run dev
|
||||
```
|
||||
|
||||
在浏览器中打开网址 `http://localhost:5173/main/index.html` 即可快速体验在浏览器中运行计算机视觉任务。
|
||||
|
||||
|
||||
|
||||
<a name="2"></a>
|
||||
## 2. npm包调用
|
||||
|
||||
本节介绍npm包的使用方式，每个demo均提供简单易用的接口，用户只需初始化上传图片即可获得结果，使用步骤如下：
|
||||
1. 调用模块
|
||||
2. 初始化模型
|
||||
3. 传入输入，执行预测
|
||||
|
||||
以 OCR 为例，在前端项目中，`@paddle-js-models/ocr`包的使用方式如下：
|
||||
|
||||
```
|
||||
// 1. 调用ocr模块
|
||||
import * as ocr from '@paddle-js-models/ocr';
|
||||
|
||||
// 2. 初始化ocr模型
|
||||
await ocr.init();
|
||||
|
||||
// 3. 传入HTMLImageElement类型的图像作为输入并获得结果
|
||||
const res = await ocr.recognize(img);
|
||||
|
||||
// 打印OCR模型得到的文本坐标以及文本内容
|
||||
console.log(res.text);
|
||||
console.log(res.points);
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
## 3. 模型替换
|
||||
|
||||
由于前端环境和计算资源限制，在前端部署深度学习模型时，我们对模型的性能有着更严格的要求，简单来说，模型需要足够轻量化。理论上模型的输入shape越小、模型大小越小，则对应的模型的flops越小，在前端运行也能更流畅。经验总结，使用`Paddle.js`部署的模型存储尽量不超过*5M*，实际情况根据硬件和计算资源情况决定。
|
||||
|
||||
在实际应用中，常常根据垂类的场景定制化模型，官方的demo支持修改传入参数替换模型。
|
||||
|
||||
以OCR demo为例，[ocr.init()函数](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/package/packages/paddlejs-models/ocr/src/index.ts#L52)中，包含默认初始化的模型链接，如果要替换模型参考下述步骤。
|
||||
|
||||
步骤1：将模型转成js格式：
|
||||
```
|
||||
# 安装paddlejsconverter
|
||||
pip3 install paddlejsconverter
|
||||
# 转换模型格式，输入模型为inference模型
|
||||
paddlejsconverter --modelPath=./inference.pdmodel --paramPath=./inference.pdiparams --outputDir=./ --useGPUOpt=True
|
||||
# 注意：useGPUOpt 选项默认不开启，如果模型用在 gpu backend（webgl/webgpu），则开启 useGPUOpt，如果模型运行在（wasm/plain js）则不要开启。
|
||||
```
|
||||
|
||||
导出成功后，本地目录下会出现 `model.json chunk_1.dat`等文件，分别是对应js模型的网络结构、模型参数二进制文件。
|
||||
|
||||
步骤2：将导出的js模型上传到支持跨域访问的服务器，服务器的CORS配置参考下图：
|
||||
|
||||
|
||||
|
||||
步骤3：修改代码替换默认的模型。以OCR demo为例，修改OCR web demo中[模型初始化代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L64)，即
|
||||
|
||||
```
|
||||
await ocr.init();
|
||||
修改为：
|
||||
await ocr.init({modelPath: "https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json"}); # 第一个参数传入新的文本检测字典类型参数
|
||||
```
|
||||
|
||||
重新在demo目录下执行下述命令，即可体验新的模型效果。
|
||||
```
|
||||
npm run dev
|
||||
```
|
||||
|
||||
<a name="4"></a>
|
||||
## 4. 自定义前后处理参数
|
||||
|
||||
**自定义前处理参数**
|
||||
|
||||
在不同计算机视觉任务中，不同的模型可能有不同的预处理参数，比如mean,std,keep_ratio等参数，替换模型后也需要对预处理参数进行修改。paddle.js发布的npm包中提供了自定义预处理参数的简单方案。只需要在调用模型初始化函数时，传入自定义的参数即可。
|
||||
|
||||
```
|
||||
# 默认参数初始化
|
||||
await model.init();
|
||||
|
||||
自定义参数初始化
|
||||
const Config = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5], keepratio: false};
|
||||
await model.init(Config);
|
||||
```
|
||||
|
||||
以OCR文本检测demo为例，修改模型前处理的mean和std参数，只需要在模型初始化时传入自定义的mean和std参数。
|
||||
```
|
||||
await ocr.init();
|
||||
修改为：
|
||||
const detConfig = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5]};
|
||||
await ocr.init(detConfig); # 第一个参数传入新的文本检测模型链接
|
||||
```
|
||||
|
||||
**自定义后处理参数**
|
||||
|
||||
同理，paddle.js发布的npm包也提供了后处理参数的自定义方案。
|
||||
|
||||
```
|
||||
# 默认参数运行
|
||||
await model.predict();
|
||||
|
||||
# 自定义后处理参数
|
||||
const postConfig = {thresh: 0.5};
|
||||
await model.predict(Config);
|
||||
```
|
||||
|
||||
以OCR文本检测 demo为例，修改文本检测后处理的参数实现扩大文本检测框的效果，修改OCR web demo中执行[模型预测代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/web_demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L99)，即：
|
||||
|
||||
```
|
||||
const res = await ocr.recognize(img, { canvas: canvas.value });
|
||||
修改为：
|
||||
// 定义超参数，将unclip_ratio参数从1.5 增大为3.5
|
||||
const detConfig = {shape: 960, thresh: 0.3, box_thresh: 0.6, unclip_ratio:3.5};
|
||||
const res = await ocr.recognize(img, { canvas: canvas.value }, detConfig);
|
||||
```
|
||||
|
||||
注：不同的任务有不同的后处理参数，详细参数参考npm包中的API。
|
||||
|
||||
<a name="5"></a>
|
||||
## 5. 其他
|
||||
|
||||
`Paddle.js`转换后的模型不仅支持浏览器中使用，也可以在百度小程序和微信小程序环境下运行。
|
||||
|
||||
|名称|目录|
|
||||
|-|-|
|
||||
|OCR文本检测| [ocrdetecXcx](./mini_program/ocrdetectXcx/) |
|
||||
|OCR文本识别| [ocrXcx](./mini_program/ocrXcx/) |
|
||||
|目标检测| coming soon |
|
||||
|图像分割| coming soon |
|
||||
|物品分类| coming soon |
|
||||
41
examples/application/js/package/README.md
Normal file
41
examples/application/js/package/README.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Paddle.js Model Module介绍
|
||||
|
||||
该部分是基于 Paddle.js 进行开发的模型库，主要提供 Web 端可直接引入使用模型的能力。
|
||||
|
||||
| demo名称 | 源码目录 | npm包 |
|
||||
| ---------------- | ------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| 人脸检测 | [facedetect](./packages/paddlejs-models/facedetect) | [@paddle-js-models/facedetect](https://www.npmjs.com/package/@paddle-js-models/facedetect) |
|
||||
| 螺丝钉检测 | [detect](./packages/paddlejs-models/detect) | [@paddle-js-models/detect](https://www.npmjs.com/package/@paddle-js-models/detect) |
|
||||
| 人像分割背景替换 | [humanseg](./packages/paddlejs-models/humanseg) | [@paddle-js-models/humanseg](https://www.npmjs.com/package/@paddle-js-models/humanseg) |
|
||||
| 手势识别AI猜丁壳 | [gesture](./packages/paddlejs-models/gesture) | [@paddle-js-models/gesture](https://www.npmjs.com/package/@paddle-js-models/gesture) |
|
||||
| 1000种物品识别 | [mobilenet](./packages/paddlejs-models/mobilenet) | [@paddle-js-models/mobilenet](https://www.npmjs.com/package/@paddle-js-models/mobilenet) |
|
||||
| 文本检测 | [ocrdetection](./packages/paddlejs-models/ocrdetection) | [@paddle-js-models/ocrdet](https://www.npmjs.com/package/@paddle-js-models/ocrdet) |
|
||||
| 文本识别 | [ocr](./packages/paddlejs-models/ocr) | [@paddle-js-models/ocr](https://www.npmjs.com/package/@paddle-js-models/ocr) |
|
||||
|
||||
## 开发使用
|
||||
|
||||
该部分是使用 `pnpm` 搭建的 Menorepo
|
||||
|
||||
### 安装依赖
|
||||
|
||||
```sh
|
||||
pnpm i
|
||||
```
|
||||
|
||||
### 开发
|
||||
参考 Package.json 使用 `yalc` 进行开发测试。
|
||||
|
||||
```sh
|
||||
pnpm run dev:xxx
|
||||
```
|
||||
|
||||
### 整体简介
|
||||
|
||||
1. 使用 rollup 一次性打包生成 commonjs 和 es 规范的代码；同时具有可扩展性；目前由于依赖的cv库有些问题；就没有配置umd打包。
|
||||
2. 打包时基于 api-extractor 实现 d.ts 文件生成，实现支持 ts 引入生成我们的包
|
||||
3. 基于 jest 支持测试并显示测试相关覆盖率等
|
||||
4. 基于 ts 和 eslint 维护代码风格，保证代码更好开发
|
||||
5. 基于 conventional-changelog-cli 实现自定义关键词生成对应生成changelog
|
||||
6. 基于 yalc 实现本地打包开发测试
|
||||
|
||||
|
||||
6927
examples/application/js/package/package-lock.json
generated
Normal file
6927
examples/application/js/package/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
51
examples/application/js/package/package.json
Normal file
51
examples/application/js/package/package.json
Normal file
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"name": "paddle-js",
|
||||
"version": "3.0.0",
|
||||
"description": "paddlejs",
|
||||
"keywords": [
|
||||
"paddlejs",
|
||||
"web AI",
|
||||
"typescript"
|
||||
],
|
||||
"author": "",
|
||||
"license": "ISC",
|
||||
"scripts": {
|
||||
"dev:@paddle-js-models/detect": "pnpm -F @paddle-js-models/detect dev",
|
||||
"dev:@paddle-js-models/facedetect": "pnpm -F @paddle-js-models/facedetect dev",
|
||||
"dev:@paddle-js-models/ocrdet": "pnpm -F @paddle-js-models/ocrdet dev",
|
||||
"dev:@paddle-js-models/ocr": "pnpm -F @paddle-js-models/ocr dev",
|
||||
"dev:@paddle-js-models/gesture": "pnpm -F @paddle-js-models/gesture dev",
|
||||
"dev:@paddle-js-models/humanseg": "pnpm -F @paddle-js-models/humanseg dev",
|
||||
"dev:@paddle-js-models/humanseg_gpu": "pnpm -F @paddle-js-models/humanseg_gpu dev",
|
||||
"dev:@paddle-js-models/mobilenet": "pnpm -F @paddle-js-models/mobilenet dev",
|
||||
"build:@paddle-js-models/detect": "pnpm -F @paddle-js-models/detect build",
|
||||
"build:@paddle-js-models/facedetect": "pnpm -F @paddle-js-models/facedetect build",
|
||||
"build:@paddle-js-models/ocrdet": "pnpm -F @paddle-js-models/ocrdet build",
|
||||
"build:@paddle-js-models/ocr": "pnpm -F @paddle-js-models/ocr build",
|
||||
"build:@paddle-js-models/gesture": "pnpm -F @paddle-js-models/gesture build",
|
||||
"build:@paddle-js-models/humanseg": "pnpm -F @paddle-js-models/humanseg build",
|
||||
"build:@paddle-js-models/humanseg_gpu": "pnpm -F @paddle-js-models/humanseg_gpu build",
|
||||
"build:@paddle-js-models/mobilenet": "pnpm -F @paddle-js-models/mobilenet build",
|
||||
"publish:@paddle-js-models/detect": "pnpm -F @paddle-js-models/detect publish",
|
||||
"publish:@paddle-js-models/facedetect": "pnpm -F @paddle-js-models/facedetect publish",
|
||||
"publish:@paddle-js-models/ocrdet": "pnpm -F @paddle-js-models/ocrdet publish",
|
||||
"publish:@paddle-js-models/ocr": "pnpm -F @paddle-js-models/ocr publish",
|
||||
"publish:@paddle-js-models/gesture": "pnpm -F @paddle-js-models/gesture publish",
|
||||
"publish:@paddle-js-models/humanseg": "pnpm -F @paddle-js-models/humanseg publish",
|
||||
"publish:@paddle-js-models/humanseg_gpu": "pnpm -F @paddle-js-models/humanseg_gpu publish",
|
||||
"publish:@paddle-js-models/mobilenet": "pnpm -F @paddle-js-models/mobilenet publish",
|
||||
"lint": "eslint --ext .js,.ts packages --fix",
|
||||
"test": "echo \"Error: no test specified\" && exit 1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@commitlint/cli": "^17.1.2",
|
||||
"@commitlint/config-conventional": "^17.1.0",
|
||||
"@typescript-eslint/eslint-plugin": "^5.37.0",
|
||||
"@typescript-eslint/parser": "^5.37.0",
|
||||
"commitlint": "^17.1.2",
|
||||
"eslint": "8.22.0",
|
||||
"husky": "^8.0.1",
|
||||
"lint-staged": "^13.0.3",
|
||||
"typescript": "^4.8.3"
|
||||
}
|
||||
}
|
||||
@@ -22,7 +22,7 @@
|
||||
"scripts": {
|
||||
"dev": "yalc publish --push",
|
||||
"prepublish": "pnpm lint & pnpm test",
|
||||
"prepublishOnly": "pnpm build & pnpm changelog",
|
||||
"prepublishOnly": "pnpm build",
|
||||
"build": "gulp build",
|
||||
"lint": "eslint --ext .js,.ts src --fix",
|
||||
"api": "api-extractor run",
|
||||
@@ -22,7 +22,7 @@
|
||||
"scripts": {
|
||||
"dev": "yalc publish --push",
|
||||
"prepublish": "pnpm lint & pnpm test",
|
||||
"prepublishOnly": "pnpm build & pnpm changelog",
|
||||
"prepublishOnly": "pnpm build",
|
||||
"build": "gulp build",
|
||||
"lint": "eslint --ext .js,.ts src --fix",
|
||||
"api": "api-extractor run",
|
||||
@@ -22,7 +22,7 @@
|
||||
"scripts": {
|
||||
"dev": "yalc publish --push",
|
||||
"prepublish": "pnpm lint & pnpm test",
|
||||
"prepublishOnly": "pnpm build & pnpm changelog",
|
||||
"prepublishOnly": "pnpm build",
|
||||
"build": "gulp build",
|
||||
"lint": "eslint --ext .js,.ts src --fix",
|
||||
"api": "api-extractor run",
|
||||
@@ -22,7 +22,7 @@
|
||||
"scripts": {
|
||||
"dev": "yalc publish --push",
|
||||
"prepublish": "pnpm lint & pnpm test",
|
||||
"prepublishOnly": "pnpm build & pnpm changelog",
|
||||
"prepublishOnly": "pnpm build",
|
||||
"build": "gulp build",
|
||||
"lint": "eslint --ext .js,.ts src --fix",
|
||||
"api": "api-extractor run",
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user