mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-05 16:48:03 +08:00
[Model] add Paddle.js web demo (#392)
* add application include paddle.js web demo and xcx * cp PR #5 * add readme * fix comments and link * fix xcx readme * fix Task 1 * fix bugs * refine readme * delete ocrxcx readme * refine readme * fix bugs * delete old readme * 200px to 300px * revert 200px to 300px Co-authored-by: Jason <jiangjiajun@baidu.com>
This commit is contained in:
35
examples/application/js/README.md
Normal file
35
examples/application/js/README.md
Normal file
@@ -0,0 +1,35 @@
|
||||
|
||||
# 前端AI应用
|
||||
|
||||
人工智能技术的快速发展带动了计算机视觉、自然语言处理领域的产业升级。另外,随着PC和移动设备上算力的稳步增强、模型压缩技术迭代更新以及各种创新需求的不断催生,在浏览器中部署AI模型实现前端智能已经具备了良好的基础条件。
|
||||
针对前端部署AI深度学习模型困难的问题,百度开源了Paddle.js前端深度学习模型部署框架,可以很容易的将深度学习模型部署到前端项目中。
|
||||
|
||||
## Paddle.js简介
|
||||
|
||||
[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)是百度`PaddlePaddle`的web方向子项目,是一个运行在浏览器中的开源深度学习框架。`Paddle.js`可以加载`PaddlePaddle`动转静的模型,经过`Paddle.js`的模型转换工具`paddlejs-converter`转换成浏览器友好的模型,易于在线推理预测使用。`Paddle.js`支持`WebGL/WebGPU/WebAssembly`的浏览器中运行,也可以在百度小程序和微信小程序环境下运行。
|
||||
|
||||
简言之,利用Paddle.js,我们可以在浏览器、小程序等前端应用场景上线AI功能,包括但不限于目标检测,图像分割,OCR,物品分类等AI能力。
|
||||
|
||||
## Web Demo使用
|
||||
|
||||
在浏览器中直接运行官方demo参考[文档](./web_demo/README.md)
|
||||
|
||||
|demo名称|web demo目录|可视化|
|
||||
|-|-|-|
|
||||
|目标检测|[ScrewDetection/FaceDetection](./web_demo/demo/src/pages/cv/detection/)| <img src="https://user-images.githubusercontent.com/26592129/196874536-b7fa2c0a-d71f-4271-8c40-f9088bfad3c9.png" height="200px">|
|
||||
|人像分割背景替换|[HumanSeg](./web_demo//demo/src/pages/cv/segmentation/HumanSeg)|<img src="https://user-images.githubusercontent.com/26592129/196874452-4ef2e770-fbb3-4a35-954b-f871716d6669.png" height="200px">|
|
||||
|物体识别|[GestureRecognition/ItemIdentification](./web_demo//demo/src/pages/cv/recognition/)|<img src="https://user-images.githubusercontent.com/26592129/196874416-454e6bb0-4ebd-4b51-a88a-8c40614290ae.png" height="200px">|
|
||||
|OCR|[TextDetection/TextRecognition](./web_demo//demo/src/pages/cv/ocr/)|<img src="https://user-images.githubusercontent.com/26592129/196874354-1b5eecb0-f273-403c-aa6c-4463bf6d78db.png" height="200px">|
|
||||
|
||||
|
||||
## 微信小程序Demo使用
|
||||
|
||||
在微信小程序运行官方demo参考[文档](./mini_program/README.md)
|
||||
|
||||
|名称|目录|
|
||||
|-|-|
|
||||
|OCR文本检测| [ocrdetecXcx](./mini_program/ocrdetectXcx/) |
|
||||
|OCR文本识别| [ocrXcx](./mini_program/ocrXcx/) |
|
||||
|目标检测| coming soon |
|
||||
|图像分割| coming soon |
|
||||
|物品分类| coming soon |
|
126
examples/application/js/mini_program/README.md
Normal file
126
examples/application/js/mini_program/README.md
Normal file
@@ -0,0 +1,126 @@
|
||||
|
||||
# Paddle.js微信小程序Demo
|
||||
|
||||
- [1.简介](#1)
|
||||
- [2. 项目启动](#2)
|
||||
* [2.1 准备工作](#21)
|
||||
* [2.2 启动步骤](#22)
|
||||
* [2.3 效果展示](#23)
|
||||
- [3. 模型推理pipeline](#3)
|
||||
- [4. 常见问题](#4)
|
||||
|
||||
<a name="1"></a>
|
||||
## 1.简介
|
||||
|
||||
|
||||
本目录下包含文本检测、文本识别小程序demo,通过使用 [Paddle.js](https://github.com/PaddlePaddle/Paddle.js) 以及 [Paddle.js微信小程序插件](https://mp.weixin.qq.com/wxopen/plugindevdoc?appid=wx7138a7bb793608c3&token=956931339&lang=zh_CN) 完成在小程序上利用用户终端算力实现文本检测框选效果。
|
||||
|
||||
<a name="2"></a>
|
||||
## 2. 项目启动
|
||||
|
||||
<a name="21"></a>
|
||||
### 2.1 准备工作
|
||||
* [申请微信小程序账号](https://mp.weixin.qq.com/)
|
||||
* [微信小程序开发者工具](https://developers.weixin.qq.com/miniprogram/dev/devtools/download.html)
|
||||
* 前端开发环境准备:node、npm
|
||||
* 小程序管理后台配置服务器域名,或打开开发者工具【不校验合法域名】
|
||||
|
||||
详情参考:https://mp.weixin.qq.com/wxamp/devprofile/get_profile?token=1132303404&lang=zh_CN)
|
||||
|
||||
<a name="22"></a>
|
||||
### 2.2 启动步骤
|
||||
|
||||
#### **1. 克隆Demo代码**
|
||||
```sh
|
||||
git clone https://github.com/PaddlePaddle/FastDeploy
|
||||
cd FastDeploy/examples/application/js/mini_program
|
||||
```
|
||||
|
||||
#### **2. 进入小程序目录,安装依赖**
|
||||
|
||||
```sh
|
||||
# 运行文本识别demo,进入到ocrXcx目录
|
||||
cd ./ocrXcx && npm install
|
||||
# 运行文本检测demo,进入到ocrdetectXcx目录
|
||||
# cd ./ocrdetectXcx && npm install
|
||||
```
|
||||
|
||||
#### **3. 微信小程序导入代码**
|
||||
打开微信开发者工具 --> 导入 --> 选定目录,输入相关信息
|
||||
|
||||
#### **4. 添加 Paddle.js微信小程序插件**
|
||||
小程序管理界面 --> 设置 --> 第三方设置 --> 插件管理 --> 添加插件 --> 搜索 `wx7138a7bb793608c3` 并添加
|
||||
[参考文档](https://developers.weixin.qq.com/miniprogram/dev/framework/plugin/using.html)
|
||||
|
||||
#### **5. 构建依赖**
|
||||
点击开发者工具中的菜单栏:工具 --> 构建 npm
|
||||
|
||||
原因:node_modules 目录不会参与编译、上传和打包中,小程序想要使用 npm 包必须走一遍“构建 npm”的过程,构建完成会生成一个 miniprogram_npm 目录,里面会存放构建打包后的 npm 包,也就是小程序真正使用的 npm 包。*
|
||||
[参考文档](https://developers.weixin.qq.com/miniprogram/dev/devtools/npm.html)
|
||||
|
||||
<a name="23"></a>
|
||||
### 2.3 效果展示
|
||||
|
||||
<img src="https://user-images.githubusercontent.com/43414102/157648579-cdbbee61-9866-4364-9edd-a97ac0eda0c1.png" width="300px">
|
||||
|
||||
<a name="3"></a>
|
||||
## 3. 模型推理pipeline
|
||||
|
||||
```typescript
|
||||
// 引入 paddlejs 和 paddlejs-plugin,注册小程序环境变量和合适的 backend
|
||||
import * as paddlejs from '@paddlejs/paddlejs-core';
|
||||
import '@paddlejs/paddlejs-backend-webgl';
|
||||
const plugin = requirePlugin('paddlejs-plugin');
|
||||
plugin.register(paddlejs, wx);
|
||||
|
||||
// 初始化推理引擎
|
||||
const runner = new paddlejs.Runner({modelPath, feedShape, mean, std});
|
||||
await runner.init();
|
||||
|
||||
// 获取图像信息
|
||||
wx.canvasGetImageData({
|
||||
canvasId: canvasId,
|
||||
x: 0,
|
||||
y: 0,
|
||||
width: canvas.width,
|
||||
height: canvas.height,
|
||||
success(res) {
|
||||
// 推理预测
|
||||
runner.predict({
|
||||
data: res.data,
|
||||
width: canvas.width,
|
||||
height: canvas.height,
|
||||
}, function (data) {
|
||||
// 获取推理结果
|
||||
console.log(data)
|
||||
});
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
<a name="4"></a>
|
||||
## 4. 常见问题
|
||||
### 4.1 出现报错 `Invalid context type [webgl2] for Canvas#getContext`
|
||||
|
||||
可以不管,不影响正常代码运行和demo功能
|
||||
|
||||
### 4.2 预览看不到结果
|
||||
|
||||
建议尝试真机调试
|
||||
|
||||
### 4.3 微信开发者工具出现黑屏,然后出现超多报错
|
||||
|
||||
重启微信开发者工具
|
||||
|
||||
### 4.4 模拟和真机调试结果不一致;模拟检测不到文本等
|
||||
|
||||
可以以真机为准;
|
||||
|
||||
模拟检测不到文本等可以尝试随意改动下代码(增删换行等)再点击编译
|
||||
|
||||
|
||||
### 4.5 手机调试或运行时出现 长时间无反应等提示
|
||||
|
||||
请继续等待,模型推理需要一定时间
|
||||
|
||||
|
12
examples/application/js/mini_program/ocrXcx/app.js
Normal file
12
examples/application/js/mini_program/ocrXcx/app.js
Normal file
@@ -0,0 +1,12 @@
|
||||
/* global wx, App */
|
||||
import * as paddlejs from '@paddlejs/paddlejs-core';
|
||||
import '@paddlejs/paddlejs-backend-webgl';
|
||||
// eslint-disable-next-line no-undef
|
||||
const plugin = requirePlugin('paddlejs-plugin');
|
||||
plugin.register(paddlejs, wx);
|
||||
|
||||
App({
|
||||
globalData: {
|
||||
Paddlejs: paddlejs.Runner
|
||||
}
|
||||
});
|
12
examples/application/js/mini_program/ocrXcx/app.json
Normal file
12
examples/application/js/mini_program/ocrXcx/app.json
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"pages": [
|
||||
"pages/index/index"
|
||||
],
|
||||
"plugins": {
|
||||
"paddlejs-plugin": {
|
||||
"version": "2.0.1",
|
||||
"provider": "wx7138a7bb793608c3"
|
||||
}
|
||||
},
|
||||
"sitemapLocation": "sitemap.json"
|
||||
}
|
72
examples/application/js/mini_program/ocrXcx/package-lock.json
generated
Normal file
72
examples/application/js/mini_program/ocrXcx/package-lock.json
generated
Normal file
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"name": "paddlejs-demo",
|
||||
"version": "0.0.1",
|
||||
"lockfileVersion": 2,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "paddlejs-demo",
|
||||
"version": "0.0.1",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
|
||||
"@paddlejs/paddlejs-core": "^2.1.18",
|
||||
"d3-polygon": "2.0.0",
|
||||
"js-clipper": "1.0.1",
|
||||
"number-precision": "1.5.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@paddlejs/paddlejs-backend-webgl": {
|
||||
"version": "1.2.9",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.9.tgz",
|
||||
"integrity": "sha512-cVDa0/Wbw2EyfsYqdYUPhFeqKsET79keEUWjyhYQmQkJfWg8j1qdR6yp7g6nx9qAGrqFvwuj1s0EqkYA1dok6A=="
|
||||
},
|
||||
"node_modules/@paddlejs/paddlejs-core": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.2.0.tgz",
|
||||
"integrity": "sha512-P3rPkF9fFHtq8uSte5gA7fJQwBNl9Ytsvj6aTcfQSsirnBO/HxMNu0gJyh7+lItvEtF92PR15eI0eOwJYfZDhQ=="
|
||||
},
|
||||
"node_modules/d3-polygon": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
|
||||
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
|
||||
},
|
||||
"node_modules/js-clipper": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
|
||||
"integrity": "sha512-0XYAS0ZoCki5K0fWwj8j8ug4mgxHXReW3ayPbVqr4zXPJuIs2pyvemL1sALadsEiAywZwW5Ify1XfU4bNJvokg=="
|
||||
},
|
||||
"node_modules/number-precision": {
|
||||
"version": "1.5.2",
|
||||
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
|
||||
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
|
||||
}
|
||||
},
|
||||
"dependencies": {
|
||||
"@paddlejs/paddlejs-backend-webgl": {
|
||||
"version": "1.2.9",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.9.tgz",
|
||||
"integrity": "sha512-cVDa0/Wbw2EyfsYqdYUPhFeqKsET79keEUWjyhYQmQkJfWg8j1qdR6yp7g6nx9qAGrqFvwuj1s0EqkYA1dok6A=="
|
||||
},
|
||||
"@paddlejs/paddlejs-core": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.2.0.tgz",
|
||||
"integrity": "sha512-P3rPkF9fFHtq8uSte5gA7fJQwBNl9Ytsvj6aTcfQSsirnBO/HxMNu0gJyh7+lItvEtF92PR15eI0eOwJYfZDhQ=="
|
||||
},
|
||||
"d3-polygon": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
|
||||
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
|
||||
},
|
||||
"js-clipper": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
|
||||
"integrity": "sha512-0XYAS0ZoCki5K0fWwj8j8ug4mgxHXReW3ayPbVqr4zXPJuIs2pyvemL1sALadsEiAywZwW5Ify1XfU4bNJvokg=="
|
||||
},
|
||||
"number-precision": {
|
||||
"version": "1.5.2",
|
||||
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
|
||||
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
|
||||
}
|
||||
}
|
||||
}
|
19
examples/application/js/mini_program/ocrXcx/package.json
Normal file
19
examples/application/js/mini_program/ocrXcx/package.json
Normal file
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "paddlejs-demo",
|
||||
"version": "0.0.1",
|
||||
"description": "",
|
||||
"main": "app.js",
|
||||
"dependencies": {
|
||||
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
|
||||
"@paddlejs/paddlejs-core": "^2.1.18",
|
||||
"d3-polygon": "2.0.0",
|
||||
"js-clipper": "1.0.1",
|
||||
"number-precision": "1.5.2"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1"
|
||||
},
|
||||
"keywords": [],
|
||||
"author": "",
|
||||
"license": "ISC"
|
||||
}
|
Binary file not shown.
After Width: | Height: | Size: 222 KiB |
578
examples/application/js/mini_program/ocrXcx/pages/index/index.js
Normal file
578
examples/application/js/mini_program/ocrXcx/pages/index/index.js
Normal file
@@ -0,0 +1,578 @@
|
||||
/* global wx, Page */
|
||||
import * as paddlejs from '@paddlejs/paddlejs-core';
|
||||
import '@paddlejs/paddlejs-backend-webgl';
|
||||
import clipper from 'js-clipper';
|
||||
import { divide, enableBoundaryChecking, plus } from 'number-precision';
|
||||
|
||||
import { recDecode } from 'recPostprocess.js';
|
||||
// eslint-disable-next-line no-undef
|
||||
const plugin = requirePlugin('paddlejs-plugin');
|
||||
const Polygon = require('d3-polygon');
|
||||
|
||||
global.wasm_url = 'pages/index/wasm/opencv_js.wasm.br';
|
||||
const CV = require('./wasm/opencv.js');
|
||||
|
||||
plugin.register(paddlejs, wx);
|
||||
|
||||
let DETSHAPE = 960;
|
||||
let RECWIDTH = 320;
|
||||
const RECHEIGHT = 32;
|
||||
|
||||
// 声明后续图像变换要用到的canvas;此时未绑定
|
||||
let canvas_det;
|
||||
let canvas_rec;
|
||||
let my_canvas;
|
||||
let my_canvas_ctx;
|
||||
|
||||
|
||||
const imgList = [
|
||||
'https://paddlejs.bj.bcebos.com/xcx/ocr.png'
|
||||
];
|
||||
|
||||
// eslint-disable-next-line max-lines-per-function
|
||||
const outputBox = (res) => {
|
||||
const thresh = 0.3;
|
||||
const box_thresh = 0.5;
|
||||
const max_candidates = 1000;
|
||||
const min_size = 3;
|
||||
const width = 960;
|
||||
const height = 960;
|
||||
const pred = res;
|
||||
const segmentation = [];
|
||||
pred.forEach(item => {
|
||||
segmentation.push(item > thresh ? 255 : 0);
|
||||
});
|
||||
|
||||
function get_mini_boxes(contour) {
|
||||
// 生成最小外接矩形
|
||||
const bounding_box = CV.minAreaRect(contour);
|
||||
const points = [];
|
||||
const mat = new CV.Mat();
|
||||
// 获取矩形的四个顶点坐标
|
||||
CV.boxPoints(bounding_box, mat);
|
||||
for (let i = 0; i < mat.data32F.length; i += 2) {
|
||||
const arr = [];
|
||||
arr[0] = mat.data32F[i];
|
||||
arr[1] = mat.data32F[i + 1];
|
||||
points.push(arr);
|
||||
}
|
||||
|
||||
function sortNumber(a, b) {
|
||||
return a[0] - b[0];
|
||||
}
|
||||
points.sort(sortNumber);
|
||||
let index_1 = 0;
|
||||
let index_2 = 1;
|
||||
let index_3 = 2;
|
||||
let index_4 = 3;
|
||||
if (points[1][1] > points[0][1]) {
|
||||
index_1 = 0;
|
||||
index_4 = 1;
|
||||
}
|
||||
else {
|
||||
index_1 = 1;
|
||||
index_4 = 0;
|
||||
}
|
||||
|
||||
if (points[3][1] > points[2][1]) {
|
||||
index_2 = 2;
|
||||
index_3 = 3;
|
||||
}
|
||||
else {
|
||||
index_2 = 3;
|
||||
index_3 = 2;
|
||||
}
|
||||
const box = [
|
||||
points[index_1],
|
||||
points[index_2],
|
||||
points[index_3],
|
||||
points[index_4]
|
||||
];
|
||||
const side = Math.min(bounding_box.size.height, bounding_box.size.width);
|
||||
mat.delete();
|
||||
return {
|
||||
points: box,
|
||||
side
|
||||
};
|
||||
}
|
||||
|
||||
function box_score_fast(bitmap, _box) {
|
||||
const h = height;
|
||||
const w = width;
|
||||
const box = JSON.parse(JSON.stringify(_box));
|
||||
const x = [];
|
||||
const y = [];
|
||||
box.forEach(item => {
|
||||
x.push(item[0]);
|
||||
y.push(item[1]);
|
||||
});
|
||||
// clip这个函数将将数组中的元素限制在a_min, a_max之间,大于a_max的就使得它等于 a_max,小于a_min,的就使得它等于a_min。
|
||||
const xmin = clip(Math.floor(Math.min(...x)), 0, w - 1);
|
||||
const xmax = clip(Math.ceil(Math.max(...x)), 0, w - 1);
|
||||
const ymin = clip(Math.floor(Math.min(...y)), 0, h - 1);
|
||||
const ymax = clip(Math.ceil(Math.max(...y)), 0, h - 1);
|
||||
// eslint-disable-next-line new-cap
|
||||
const mask = new CV.Mat.zeros(ymax - ymin + 1, xmax - xmin + 1, CV.CV_8UC1);
|
||||
box.forEach(item => {
|
||||
item[0] = Math.max(item[0] - xmin, 0);
|
||||
item[1] = Math.max(item[1] - ymin, 0);
|
||||
});
|
||||
const npts = 4;
|
||||
const point_data = new Uint8Array(box.flat());
|
||||
const points = CV.matFromArray(npts, 1, CV.CV_32SC2, point_data);
|
||||
const pts = new CV.MatVector();
|
||||
pts.push_back(points);
|
||||
const color = new CV.Scalar(255);
|
||||
// 多个多边形填充
|
||||
CV.fillPoly(mask, pts, color, 1);
|
||||
const sliceArr = [];
|
||||
for (let i = ymin; i < ymax + 1; i++) {
|
||||
sliceArr.push(...bitmap.slice(960 * i + xmin, 960 * i + xmax + 1));
|
||||
}
|
||||
const mean = num_mean(sliceArr, mask.data);
|
||||
mask.delete();
|
||||
points.delete();
|
||||
pts.delete();
|
||||
return mean;
|
||||
}
|
||||
|
||||
function clip(data, min, max) {
|
||||
return data < min ? min : data > max ? max : data;
|
||||
}
|
||||
|
||||
function unclip(box) {
|
||||
const unclip_ratio = 1.6;
|
||||
const area = Math.abs(Polygon.polygonArea(box));
|
||||
const length = Polygon.polygonLength(box);
|
||||
const distance = area * unclip_ratio / length;
|
||||
const tmpArr = [];
|
||||
box.forEach(item => {
|
||||
const obj = {
|
||||
X: 0,
|
||||
Y: 0
|
||||
};
|
||||
obj.X = item[0];
|
||||
obj.Y = item[1];
|
||||
tmpArr.push(obj);
|
||||
});
|
||||
const offset = new clipper.ClipperOffset();
|
||||
offset.AddPath(tmpArr, clipper.JoinType.jtRound, clipper.EndType.etClosedPolygon);
|
||||
const expanded = [];
|
||||
offset.Execute(expanded, distance);
|
||||
let expandedArr = [];
|
||||
expanded[0] && expanded[0].forEach(item => {
|
||||
expandedArr.push([item.X, item.Y]);
|
||||
});
|
||||
expandedArr = [].concat(...expandedArr);
|
||||
return expandedArr;
|
||||
}
|
||||
|
||||
function num_mean(data, mask) {
|
||||
let sum = 0;
|
||||
let length = 0;
|
||||
for (let i = 0; i < data.length; i++) {
|
||||
if (mask[i]) {
|
||||
sum = plus(sum, data[i]);
|
||||
length++;
|
||||
}
|
||||
}
|
||||
return divide(sum, length);
|
||||
}
|
||||
|
||||
// eslint-disable-next-line new-cap
|
||||
const src = new CV.matFromArray(960, 960, CV.CV_8UC1, segmentation);
|
||||
const contours = new CV.MatVector();
|
||||
const hierarchy = new CV.Mat();
|
||||
// 获取轮廓
|
||||
CV.findContours(src, contours, hierarchy, CV.RETR_LIST, CV.CHAIN_APPROX_SIMPLE);
|
||||
const num_contours = Math.min(contours.size(), max_candidates);
|
||||
const boxes = [];
|
||||
const scores = [];
|
||||
const arr = [];
|
||||
for (let i = 0; i < num_contours; i++) {
|
||||
const contour = contours.get(i);
|
||||
let {
|
||||
points,
|
||||
side
|
||||
} = get_mini_boxes(contour);
|
||||
if (side < min_size) {
|
||||
continue;
|
||||
}
|
||||
const score = box_score_fast(pred, points);
|
||||
if (box_thresh > score) {
|
||||
continue;
|
||||
}
|
||||
let box = unclip(points);
|
||||
// eslint-disable-next-line new-cap
|
||||
const boxMap = new CV.matFromArray(box.length / 2, 1, CV.CV_32SC2, box);
|
||||
const resultObj = get_mini_boxes(boxMap);
|
||||
box = resultObj.points;
|
||||
side = resultObj.side;
|
||||
if (side < min_size + 2) {
|
||||
continue;
|
||||
}
|
||||
box.forEach(item => {
|
||||
item[0] = clip(Math.round(item[0]), 0, 960);
|
||||
item[1] = clip(Math.round(item[1]), 0, 960);
|
||||
});
|
||||
boxes.push(box);
|
||||
scores.push(score);
|
||||
arr.push(i);
|
||||
boxMap.delete();
|
||||
}
|
||||
src.delete();
|
||||
contours.delete();
|
||||
hierarchy.delete();
|
||||
return {
|
||||
boxes,
|
||||
scores
|
||||
};
|
||||
};
|
||||
|
||||
const sorted_boxes = (box) => {
|
||||
function sortNumber(a, b) {
|
||||
return a[0][1] - b[0][1];
|
||||
}
|
||||
|
||||
const boxes = box.sort(sortNumber);
|
||||
const num_boxes = boxes.length;
|
||||
for (let i = 0; i < num_boxes - 1; i++) {
|
||||
if (Math.abs(boxes[i + 1][0][1] - boxes[i][0][1]) < 10
|
||||
&& boxes[i + 1][0][0] < boxes[i][0][0]) {
|
||||
const tmp = boxes[i];
|
||||
boxes[i] = boxes[i + 1];
|
||||
boxes[i + 1] = tmp;
|
||||
}
|
||||
}
|
||||
return boxes;
|
||||
}
|
||||
|
||||
function flatten(arr) {
|
||||
return arr.toString().split(',').map(item => +item);
|
||||
}
|
||||
|
||||
function int(num) {
|
||||
return num > 0 ? Math.floor(num) : Math.ceil(num);
|
||||
}
|
||||
|
||||
function clip(data, min, max) {
|
||||
return data < min ? min : data > max ? max : data;
|
||||
}
|
||||
|
||||
function get_rotate_crop_image(img, points) {
|
||||
const img_crop_width = int(Math.max(
|
||||
linalg_norm(points[0], points[1]),
|
||||
linalg_norm(points[2], points[3])
|
||||
));
|
||||
const img_crop_height = int(Math.max(
|
||||
linalg_norm(points[0], points[3]),
|
||||
linalg_norm(points[1], points[2])
|
||||
));
|
||||
const pts_std = [
|
||||
[0, 0],
|
||||
[img_crop_width, 0],
|
||||
[img_crop_width, img_crop_height],
|
||||
[0, img_crop_height]
|
||||
];
|
||||
const srcTri = CV.matFromArray(4, 1, CV.CV_32FC2, flatten(points));
|
||||
const dstTri = CV.matFromArray(4, 1, CV.CV_32FC2, flatten(pts_std));
|
||||
// 获取到目标矩阵
|
||||
const M = CV.getPerspectiveTransform(srcTri, dstTri);
|
||||
const src = CV.imread(img);
|
||||
const dst = new CV.Mat();
|
||||
const dsize = new CV.Size(img_crop_width, img_crop_height);
|
||||
// 透视转换
|
||||
CV.warpPerspective(src, dst, M, dsize, CV.INTER_CUBIC, CV.BORDER_REPLICATE, new CV.Scalar());
|
||||
|
||||
const dst_img_height = dst.rows;
|
||||
const dst_img_width = dst.cols;
|
||||
let dst_rot;
|
||||
// 图像旋转
|
||||
if (dst_img_height / dst_img_width >= 1.5) {
|
||||
dst_rot = new CV.Mat();
|
||||
const dsize_rot = new CV.Size(dst.rows, dst.cols);
|
||||
const center = new CV.Point(dst.cols / 2, dst.cols / 2);
|
||||
const M = CV.getRotationMatrix2D(center, 90, 1);
|
||||
CV.warpAffine(dst, dst_rot, M, dsize_rot, CV.INTER_CUBIC, CV.BORDER_REPLICATE, new CV.Scalar());
|
||||
}
|
||||
|
||||
const dst_resize = new CV.Mat();
|
||||
const dsize_resize = new CV.Size(0, 0);
|
||||
let scale;
|
||||
if (dst_rot) {
|
||||
scale = RECHEIGHT / dst_rot.rows;
|
||||
CV.resize(dst_rot, dst_resize, dsize_resize, scale, scale, CV.INTER_AREA);
|
||||
dst_rot.delete();
|
||||
}
|
||||
else {
|
||||
scale = RECHEIGHT / dst_img_height;
|
||||
CV.resize(dst, dst_resize, dsize_resize, scale, scale, CV.INTER_AREA);
|
||||
}
|
||||
|
||||
canvas_det.width = dst_resize.cols;
|
||||
canvas_det.height = dst_resize.rows;
|
||||
canvas_det.getContext('2d').clearRect(0, 0, canvas_det.width, canvas_det.height);
|
||||
CV.imshow(canvas_det, dst_resize);
|
||||
|
||||
src.delete();
|
||||
dst.delete();
|
||||
dst_resize.delete();
|
||||
srcTri.delete();
|
||||
dstTri.delete();
|
||||
}
|
||||
|
||||
function linalg_norm(x, y) {
|
||||
return Math.sqrt(Math.pow(x[0] - y[0], 2) + Math.pow(x[1] - y[1], 2));
|
||||
}
|
||||
|
||||
function resize_norm_img_splice(
|
||||
image,
|
||||
origin_width,
|
||||
origin_height,
|
||||
index = 0
|
||||
) {
|
||||
canvas_rec.width = RECWIDTH;
|
||||
canvas_rec.height = RECHEIGHT;
|
||||
const ctx = canvas_rec.getContext('2d');
|
||||
ctx.fillStyle = '#fff';
|
||||
ctx.clearRect(0, 0, canvas_rec.width, canvas_rec.height);
|
||||
// ctx.drawImage(image, -index * RECWIDTH, 0, origin_width, origin_height);
|
||||
ctx.putImageData(image, -index * RECWIDTH, 0);
|
||||
}
|
||||
|
||||
// 声明检测和识别Runner;未初始化
|
||||
let detectRunner;
|
||||
let recRunner;
|
||||
|
||||
Page({
|
||||
data: {
|
||||
photo_src:'',
|
||||
imgList: imgList,
|
||||
imgInfo: {},
|
||||
result: '',
|
||||
select_mode: false,
|
||||
loaded: false
|
||||
},
|
||||
switch_choose(){
|
||||
this.setData({
|
||||
select_mode: true
|
||||
})
|
||||
},
|
||||
switch_example(){
|
||||
this.setData({
|
||||
select_mode: false
|
||||
})
|
||||
},
|
||||
chose_photo:function(evt){
|
||||
let _this = this
|
||||
wx.chooseImage({
|
||||
count: 1,
|
||||
sizeType: ['original', 'compressed'],
|
||||
sourceType: ['album', 'camera'],
|
||||
success(res) {
|
||||
console.log(res.tempFilePaths) //一个数组,每个元素都是“http://...”图片地址
|
||||
_this.setData({
|
||||
photo_src: res.tempFilePaths[0]
|
||||
})
|
||||
}
|
||||
})
|
||||
},
|
||||
reselect:function(evt){
|
||||
let _this = this
|
||||
wx.chooseImage({
|
||||
count: 1,
|
||||
sizeType: ['original', 'compressed'],
|
||||
sourceType: ['album', 'camera'],
|
||||
success(res) {
|
||||
_this.setData({
|
||||
photo_src: res.tempFilePaths[0]
|
||||
})
|
||||
}
|
||||
})
|
||||
},
|
||||
photo_preview:function(evt){
|
||||
let _this = this;
|
||||
let imgs = [];
|
||||
imgs.push(_this.data.photo_src);
|
||||
wx.previewImage({
|
||||
urls:imgs
|
||||
})
|
||||
},
|
||||
|
||||
predect_choose_img() {
|
||||
console.log(this.data.photo_src)
|
||||
this.getImageInfo(this.data.photo_src);
|
||||
},
|
||||
|
||||
onLoad() {
|
||||
enableBoundaryChecking(false);
|
||||
// 绑定canvas;该操作是异步,因此最好加延迟保证后续使用时已完成绑定
|
||||
wx.createSelectorQuery()
|
||||
.select('#canvas_det')
|
||||
.fields({ node: true, size: true })
|
||||
.exec(async(res) => {
|
||||
canvas_det = res[0].node;
|
||||
});
|
||||
|
||||
wx.createSelectorQuery()
|
||||
.select('#canvas_rec')
|
||||
.fields({ node: true, size: true })
|
||||
.exec(async(res) => {
|
||||
canvas_rec = res[0].node;
|
||||
});
|
||||
|
||||
wx.createSelectorQuery()
|
||||
.select('#myCanvas')
|
||||
.fields({ node: true, size: true })
|
||||
.exec((res) => {
|
||||
my_canvas = res[0].node;
|
||||
my_canvas_ctx = my_canvas.getContext('2d');
|
||||
});
|
||||
|
||||
const me = this;
|
||||
// 初始化Runner
|
||||
detectRunner = new paddlejs.Runner({
|
||||
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_det_infer_js_960/model.json',
|
||||
mean: [0.485, 0.456, 0.406],
|
||||
std: [0.229, 0.224, 0.225],
|
||||
bgr: true,
|
||||
webglFeedProcess: true
|
||||
});
|
||||
recRunner = new paddlejs.Runner({
|
||||
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_rec_infer_js/model.json',
|
||||
fill: '#000',
|
||||
mean: [0.5, 0.5, 0.5],
|
||||
std: [0.5, 0.5, 0.5],
|
||||
bgr: true,
|
||||
webglFeedProcess: true
|
||||
});
|
||||
// 等待模型数据全部加载完成
|
||||
Promise.all([detectRunner.init(), recRunner.init()]).then(_ => {
|
||||
me.setData({
|
||||
loaded: true
|
||||
});
|
||||
});
|
||||
|
||||
},
|
||||
|
||||
selectImage(event) {
|
||||
const imgPath = this.data.imgList[event.target.dataset.index];
|
||||
this.getImageInfo(imgPath);
|
||||
},
|
||||
|
||||
getImageInfo(imgPath) {
|
||||
const me = this;
|
||||
wx.getImageInfo({
|
||||
src: imgPath,
|
||||
success: (imgInfo) => {
|
||||
const {
|
||||
path,
|
||||
width,
|
||||
height
|
||||
} = imgInfo;
|
||||
const canvasPath = imgPath.includes('http') ? path : imgPath;
|
||||
|
||||
let sw = 960;
|
||||
let sh = 960;
|
||||
let x = 0;
|
||||
let y = 0;
|
||||
|
||||
if (height / width >= 1) {
|
||||
sw = Math.round(sh * width / height);
|
||||
x = Math.floor((960 - sw) / 2);
|
||||
}
|
||||
else {
|
||||
sh = Math.round(sw * height / width);
|
||||
y = Math.floor((960 - sh) / 2);
|
||||
}
|
||||
my_canvas.width = sw;
|
||||
my_canvas.height = sh;
|
||||
|
||||
// 微信上canvas输入图片
|
||||
const image = my_canvas.createImage();
|
||||
image.src = canvasPath;
|
||||
image.onload = () => {
|
||||
my_canvas_ctx.clearRect(0, 0, my_canvas.width, my_canvas.height);
|
||||
my_canvas_ctx.drawImage(image, x, y, sw, sh);
|
||||
const imageData = my_canvas_ctx.getImageData(0, 0, sw, sh);
|
||||
// 开始识别
|
||||
me.recognize({
|
||||
data: imageData.data,
|
||||
width: 960,
|
||||
height: 960
|
||||
}, {canvasPath, sw, sh, x, y});
|
||||
}
|
||||
}
|
||||
});
|
||||
},
|
||||
|
||||
async recognize(res, img) {
|
||||
const me = this;
|
||||
// 文本框选坐标点
|
||||
let points;
|
||||
await detectRunner.predict(res, function (detectRes) {
|
||||
points = outputBox(detectRes);
|
||||
});
|
||||
|
||||
// 绘制文本框
|
||||
me.drawCanvasPoints(img, points.boxes);
|
||||
|
||||
// 排序,使得最后结果输出尽量按照从上到下的顺序
|
||||
const boxes = sorted_boxes(points.boxes);
|
||||
|
||||
const text_list = [];
|
||||
|
||||
for (let i = 0; i < boxes.length; i++) {
|
||||
const tmp_box = JSON.parse(JSON.stringify(boxes[i]));
|
||||
// 获取tmp_box对应图片到canvas_det
|
||||
get_rotate_crop_image(res, tmp_box);
|
||||
// 这里是计算要识别文字的图片片段是否大于识别模型要求的输入宽度;超过了的话会分成多次识别,再拼接结果
|
||||
const width_num = Math.ceil(canvas_det.width / RECWIDTH);
|
||||
|
||||
let text_list_tmp = '';
|
||||
for (let j = 0; j < width_num; j++) {
|
||||
// 根据原图的宽度进行裁剪拼接,超出指定宽度会被截断;然后再次识别,最后拼接起来
|
||||
resize_norm_img_splice(canvas_det.getContext('2d').getImageData(0, 0, canvas_det.width, canvas_det.height), canvas_det.width, canvas_det.height, j);
|
||||
|
||||
const imgData = canvas_rec.getContext('2d').getImageData(0, 0, canvas_rec.width, canvas_rec.height);
|
||||
|
||||
await recRunner.predict(imgData, function(output){
|
||||
// 将输出向量转化为idx再传化为对应字符
|
||||
const text = recDecode(output);
|
||||
text_list_tmp = text_list_tmp.concat(text.text);
|
||||
});
|
||||
}
|
||||
text_list.push(text_list_tmp);
|
||||
}
|
||||
me.setData({
|
||||
result: JSON.stringify(boxes) + JSON.stringify(text_list)
|
||||
});
|
||||
},
|
||||
|
||||
drawCanvasPoints(img, points) {
|
||||
// 设置线条
|
||||
my_canvas_ctx.strokeStyle = 'blue';
|
||||
my_canvas_ctx.lineWidth = 5;
|
||||
|
||||
// 先绘制图片
|
||||
const image = my_canvas.createImage();
|
||||
image.src = img.canvasPath;
|
||||
image.onload = () => {
|
||||
my_canvas_ctx.clearRect(0, 0, my_canvas_ctx.width, my_canvas_ctx.height);
|
||||
my_canvas_ctx.drawImage(image, img.x, img.y, img.sw, img.sh);
|
||||
// 绘制线框
|
||||
points.length && points.forEach(point => {
|
||||
my_canvas_ctx.beginPath();
|
||||
// 设置路径起点坐标
|
||||
my_canvas_ctx.moveTo(point[0][0], point[0][1]);
|
||||
my_canvas_ctx.lineTo(point[1][0], point[1][1]);
|
||||
my_canvas_ctx.lineTo(point[2][0], point[2][1]);
|
||||
my_canvas_ctx.lineTo(point[3][0], point[3][1]);
|
||||
my_canvas_ctx.lineTo(point[0][0], point[0][1]);
|
||||
my_canvas_ctx.stroke();
|
||||
my_canvas_ctx.closePath();
|
||||
});
|
||||
}
|
||||
|
||||
}
|
||||
});
|
@@ -0,0 +1,2 @@
|
||||
{
|
||||
}
|
@@ -0,0 +1,56 @@
|
||||
<view>
|
||||
<view>
|
||||
<button bindtap="switch_choose" wx:if="{{!select_mode}}">切换选择本地图片模式</button>
|
||||
<button bindtap="switch_example" wx:else>切换示例图片模式</button>
|
||||
</view>
|
||||
<view class="photo_box" wx:if="{{select_mode}}">
|
||||
<view wx:if="{{photo_src != ''}}" class="photo_preview">
|
||||
<image src="{{photo_src}}" mode="aspectFit" bindtap="photo_preview"></image>
|
||||
<view class="reselect" bindtap="reselect">重新选择</view>
|
||||
</view>
|
||||
<view wx:else class="photo_text" bindtap="chose_photo">点击拍照或上传本地照片</view>
|
||||
<button bindtap="predect_choose_img">检测</button>
|
||||
</view>
|
||||
<view wx:else>
|
||||
<text class="title">点击图片进行预测</text>
|
||||
<scroll-view class="imgWrapper" scroll-x="true">
|
||||
<image
|
||||
class="img {{selectedIndex == index ? 'selected' : ''}}"
|
||||
wx:for="{{imgList}}"
|
||||
wx:key="index"
|
||||
src="{{item}}"
|
||||
mode="aspectFit"
|
||||
bindtap="selectImage"
|
||||
data-index="{{index}}"
|
||||
></image>
|
||||
</scroll-view>
|
||||
</view>
|
||||
<view class="img-view">
|
||||
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
|
||||
<canvas
|
||||
id="myCanvas"
|
||||
type="2d"
|
||||
style="width: 960px; height: 960px;"
|
||||
></canvas>
|
||||
</scroll-view>
|
||||
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
|
||||
<canvas
|
||||
id="canvas_det"
|
||||
type="2d"
|
||||
style="width: 960px; height: 960px;"
|
||||
></canvas>
|
||||
</scroll-view>
|
||||
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
|
||||
<canvas
|
||||
id="canvas_rec"
|
||||
type="2d"
|
||||
style="width: 960px; height: 960px;"
|
||||
></canvas>
|
||||
</scroll-view>
|
||||
<text class="result" wx:if="{{result}}" style="height: 300rpx;">文本框选坐标:{{result}}</text>
|
||||
</view>
|
||||
</view>
|
||||
|
||||
<view class="mask" wx:if="{{!loaded}}">
|
||||
<text class="loading">loading…</text>
|
||||
</view>
|
@@ -0,0 +1,78 @@
|
||||
.photo_box{
|
||||
width: 750rpx;
|
||||
border: 1px solid #cccccc;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
.photo_text{
|
||||
width: 100%;
|
||||
line-height: 500rpx;
|
||||
text-align: center;
|
||||
}
|
||||
.photo_preview image{
|
||||
width: 750rpx;
|
||||
}
|
||||
.photo_preview .reselect{
|
||||
width: 750rpx;
|
||||
height: 100rpx;
|
||||
background-color: #3F8EFF;
|
||||
text-align: center;
|
||||
line-height: 100rpx;
|
||||
border-top: 1px solid #cccccc;
|
||||
}
|
||||
|
||||
text {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.title {
|
||||
margin-top: 10px;
|
||||
font-size: 16px;
|
||||
line-height: 32px;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.imgWrapper {
|
||||
margin: 10px 10px 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.img {
|
||||
width: 960px;
|
||||
height: 960px;
|
||||
border: 1px solid #f1f1f1;
|
||||
}
|
||||
|
||||
.result {
|
||||
margin-top: 5px;
|
||||
}
|
||||
|
||||
.selected {
|
||||
border: 1px solid #999;
|
||||
}
|
||||
|
||||
.select-btn {
|
||||
margin-top: 20px;
|
||||
width: 60%;
|
||||
}
|
||||
|
||||
.mask {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
background-color: rgba(0, 0, 0, .7);
|
||||
}
|
||||
|
||||
.loading {
|
||||
color: #fff;
|
||||
font-size: 20px;
|
||||
position: absolute;
|
||||
top: 50%;
|
||||
left: 50%;
|
||||
transform: translate(-50%, -50%);
|
||||
}
|
||||
|
||||
.img-view {
|
||||
padding-bottom: 20px;
|
||||
border-bottom: 1px solid #f1f1f1;
|
||||
}
|
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -0,0 +1,63 @@
|
||||
import { character } from 'ppocr_keys_v1.js';
|
||||
|
||||
const ocr_character = character;
|
||||
let preds_idx = [];
|
||||
let preds_prob = [];
|
||||
|
||||
function init(preds) {
|
||||
preds_idx = [];
|
||||
preds_prob = [];
|
||||
// preds: [1, ?, 6625]
|
||||
const pred_len = 6625;
|
||||
for (let i = 0; i < preds.length; i += pred_len) {
|
||||
const tmpArr = preds.slice(i, i + pred_len - 1);
|
||||
const tmpMax = Math.max(...tmpArr);
|
||||
const tmpIdx = tmpArr.indexOf(tmpMax);
|
||||
preds_prob.push(tmpMax);
|
||||
preds_idx.push(tmpIdx);
|
||||
}
|
||||
}
|
||||
|
||||
function get_ignored_tokens() {
|
||||
return [0];
|
||||
}
|
||||
|
||||
function decode(text_index, text_prob, is_remove_duplicate = false) {
|
||||
const ignored_tokens = get_ignored_tokens();
|
||||
const char_list = [];
|
||||
const conf_list = [];
|
||||
for (let idx = 0; idx < text_index.length; idx++) {
|
||||
if (text_index[idx] in ignored_tokens) {
|
||||
continue;
|
||||
}
|
||||
if (is_remove_duplicate) {
|
||||
if (idx > 0 && text_index[idx - 1] === text_index[idx]) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
char_list.push(ocr_character[text_index[idx] - 1]);
|
||||
if (text_prob) {
|
||||
conf_list.push(text_prob[idx]);
|
||||
}
|
||||
else {
|
||||
conf_list.push(1);
|
||||
}
|
||||
}
|
||||
let text = '';
|
||||
let mean = 0;
|
||||
|
||||
if (char_list.length) {
|
||||
text = char_list.join('');
|
||||
let sum = 0;
|
||||
conf_list.forEach(item => {
|
||||
sum += item;
|
||||
});
|
||||
mean = sum / conf_list.length;
|
||||
}
|
||||
return { text, mean };
|
||||
}
|
||||
|
||||
export function recDecode(preds) {
|
||||
init(preds);
|
||||
return decode(preds_idx, preds_prob, true);
|
||||
}
|
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"description": "项目配置文件,详见文档:https://developers.weixin.qq.com/miniprogram/dev/devtools/projectconfig.html",
|
||||
"packOptions": {
|
||||
"ignore": [],
|
||||
"include": []
|
||||
},
|
||||
"setting": {
|
||||
"urlCheck": false,
|
||||
"es6": true,
|
||||
"enhance": true,
|
||||
"postcss": true,
|
||||
"preloadBackgroundData": false,
|
||||
"minified": true,
|
||||
"newFeature": false,
|
||||
"coverView": true,
|
||||
"nodeModules": true,
|
||||
"autoAudits": false,
|
||||
"showShadowRootInWxmlPanel": true,
|
||||
"scopeDataCheck": false,
|
||||
"uglifyFileName": false,
|
||||
"checkInvalidKey": true,
|
||||
"checkSiteMap": true,
|
||||
"uploadWithSourceMap": true,
|
||||
"compileHotReLoad": false,
|
||||
"lazyloadPlaceholderEnable": false,
|
||||
"useMultiFrameRuntime": true,
|
||||
"useApiHook": true,
|
||||
"useApiHostProcess": true,
|
||||
"babelSetting": {
|
||||
"ignore": [],
|
||||
"disablePlugins": [],
|
||||
"outputPath": ""
|
||||
},
|
||||
"enableEngineNative": false,
|
||||
"useIsolateContext": true,
|
||||
"userConfirmedBundleSwitch": false,
|
||||
"packNpmManually": false,
|
||||
"packNpmRelationList": [],
|
||||
"minifyWXSS": true,
|
||||
"disableUseStrict": false,
|
||||
"minifyWXML": true,
|
||||
"showES6CompileOption": false,
|
||||
"useCompilerPlugins": false,
|
||||
"useStaticServer": true,
|
||||
"ignoreUploadUnusedFiles": false
|
||||
},
|
||||
"compileType": "miniprogram",
|
||||
"libVersion": "2.22.1",
|
||||
"appid": "wx78461a9c81d1234c",
|
||||
"projectname": "mobilenet",
|
||||
"simulatorType": "wechat",
|
||||
"simulatorPluginLibVersion": {},
|
||||
"condition": {},
|
||||
"editorSetting": {
|
||||
"tabIndent": "insertSpaces",
|
||||
"tabSize": 2
|
||||
}
|
||||
}
|
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"projectname": "ocrXcx",
|
||||
"setting": {
|
||||
"compileHotReLoad": true
|
||||
},
|
||||
"description": "项目私有配置文件。此文件中的内容将覆盖 project.config.json 中的相同字段。项目的改动优先同步到此文件中。详见文档:https://developers.weixin.qq.com/miniprogram/dev/devtools/projectconfig.html",
|
||||
"libVersion": "2.23.4"
|
||||
}
|
7
examples/application/js/mini_program/ocrXcx/sitemap.json
Normal file
7
examples/application/js/mini_program/ocrXcx/sitemap.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"desc": "关于本文件的更多信息,请参考文档 https://developers.weixin.qq.com/miniprogram/dev/framework/sitemap.html",
|
||||
"rules": [{
|
||||
"action": "allow",
|
||||
"page": "*"
|
||||
}]
|
||||
}
|
12
examples/application/js/mini_program/ocrdetectXcx/app.js
Normal file
12
examples/application/js/mini_program/ocrdetectXcx/app.js
Normal file
@@ -0,0 +1,12 @@
|
||||
/* global wx, App */
|
||||
import * as paddlejs from '@paddlejs/paddlejs-core';
|
||||
import '@paddlejs/paddlejs-backend-webgl';
|
||||
// eslint-disable-next-line no-undef
|
||||
const plugin = requirePlugin('paddlejs-plugin');
|
||||
plugin.register(paddlejs, wx);
|
||||
|
||||
App({
|
||||
globalData: {
|
||||
Paddlejs: paddlejs.Runner
|
||||
}
|
||||
});
|
12
examples/application/js/mini_program/ocrdetectXcx/app.json
Normal file
12
examples/application/js/mini_program/ocrdetectXcx/app.json
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"pages": [
|
||||
"pages/index/index"
|
||||
],
|
||||
"plugins": {
|
||||
"paddlejs-plugin": {
|
||||
"version": "2.0.1",
|
||||
"provider": "wx7138a7bb793608c3"
|
||||
}
|
||||
},
|
||||
"sitemapLocation": "sitemap.json"
|
||||
}
|
33
examples/application/js/mini_program/ocrdetectXcx/package-lock.json
generated
Normal file
33
examples/application/js/mini_program/ocrdetectXcx/package-lock.json
generated
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"name": "paddlejs-demo",
|
||||
"version": "0.0.1",
|
||||
"lockfileVersion": 1,
|
||||
"requires": true,
|
||||
"dependencies": {
|
||||
"@paddlejs/paddlejs-backend-webgl": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.0.tgz",
|
||||
"integrity": "sha512-bKJKJkGldC3NPOuJyk+372z0XW1dd1D9lR0f9OHqWQboY0Mkah+gX+8tkerrNg+QjYz88IW0iJaRKB0jm+6d9g=="
|
||||
},
|
||||
"@paddlejs/paddlejs-core": {
|
||||
"version": "2.1.18",
|
||||
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.1.18.tgz",
|
||||
"integrity": "sha512-QrXxwaHm4llp1sxbUq/oCCqlYx4ciXanBn/Lfq09UqR4zkYi5SptacQlIxgJ70HOO6RWIxjWN4liQckMwa2TkA=="
|
||||
},
|
||||
"d3-polygon": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
|
||||
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
|
||||
},
|
||||
"js-clipper": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
|
||||
"integrity": "sha1-TWsHQ0pECOfBKeMiAc5m0hR07SE="
|
||||
},
|
||||
"number-precision": {
|
||||
"version": "1.5.2",
|
||||
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
|
||||
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
|
||||
}
|
||||
}
|
||||
}
|
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "paddlejs-demo",
|
||||
"version": "0.0.1",
|
||||
"description": "",
|
||||
"main": "app.js",
|
||||
"dependencies": {
|
||||
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
|
||||
"@paddlejs/paddlejs-core": "^2.1.18",
|
||||
"d3-polygon": "2.0.0",
|
||||
"js-clipper": "1.0.1",
|
||||
"number-precision": "1.5.2"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1"
|
||||
},
|
||||
"keywords": [],
|
||||
"author": "",
|
||||
"license": "ISC"
|
||||
}
|
Binary file not shown.
After Width: | Height: | Size: 222 KiB |
@@ -0,0 +1,337 @@
|
||||
/* global wx, Page */
|
||||
import * as paddlejs from '@paddlejs/paddlejs-core';
|
||||
import '@paddlejs/paddlejs-backend-webgl';
|
||||
import clipper from 'js-clipper';
|
||||
import { divide, enableBoundaryChecking, plus } from 'number-precision';
|
||||
// eslint-disable-next-line no-undef
|
||||
const plugin = requirePlugin('paddlejs-plugin');
|
||||
const Polygon = require('d3-polygon');
|
||||
|
||||
global.wasm_url = 'pages/index/wasm/opencv_js.wasm.br';
|
||||
const CV = require('./wasm/opencv.js');
|
||||
|
||||
plugin.register(paddlejs, wx);
|
||||
|
||||
const imgList = [
|
||||
'https://paddlejs.bj.bcebos.com/xcx/ocr.png',
|
||||
'./img/width.png'
|
||||
];
|
||||
|
||||
// eslint-disable-next-line max-lines-per-function
|
||||
const outputBox = res => {
|
||||
const thresh = 0.3;
|
||||
const box_thresh = 0.5;
|
||||
const max_candidates = 1000;
|
||||
const min_size = 3;
|
||||
const width = 960;
|
||||
const height = 960;
|
||||
const pred = res;
|
||||
const segmentation = [];
|
||||
pred.forEach(item => {
|
||||
segmentation.push(item > thresh ? 255 : 0);
|
||||
});
|
||||
|
||||
function get_mini_boxes(contour) {
|
||||
// 生成最小外接矩形
|
||||
const bounding_box = CV.minAreaRect(contour);
|
||||
const points = [];
|
||||
const mat = new CV.Mat();
|
||||
// 获取矩形的四个顶点坐标
|
||||
CV.boxPoints(bounding_box, mat);
|
||||
for (let i = 0; i < mat.data32F.length; i += 2) {
|
||||
const arr = [];
|
||||
arr[0] = mat.data32F[i];
|
||||
arr[1] = mat.data32F[i + 1];
|
||||
points.push(arr);
|
||||
}
|
||||
|
||||
function sortNumber(a, b) {
|
||||
return a[0] - b[0];
|
||||
}
|
||||
points.sort(sortNumber);
|
||||
let index_1 = 0;
|
||||
let index_2 = 1;
|
||||
let index_3 = 2;
|
||||
let index_4 = 3;
|
||||
if (points[1][1] > points[0][1]) {
|
||||
index_1 = 0;
|
||||
index_4 = 1;
|
||||
}
|
||||
else {
|
||||
index_1 = 1;
|
||||
index_4 = 0;
|
||||
}
|
||||
|
||||
if (points[3][1] > points[2][1]) {
|
||||
index_2 = 2;
|
||||
index_3 = 3;
|
||||
}
|
||||
else {
|
||||
index_2 = 3;
|
||||
index_3 = 2;
|
||||
}
|
||||
const box = [
|
||||
points[index_1],
|
||||
points[index_2],
|
||||
points[index_3],
|
||||
points[index_4]
|
||||
];
|
||||
const side = Math.min(bounding_box.size.height, bounding_box.size.width);
|
||||
mat.delete();
|
||||
return {
|
||||
points: box,
|
||||
side
|
||||
};
|
||||
}
|
||||
|
||||
function box_score_fast(bitmap, _box) {
|
||||
const h = height;
|
||||
const w = width;
|
||||
const box = JSON.parse(JSON.stringify(_box));
|
||||
const x = [];
|
||||
const y = [];
|
||||
box.forEach(item => {
|
||||
x.push(item[0]);
|
||||
y.push(item[1]);
|
||||
});
|
||||
// clip这个函数将将数组中的元素限制在a_min, a_max之间,大于a_max的就使得它等于 a_max,小于a_min,的就使得它等于a_min。
|
||||
const xmin = clip(Math.floor(Math.min(...x)), 0, w - 1);
|
||||
const xmax = clip(Math.ceil(Math.max(...x)), 0, w - 1);
|
||||
const ymin = clip(Math.floor(Math.min(...y)), 0, h - 1);
|
||||
const ymax = clip(Math.ceil(Math.max(...y)), 0, h - 1);
|
||||
// eslint-disable-next-line new-cap
|
||||
const mask = new CV.Mat.zeros(ymax - ymin + 1, xmax - xmin + 1, CV.CV_8UC1);
|
||||
box.forEach(item => {
|
||||
item[0] = Math.max(item[0] - xmin, 0);
|
||||
item[1] = Math.max(item[1] - ymin, 0);
|
||||
});
|
||||
const npts = 4;
|
||||
const point_data = new Uint8Array(box.flat());
|
||||
const points = CV.matFromArray(npts, 1, CV.CV_32SC2, point_data);
|
||||
const pts = new CV.MatVector();
|
||||
pts.push_back(points);
|
||||
const color = new CV.Scalar(255);
|
||||
// 多个多边形填充
|
||||
CV.fillPoly(mask, pts, color, 1);
|
||||
const sliceArr = [];
|
||||
for (let i = ymin; i < ymax + 1; i++) {
|
||||
sliceArr.push(...bitmap.slice(960 * i + xmin, 960 * i + xmax + 1));
|
||||
}
|
||||
const mean = num_mean(sliceArr, mask.data);
|
||||
mask.delete();
|
||||
points.delete();
|
||||
pts.delete();
|
||||
return mean;
|
||||
}
|
||||
|
||||
function clip(data, min, max) {
|
||||
return data < min ? min : data > max ? max : data;
|
||||
}
|
||||
|
||||
function unclip(box) {
|
||||
const unclip_ratio = 1.6;
|
||||
const area = Math.abs(Polygon.polygonArea(box));
|
||||
const length = Polygon.polygonLength(box);
|
||||
const distance = area * unclip_ratio / length;
|
||||
const tmpArr = [];
|
||||
box.forEach(item => {
|
||||
const obj = {
|
||||
X: 0,
|
||||
Y: 0
|
||||
};
|
||||
obj.X = item[0];
|
||||
obj.Y = item[1];
|
||||
tmpArr.push(obj);
|
||||
});
|
||||
const offset = new clipper.ClipperOffset();
|
||||
offset.AddPath(tmpArr, clipper.JoinType.jtRound, clipper.EndType.etClosedPolygon);
|
||||
const expanded = [];
|
||||
offset.Execute(expanded, distance);
|
||||
let expandedArr = [];
|
||||
expanded[0] && expanded[0].forEach(item => {
|
||||
expandedArr.push([item.X, item.Y]);
|
||||
});
|
||||
expandedArr = [].concat(...expandedArr);
|
||||
return expandedArr;
|
||||
}
|
||||
|
||||
function num_mean(data, mask) {
|
||||
let sum = 0;
|
||||
let length = 0;
|
||||
for (let i = 0; i < data.length; i++) {
|
||||
if (mask[i]) {
|
||||
sum = plus(sum, data[i]);
|
||||
length++;
|
||||
}
|
||||
}
|
||||
return divide(sum, length);
|
||||
}
|
||||
|
||||
// eslint-disable-next-line new-cap
|
||||
const src = new CV.matFromArray(960, 960, CV.CV_8UC1, segmentation);
|
||||
const contours = new CV.MatVector();
|
||||
const hierarchy = new CV.Mat();
|
||||
// 获取轮廓
|
||||
CV.findContours(src, contours, hierarchy, CV.RETR_LIST, CV.CHAIN_APPROX_SIMPLE);
|
||||
const num_contours = Math.min(contours.size(), max_candidates);
|
||||
const boxes = [];
|
||||
const scores = [];
|
||||
const arr = [];
|
||||
for (let i = 0; i < num_contours; i++) {
|
||||
const contour = contours.get(i);
|
||||
let {
|
||||
points,
|
||||
side
|
||||
} = get_mini_boxes(contour);
|
||||
if (side < min_size) {
|
||||
continue;
|
||||
}
|
||||
const score = box_score_fast(pred, points);
|
||||
if (box_thresh > score) {
|
||||
continue;
|
||||
}
|
||||
let box = unclip(points);
|
||||
// eslint-disable-next-line new-cap
|
||||
const boxMap = new CV.matFromArray(box.length / 2, 1, CV.CV_32SC2, box);
|
||||
const resultObj = get_mini_boxes(boxMap);
|
||||
box = resultObj.points;
|
||||
side = resultObj.side;
|
||||
if (side < min_size + 2) {
|
||||
continue;
|
||||
}
|
||||
box.forEach(item => {
|
||||
item[0] = clip(Math.round(item[0]), 0, 960);
|
||||
item[1] = clip(Math.round(item[1]), 0, 960);
|
||||
});
|
||||
boxes.push(box);
|
||||
scores.push(score);
|
||||
arr.push(i);
|
||||
boxMap.delete();
|
||||
}
|
||||
src.delete();
|
||||
contours.delete();
|
||||
hierarchy.delete();
|
||||
return {
|
||||
boxes,
|
||||
scores
|
||||
};
|
||||
};
|
||||
|
||||
let detectRunner;
|
||||
|
||||
Page({
|
||||
data: {
|
||||
imgList: imgList,
|
||||
imgInfo: {},
|
||||
result: '',
|
||||
loaded: false
|
||||
},
|
||||
|
||||
onLoad() {
|
||||
enableBoundaryChecking(false);
|
||||
const me = this;
|
||||
detectRunner = new paddlejs.Runner({
|
||||
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_det_infer_js_960/model.json',
|
||||
mean: [0.485, 0.456, 0.406],
|
||||
std: [0.229, 0.224, 0.225],
|
||||
bgr: true,
|
||||
webglFeedProcess: true
|
||||
});
|
||||
detectRunner.init().then(_ => {
|
||||
me.setData({
|
||||
loaded: true
|
||||
});
|
||||
});
|
||||
},
|
||||
|
||||
selectImage(event) {
|
||||
const imgPath = this.data.imgList[event.target.dataset.index];
|
||||
this.getImageInfo(imgPath);
|
||||
},
|
||||
|
||||
getImageInfo(imgPath) {
|
||||
const me = this;
|
||||
wx.getImageInfo({
|
||||
src: imgPath,
|
||||
success: imgInfo => {
|
||||
const {
|
||||
path,
|
||||
width,
|
||||
height
|
||||
} = imgInfo;
|
||||
|
||||
const canvasPath = imgPath.includes('http') ? path : imgPath;
|
||||
const canvasId = 'myCanvas';
|
||||
const ctx = wx.createCanvasContext(canvasId);
|
||||
let sw = 960;
|
||||
let sh = 960;
|
||||
let x = 0;
|
||||
let y = 0;
|
||||
if (height / width >= 1) {
|
||||
sw = Math.round(sh * width / height);
|
||||
x = Math.floor((960 - sw) / 2);
|
||||
}
|
||||
else {
|
||||
sh = Math.round(sw * height / width);
|
||||
y = Math.floor((960 - sh) / 2);
|
||||
}
|
||||
ctx.drawImage(canvasPath, x, y, sw, sh);
|
||||
ctx.draw(false, () => {
|
||||
// API 1.9.0 获取图像数据
|
||||
wx.canvasGetImageData({
|
||||
canvasId: canvasId,
|
||||
x: 0,
|
||||
y: 0,
|
||||
width: 960,
|
||||
height: 960,
|
||||
success(res) {
|
||||
me.predict({
|
||||
data: res.data,
|
||||
width: 960,
|
||||
height: 960
|
||||
}, {
|
||||
canvasPath,
|
||||
sw,
|
||||
sh,
|
||||
x,
|
||||
y
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
});
|
||||
},
|
||||
|
||||
predict(res, img) {
|
||||
const me = this;
|
||||
detectRunner.predict(res, function (data) {
|
||||
// 获取坐标
|
||||
const points = outputBox(data);
|
||||
me.drawCanvasPoints(img, points.boxes);
|
||||
me.setData({
|
||||
result: JSON.stringify(points.boxes)
|
||||
});
|
||||
});
|
||||
},
|
||||
|
||||
drawCanvasPoints(img, points) {
|
||||
const canvasId = 'result';
|
||||
const ctx = wx.createCanvasContext(canvasId);
|
||||
ctx.drawImage(img.canvasPath, img.x, img.y, img.sw, img.sh);
|
||||
points.length && points.forEach(point => {
|
||||
// 开始一个新的绘制路径
|
||||
ctx.beginPath();
|
||||
// 设置线条颜色为蓝色
|
||||
ctx.strokeStyle = 'blue';
|
||||
// 设置路径起点坐标
|
||||
ctx.moveTo(point[0][0], point[0][1]);
|
||||
ctx.lineTo(point[1][0], point[1][1]);
|
||||
ctx.lineTo(point[2][0], point[2][1]);
|
||||
ctx.lineTo(point[3][0], point[3][1]);
|
||||
ctx.closePath();
|
||||
ctx.stroke();
|
||||
});
|
||||
ctx.draw();
|
||||
}
|
||||
});
|
@@ -0,0 +1,2 @@
|
||||
{
|
||||
}
|
@@ -0,0 +1,29 @@
|
||||
<view>
|
||||
<view class="img-view">
|
||||
<text class="title">点击图片进行预测</text>
|
||||
<scroll-view class="imgWrapper" scroll-x="true">
|
||||
<image
|
||||
class="img {{selectedIndex == index ? 'selected' : ''}}"
|
||||
wx:for="{{imgList}}"
|
||||
wx:key="index"
|
||||
src="{{item}}"
|
||||
mode="aspectFit"
|
||||
bindtap="selectImage"
|
||||
data-index="{{index}}"
|
||||
></image>
|
||||
<canvas
|
||||
canvas-id="myCanvas"
|
||||
style="width: 960px; height: 960px; position: absolute; z-index: -1"
|
||||
></canvas>
|
||||
<canvas
|
||||
canvas-id="result"
|
||||
style="width: 960px; height: 960px;"
|
||||
></canvas>
|
||||
</scroll-view>
|
||||
<text class="result" wx:if="{{result}}" style="height: 300rpx;">文本框选坐标:{{result}}</text>
|
||||
</view>
|
||||
</view>
|
||||
|
||||
<view class="mask" wx:if="{{!loaded}}">
|
||||
<text class="loading">loading…</text>
|
||||
</view>
|
@@ -0,0 +1,56 @@
|
||||
text {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.title {
|
||||
margin-top: 10px;
|
||||
font-size: 16px;
|
||||
line-height: 32px;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.imgWrapper {
|
||||
margin: 10px 10px 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.img {
|
||||
width: 960px;
|
||||
height: 960px;
|
||||
border: 1px solid #f1f1f1;
|
||||
}
|
||||
|
||||
.result {
|
||||
margin-top: 5px;
|
||||
}
|
||||
|
||||
.selected {
|
||||
border: 1px solid #999;
|
||||
}
|
||||
|
||||
.select-btn {
|
||||
margin-top: 20px;
|
||||
width: 60%;
|
||||
}
|
||||
|
||||
.mask {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
background-color: rgba(0, 0, 0, .7);
|
||||
}
|
||||
|
||||
.loading {
|
||||
color: #fff;
|
||||
font-size: 20px;
|
||||
position: absolute;
|
||||
top: 50%;
|
||||
left: 50%;
|
||||
transform: translate(-50%, -50%);
|
||||
}
|
||||
|
||||
.img-view {
|
||||
padding-bottom: 20px;
|
||||
border-bottom: 1px solid #f1f1f1;
|
||||
}
|
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"description": "项目配置文件",
|
||||
"packOptions": {
|
||||
"ignore": []
|
||||
},
|
||||
"setting": {
|
||||
"urlCheck": false,
|
||||
"es6": true,
|
||||
"enhance": true,
|
||||
"postcss": true,
|
||||
"preloadBackgroundData": false,
|
||||
"minified": true,
|
||||
"newFeature": false,
|
||||
"coverView": true,
|
||||
"nodeModules": true,
|
||||
"autoAudits": false,
|
||||
"showShadowRootInWxmlPanel": true,
|
||||
"scopeDataCheck": false,
|
||||
"uglifyFileName": false,
|
||||
"checkInvalidKey": true,
|
||||
"checkSiteMap": true,
|
||||
"uploadWithSourceMap": true,
|
||||
"compileHotReLoad": false,
|
||||
"lazyloadPlaceholderEnable": false,
|
||||
"useMultiFrameRuntime": true,
|
||||
"useApiHook": true,
|
||||
"useApiHostProcess": true,
|
||||
"babelSetting": {
|
||||
"ignore": [],
|
||||
"disablePlugins": [],
|
||||
"outputPath": ""
|
||||
},
|
||||
"enableEngineNative": false,
|
||||
"useIsolateContext": true,
|
||||
"userConfirmedBundleSwitch": false,
|
||||
"packNpmManually": false,
|
||||
"packNpmRelationList": [],
|
||||
"minifyWXSS": true,
|
||||
"disableUseStrict": false,
|
||||
"minifyWXML": true,
|
||||
"showES6CompileOption": false,
|
||||
"useCompilerPlugins": false
|
||||
},
|
||||
"compileType": "miniprogram",
|
||||
"libVersion": "2.22.1",
|
||||
"appid": "wxc43cbd2fafe0aec2",
|
||||
"projectname": "mobilenet",
|
||||
"debugOptions": {
|
||||
"hidedInDevtools": []
|
||||
},
|
||||
"scripts": {},
|
||||
"isGameTourist": false,
|
||||
"simulatorType": "wechat",
|
||||
"simulatorPluginLibVersion": {},
|
||||
"condition": {
|
||||
"search": {
|
||||
"list": []
|
||||
},
|
||||
"conversation": {
|
||||
"list": []
|
||||
},
|
||||
"game": {
|
||||
"list": []
|
||||
},
|
||||
"plugin": {
|
||||
"list": []
|
||||
},
|
||||
"gamePlugin": {
|
||||
"list": []
|
||||
},
|
||||
"miniprogram": {
|
||||
"list": []
|
||||
}
|
||||
}
|
||||
}
|
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"desc": "关于本文件的更多信息,请参考文档 https://developers.weixin.qq.com/miniprogram/dev/framework/sitemap.html",
|
||||
"rules": [{
|
||||
"action": "allow",
|
||||
"page": "*"
|
||||
}]
|
||||
}
|
172
examples/application/js/web_demo/README.md
Normal file
172
examples/application/js/web_demo/README.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# Web Demo介绍
|
||||
|
||||
- [简介](#0)
|
||||
- [1. 快速开始](#1)
|
||||
- [2. npm包调用](#2)
|
||||
- [3. 模型替换](#3)
|
||||
- [4. 自定义前后处理参数](#4)
|
||||
- [5. 其他](#5)
|
||||
|
||||
<a name="0"></a>
|
||||
## 简介
|
||||
|
||||
本项目基于[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)在浏览器中实现目标检测,人像分割,OCR,物品分类等计算机视觉任务。
|
||||
|
||||
|
||||
|demo名称|web demo组件|源码目录|npm包|
|
||||
|-|-|-|-|
|
||||
|人脸检测|[FaceDetection](./demo/src/pages/cv/detection/FaceDetection/)| [facedetect](./packages/paddlejs-models/facedetect)|[@paddle-js-models/facedetect](https://www.npmjs.com/package/@paddle-js-models/facedetect)|
|
||||
|螺丝钉检测|[ScrewDetection](./demo/src/pages/cv/detection/ScrewDetection)| [detect](./packages/paddlejs-models/detect)|[@paddle-js-models/detect](https://www.npmjs.com/package/@paddle-js-models/detect)|
|
||||
|人像分割背景替换|[HumanSeg](./demo/src/pages/cv/segmentation/HumanSeg)|[humanseg](./packages/paddlejs-models/humanseg)|[@paddle-js-models/humanseg](https://www.npmjs.com/package/@paddle-js-models/humanseg)|
|
||||
|手势识别AI猜丁壳|[GestureRecognition](./demo/src/pages/cv/recognition/GestureRecognition)|[gesture](./packages/paddlejs-models/gesture)|[@paddle-js-models/gesture](https://www.npmjs.com/package/@paddle-js-models/gesture)|
|
||||
|1000种物品识别|[ItemIdentification](./demo/src/pages/cv/recognition/ItemIdentification)|[mobilenet](./packages/paddlejs-models/mobilenet)|[@paddle-js-models/mobilenet](https://www.npmjs.com/package/@paddle-js-models/mobilenet)|
|
||||
|文本检测|[TextDetection](./demo/src/pages/cv/ocr/TextDetection)|[ocrdetection](./packages/paddlejs-models/ocrdetection)|[@paddle-js-models/ocrdet](https://www.npmjs.com/package/@paddle-js-models/ocrdet)|
|
||||
|文本识别|[TextRecognition](./demo/src/pages/cv/ocr/TextRecognition)|[ocr](./packages/paddlejs-models/ocr)|[@paddle-js-models/ocr](https://www.npmjs.com/package/@paddle-js-models/ocr)|
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. 快速开始
|
||||
|
||||
本节介绍如何在浏览器中直接运行官方demo。
|
||||
|
||||
**1. 安装Node.js**
|
||||
|
||||
从`Node.js`官网https://nodejs.org/en/download/ 下载适合自己平台的`Node.js`安装包并安装。
|
||||
|
||||
**2. 安装demo依赖并启动**
|
||||
在`./web_demo/demo`目录下执行如下指令:
|
||||
|
||||
```
|
||||
# 安装依赖
|
||||
npm install
|
||||
# 启动demo
|
||||
npm run dev
|
||||
```
|
||||
|
||||
在浏览器中打开网址 `http://localhost:5173/main/index.html` 即可快速体验在浏览器中运行计算机视觉任务。
|
||||
|
||||

|
||||
|
||||
<a name="2"></a>
|
||||
## 2. npm包调用
|
||||
|
||||
本节介绍npm包的使用方式,每个demo均提供简单易用的接口,用户只需初始化上传图片即可获得结果,使用步骤如下:
|
||||
1. 调用模块
|
||||
2. 初始化模型
|
||||
3. 传入输入,执行预测
|
||||
|
||||
以 OCR 为例,在前端项目中,`@paddle-js-models/ocr`包的使用方式如下:
|
||||
|
||||
```
|
||||
// 1. 调用ocr模块
|
||||
import * as ocr from '@paddle-js-models/ocr';
|
||||
|
||||
// 2. 初始化ocr模型
|
||||
await ocr.init();
|
||||
|
||||
// 3. 传入HTMLImageElement类型的图像作为输入并获得结果
|
||||
const res = await ocr.recognize(img);
|
||||
|
||||
// 打印OCR模型得到的文本坐标以及文本内容
|
||||
console.log(res.text);
|
||||
console.log(res.points);
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
## 3. 模型替换
|
||||
|
||||
由于前端环境和计算资源限制,在前端部署深度学习模型时,我们对模型的性能有着更严格的要求,简单来说,模型需要足够轻量化。理论上模型的输入shape越小、模型大小越小,则对应的模型的flops越小,在前端运行也能更流畅。经验总结,使用`Paddle.js`部署的模型存储尽量不超过*5M*,实际情况根据硬件和计算资源情况决定。
|
||||
|
||||
在实际应用中,常常根据垂类的场景定制化模型,官方的demo支持修改传入参数替换模型。
|
||||
|
||||
以OCR demo为例,[ocr.init()函数](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo/packages/paddlejs-models/ocr/src/index.ts#L52)中,包含默认初始化的模型链接,如果要替换模型参考下述步骤。
|
||||
|
||||
步骤1:将模型转成js格式:
|
||||
```
|
||||
# 安装paddlejsconverter
|
||||
pip3 install paddlejsconverter
|
||||
# 转换模型格式,输入模型为inference模型
|
||||
paddlejsconverter --modelPath=./inference.pdmodel --paramPath=./inference.pdiparams --outputDir=./ --useGPUOpt=True
|
||||
# 注意:useGPUOpt 选项默认不开启,如果模型用在 gpu backend(webgl/webgpu),则开启 useGPUOpt,如果模型运行在(wasm/plain js)则不要开启。
|
||||
```
|
||||
|
||||
导出成功后,本地目录下会出现 `model.json chunk_1.dat`等文件,分别是对应js模型的网络结构、模型参数二进制文件。
|
||||
|
||||
步骤2:将导出的js模型上传到支持跨域访问的服务器,服务器的CORS配置参考下图:
|
||||

|
||||
|
||||
|
||||
步骤3:修改代码替换默认的模型。以OCR demo为例,修改OCR web demo中[模型初始化代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo/demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L64),即
|
||||
|
||||
```
|
||||
await ocr.init();
|
||||
修改为:
|
||||
await ocr.init({modelPath: "https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json"}); # 第一个参数传入新的文本检测字典类型参数
|
||||
```
|
||||
|
||||
重新在demo目录下执行下述命令,即可体验新的模型效果。
|
||||
```
|
||||
npm run dev
|
||||
```
|
||||
|
||||
<a name="4"></a>
|
||||
## 4. 自定义前后处理参数
|
||||
|
||||
**自定义前处理参数**
|
||||
|
||||
在不同计算机视觉任务中,不同的模型可能有不同的预处理参数,比如mean,std,keep_ratio等参数,替换模型后也需要对预处理参数进行修改。paddle.js发布的npm包中提供了自定义预处理参数的简单方案。只需要在调用模型初始化函数时,传入自定义的参数即可。
|
||||
|
||||
```
|
||||
# 默认参数初始化
|
||||
await model.init();
|
||||
|
||||
自定义参数初始化
|
||||
const Config = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5], keepratio: false};
|
||||
await model.init(Config);
|
||||
```
|
||||
|
||||
以OCR文本检测demo为例,修改模型前处理的mean和std参数,只需要在模型初始化时传入自定义的mean和std参数。
|
||||
```
|
||||
await ocr.init();
|
||||
修改为:
|
||||
const detConfig = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5]};
|
||||
await ocr.init(detConfig); # 第一个参数传入新的文本检测模型链接
|
||||
```
|
||||
|
||||
**自定义后处理参数**
|
||||
|
||||
同理,paddle.js发布的npm包也提供了后处理参数的自定义方案。
|
||||
|
||||
```
|
||||
# 默认参数运行
|
||||
await model.predict();
|
||||
|
||||
# 自定义后处理参数
|
||||
const postConfig = {thresh: 0.5};
|
||||
await model.predict(Config);
|
||||
```
|
||||
|
||||
以OCR文本检测 demo为例,修改文本检测后处理的参数实现扩大文本检测框的效果,修改OCR web demo中执行[模型预测代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/web_demo/demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L99),即:
|
||||
|
||||
```
|
||||
const res = await ocr.recognize(img, { canvas: canvas.value });
|
||||
修改为:
|
||||
// 定义超参数,将unclip_ratio参数从1.5 增大为3.5
|
||||
const detConfig = {shape: 960, thresh: 0.3, box_thresh: 0.6, unclip_ratio:3.5};
|
||||
const res = await ocr.recognize(img, { canvas: canvas.value }, detConfig);
|
||||
```
|
||||
|
||||
注:不同的任务有不同的后处理参数,详细参数参考npm包中的API。
|
||||
|
||||
<a name="5"></a>
|
||||
## 5. 其他
|
||||
|
||||
`Paddle.js`转换后的模型不仅支持浏览器中使用,也可以在百度小程序和微信小程序环境下运行。
|
||||
|
||||
|名称|目录|
|
||||
|-|-|
|
||||
|OCR文本检测| [ocrdetecXcx](../mini_program/ocrdetectXcx/) |
|
||||
|OCR文本识别| [ocrXcx](../mini_program/ocrXcx/) |
|
||||
|目标检测| coming soon |
|
||||
|图像分割| coming soon |
|
||||
|物品分类| coming soon |
|
73
examples/application/js/web_demo/converter/DEVELOPMENT.md
Normal file
73
examples/application/js/web_demo/converter/DEVELOPMENT.md
Normal file
@@ -0,0 +1,73 @@
|
||||
[中文版](./DEVELOPMENT_cn.md)
|
||||
# paddlejs-converter
|
||||
|
||||
paddlejs-converter is a model transformation tool for Paddle.js. Its role is to convert PaddlePaddle models (also known as fluid models) into a browser-friendly format that Paddle.js can use to load and predict usage in browsers as well as other environments. In addition, paddlejs-converter provides powerful model optimization capabilities to help developers optimize the model structure and improve runtime performance.
|
||||
|
||||
## 1. Tutorial
|
||||
|
||||
### 1.1. Environment Construction
|
||||
#### Python Version
|
||||
Confirm whether the python environment and version of the running platform meet the requirements. If Python 3 is used, you may need to change the `python` in subsequent commands to `python3`:
|
||||
- Python3: 3.5.1+ / 3.6 / 3.7
|
||||
- Python2: 2.7.15+
|
||||
|
||||
#### Install Virtual Environment
|
||||
*Since the development environment may have multiple versions of Python installed, there may be different versions of dependent packages. In order to avoid conflicts, it is strongly recommended to use Python virtual environment to execute the commands required by the conversion tool to avoid various problems. If you are not using a virtual environment or if you have a virtual environment installed, you can skip this step.*
|
||||
|
||||
Take Anaconda as an example:
|
||||
Go to [Anaconda](https://www.anaconda.com/) main page,Select the corresponding platform and python version of anaconda and install it according to the official prompts;
|
||||
|
||||
After installation, execute the following command on the command line to create a python virtual environment:
|
||||
``` bash
|
||||
conda create --name <your_env_name>
|
||||
```
|
||||
|
||||
Execute the following command to switch to the virtual environment
|
||||
``` bash
|
||||
# Linux or macOS
|
||||
source activate <your_env_name>
|
||||
|
||||
# Windows
|
||||
activate <your_env_name>
|
||||
```
|
||||
|
||||
#### Installation Dependency
|
||||
- If you don't need to optimize model, execute the command:
|
||||
``` bash
|
||||
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
|
||||
```
|
||||
- Otherwise,execute the command:
|
||||
``` bash
|
||||
python -m pip install paddlepaddle paddlelite==2.6.0 -i https://mirror.baidu.com/pypi/simple
|
||||
```
|
||||
|
||||
### 1.2. Get Start
|
||||
- If the weight file of fluid model to be converted is merged format which means one model corresponds to one weight file, then execute:
|
||||
``` bash
|
||||
python convertToPaddleJSModel.py --modelPath=<fluid_model_file_path> --paramPath=<fluid_param_file_path> --outputDir=<paddlejs_model_directory>
|
||||
```
|
||||
- Otherwise,execute:
|
||||
``` bash
|
||||
# Note that in this way, you need to ensure that the model file name '__ model__ ' in the inputDir
|
||||
python convertToPaddleJSModel.py --inputDir=<fluid_model_directory> --outputDir=<paddlejs_model_directory>
|
||||
````
|
||||
The model converter generates the following two types of files for Paddle.js:
|
||||
|
||||
- model.json (Contains the model structure and parameter list)
|
||||
- chunk_\*.dat (The collection of binary weight files)
|
||||
|
||||
## 2. Detailed Documentation
|
||||
|
||||
Parameter | description
|
||||
:-: | :-:
|
||||
--inputDir | The fluid model directory, If and only if weight files are not merged format, `modelPath` and `paramPath` below will be ignored,and the model file name should be `__model__`.
|
||||
--modelPath | The model file path, used when the weight file is merged.
|
||||
--paramPath | The weight file path,used when the weight file is merged.
|
||||
--outputDir | `Necessary`, the output model directory generated after converting.
|
||||
--disableOptimize | Whether to disable optimize model, `1`is to disable, `0`is use optimize(need to install PaddleLite), default 0.
|
||||
--logModelInfo | Whether to print model structure information, `0` means not to print, `1` means to print, default 0.
|
||||
--sliceDataSize | Shard size (in KB) of each weight file. Default size is 4096.
|
||||
--useGPUOpt | Whether to use gpu opt, default is False.
|
||||
|
||||
## 3. Other information
|
||||
If the model to be converted is in `tensorflow / Cafe / onnx` format, there is [X2Paddle](https://github.com/PaddlePaddle/X2Paddle) tool in PaddlePaddle program for converting other models with different formats to fluid model, and then you can use paddlejs-converter to get a Paddle.js model.
|
73
examples/application/js/web_demo/converter/DEVELOPMENT_cn.md
Normal file
73
examples/application/js/web_demo/converter/DEVELOPMENT_cn.md
Normal file
@@ -0,0 +1,73 @@
|
||||
[English](./README.md)
|
||||
# paddlejs-converter
|
||||
|
||||
paddlejs-converter 是适用于 Paddle.js 的模型转换工具,其作用是将 PaddlePaddle 模型(或称为 fluid 模型)转化为浏览器友好的格式,以供Paddle.js在浏览器等环境中加载预测使用。此外,paddlejs-converter 还提供了强大的模型优化能力,帮助开发者对模型结构进行优化,提高运行时性能。
|
||||
|
||||
## 1. 使用教程
|
||||
|
||||
### 1.1. 环境搭建
|
||||
#### Python 版本确认
|
||||
确认运行平台的 Python 环境与版本是否满足要求,若使用 Python3 ,则可能需要将后续命令中的 `python` 换成 `python3`:
|
||||
- Python3: 3.5.1+ / 3.6 / 3.7
|
||||
- Python2: 2.7.15+
|
||||
|
||||
#### 安装虚拟环境
|
||||
*由于开发环境可能安装了多个版本的 Python,相关依赖包可能存在不同的版本,为避免产生冲突,**强烈建议**使用 Python 虚拟环境执行转换工具所需的各项命令,以免产生各种问题。若不使用虚拟环境或已安装虚拟环境,可跳过该步骤。*
|
||||
|
||||
以 Anaconda 为例:
|
||||
前往 [Anaconda](https://www.anaconda.com/) 主页,选择对应平台、Python 版本的 Anaconda 按照官方提示,进行安装;
|
||||
|
||||
安装完毕后,在命令行执行以下命令,创建Python 虚拟环境:
|
||||
``` bash
|
||||
conda create --name <your_env_name>
|
||||
```
|
||||
|
||||
执行以下命令,切换至虚拟环境
|
||||
``` bash
|
||||
# Linux 或 macOS下请执行
|
||||
source activate <your_env_name>
|
||||
|
||||
# Windows 下请执行
|
||||
activate <your_env_name>
|
||||
```
|
||||
|
||||
#### 安装依赖
|
||||
- 如果`不需要`使用优化模型的能力,执行命令:
|
||||
``` bash
|
||||
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
|
||||
```
|
||||
- 如果`需要`使用优化模型的能力,执行命令:
|
||||
``` bash
|
||||
python -m pip install paddlepaddle paddlelite==2.6.0 -i https://mirror.baidu.com/pypi/simple
|
||||
```
|
||||
|
||||
### 1.2. 快速上手
|
||||
- 如果待转换的 fluid 模型为`合并参数文件`,即一个模型对应一个参数文件:
|
||||
``` bash
|
||||
python convertToPaddleJSModel.py --modelPath=<fluid_model_file_path> --paramPath=<fluid_param_file_path> --outputDir=<paddlejs_model_directory>
|
||||
```
|
||||
- 如果待转换的 fluid 模型为`分片参数文件`,即一个模型文件对应多个参数文件:
|
||||
``` bash
|
||||
# 注意,使用这种方式调用转换器,需要保证 inputDir 中,模型文件名为'__model__'
|
||||
python convertToPaddleJSModel.py --inputDir=<fluid_model_directory> --outputDir=<paddlejs_model_directory>
|
||||
````
|
||||
模型转换器将生成以下两种类型的文件以供 Paddle.js 使用:
|
||||
|
||||
- model.json (模型结构与参数清单)
|
||||
- chunk_\*.dat (二进制参数文件集合)
|
||||
|
||||
## 2. 详细文档
|
||||
参数 | 描述
|
||||
:-: | :-:
|
||||
--inputDir | fluid 模型所在目录,当且仅当使用分片参数文件时使用该参数,将忽略 `modelPath` 和 `paramPath` 参数,且模型文件名必须为`__model__`
|
||||
--modelPath | fluid 模型文件所在路径,使用合并参数文件时使用该参数
|
||||
--paramPath | fluid 参数文件所在路径,使用合并参数文件时使用该参数
|
||||
--outputDir | `必要参数`, Paddle.js 模型输出路径
|
||||
--disableOptimize | 是否关闭模型优化, `1` 为关闭优化,`0` 为开启优化(需安装 PaddleLite ),默认执行优化
|
||||
--logModelInfo | 是否打印模型结构信息, `0` 为不打印, `1` 为打印,默认不打印
|
||||
--sliceDataSize | 分片输出 Paddle.js 参数文件时,每片文件的大小,单位:KB,默认 4096
|
||||
--useGPUOpt | 是否开启模型 GPU 优化,默认不开启(当模型准备运行在 webgl/webgpu 计算方案时,可以设置为 True 开启,在 wasm/plainjs 方案,则不用开启)
|
||||
|
||||
## 3. 其他信息
|
||||
若需要转换的模型为 `TensorFlow/Caffe/ONNX` 格式,可使用 PaddlePaddle 项目下的 `X2Paddle`工具,将其他格式的模型转为 fluid 模型后,再使用本工具转化为 Paddle.js 模型。
|
||||
详细请参考 [X2Paddle 项目](https://github.com/PaddlePaddle/X2Paddle)
|
29
examples/application/js/web_demo/converter/README.md
Normal file
29
examples/application/js/web_demo/converter/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# PaddleJsConverter
|
||||
|
||||
## Installation
|
||||
|
||||
System Requirements:
|
||||
|
||||
* paddlepaddle >= 2.0.0
|
||||
* paddlejslite >= 0.0.2
|
||||
* Python3: 3.5.1+ / 3.6 / 3.7
|
||||
* Python2: 2.7.15+
|
||||
|
||||
#### Install PaddleJsConverter
|
||||
|
||||
<img src="https://img.shields.io/pypi/v/paddlejsconverter" alt="version">
|
||||
|
||||
```shell
|
||||
pip install paddlejsconverter
|
||||
|
||||
# or
|
||||
pip3 install paddlejsconverter
|
||||
```
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
```shell
|
||||
paddlejsconverter --modelPath=user_model_path --paramPath=user_model_params_path --outputDir=model_saved_path --useGPUOpt=True
|
||||
```
|
||||
注意:useGPUOpt 选项默认不开启,如果模型用在 gpu backend(webgl/webgpu),则开启 useGPUOpt,如果模型运行在(wasm/plain js)则不要开启。
|
82
examples/application/js/web_demo/converter/RNN.md
Normal file
82
examples/application/js/web_demo/converter/RNN.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# RNN算子计算过程
|
||||
|
||||
## 一、RNN理解
|
||||
|
||||
**RNN** 是循环神经网络,由输入层、隐藏层和输出层组成,擅长对序列数据进行处理。
|
||||
|
||||

|
||||
paddle官网文档:https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/RNN_cn.html#rnn
|
||||
|
||||
paddle源码实现:https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rnn_op.h#L812
|
||||
|
||||
##二、RNN计算方式
|
||||
|
||||
t 时刻,输入层为  ,隐藏层为  ,输出层为  。由上图可知, 的值不仅仅取决于  ,还取决于  。计算公式如下:
|
||||
|
||||

|
||||
|
||||
## 三、pdjs中RNN算子实现
|
||||
|
||||
因为 RNN 有梯度消失问题,不能获取更多上下文信息,所以 CRNN 中使用的是 **LSTM(Long Short Term Memory)**,LSTM 是一种特殊的 RNN,能够保存长期的依赖关系。
|
||||
|
||||
基于图像的序列,两个方向的上下文是相互有用且互补的。由于 LSTM 是单向的,所以将两个 LSTM,一个向前和一个向后组合到一个**双向 LSTM** 中。此外,可以堆叠多层双向 LSTM。ch_PP-OCRv2_rec_infer 识别模型就是使用的双层双向 LSTM 结构。计算过程如下图所示:
|
||||
|
||||
#### 以ch_ppocr_mobile_v2.0_rec_infer 模型 rnn算子为例:
|
||||
```javascript
|
||||
{
|
||||
Attr: {
|
||||
mode: 'LSTM'
|
||||
// 是否双向,为true则正向反向都需要遍历
|
||||
is_bidirec: true
|
||||
// 隐藏层层数,代表循环次数
|
||||
num_layers: 2
|
||||
}
|
||||
|
||||
Input: [
|
||||
transpose_1.tmp_0[25, 1, 288]
|
||||
]
|
||||
|
||||
PreState: [
|
||||
fill_constant_batch_size_like_0.tmp_0[4, 1, 48],
|
||||
fill_constant_batch_size_like_1.tmp_0[4, 1, 48]
|
||||
]
|
||||
|
||||
WeightList: [
|
||||
lstm_cell_0.w_0[192, 288], lstm_cell_0.w_1[192, 48],
|
||||
lstm_cell_1.w_0[192, 288], lstm_cell_1.w_1[192, 48],
|
||||
lstm_cell_2.w_0[192, 96], lstm_cell_2.w_1[192, 48],
|
||||
lstm_cell_3.w_0[192, 96], lstm_cell_3.w_1[192, 48],
|
||||
lstm_cell_0.b_0[192], lstm_cell_0.b_1[192],
|
||||
lstm_cell_1.b_0[192], lstm_cell_1.b_1[192],
|
||||
lstm_cell_2.b_0[192], lstm_cell_2.b_1[192],
|
||||
lstm_cell_3.b_0[192], lstm_cell_3.b_1[192]
|
||||
]
|
||||
|
||||
Output: [
|
||||
lstm_0.tmp_0[25, 1, 96]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### 整体计算过程
|
||||

|
||||
#### rnn 计算中新增op:
|
||||
1)rnn_origin
|
||||
|
||||
计算公式: blas.MatMul(Input, WeightList_ih, blas_ih) + blas.MatMul(PreState, WeightList_hh, blas_hh)
|
||||
|
||||
2)rnn_matmul
|
||||
|
||||
计算公式:rnn_matmul = rnn_origin + Matmul( $ S_{t-1} $, WeightList_hh)
|
||||
|
||||
3)rnn_cell
|
||||
|
||||
计算方式:将rnn_matmul op输出结果分割成4份,每份执行不同激活函数计算,最后输出lstm_x_y.tmp_c[1, 1, 48]。x∈[0, 3],y∈[0, 24]。
|
||||
详见算子实现:[rnn_cell](../paddlejs-backend-webgl/src/ops/shader/rnn/rnn_cell.ts)
|
||||
)
|
||||
|
||||
4)rnn_hidden
|
||||
计算方式:将rnn_matmul op输出结果分割成4份,每份执行不同激活函数计算,最后输出lstm_x_y.tmp_h[1, 1, 48]。x∈[0, 3],y∈[0, 24]。
|
||||
详见算子实现:[rnn_hidden](../paddlejs-backend-webgl/src/ops/shader/rnn/rnn_hidden.ts)
|
||||
|
||||
|
558
examples/application/js/web_demo/converter/convertModel.py
Normal file
558
examples/application/js/web_demo/converter/convertModel.py
Normal file
@@ -0,0 +1,558 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
import json
|
||||
import collections
|
||||
import math
|
||||
import sys
|
||||
import os
|
||||
import struct
|
||||
import argparse
|
||||
import shutil
|
||||
import stat
|
||||
import traceback
|
||||
import numpy as np
|
||||
import paddle.fluid as fluid
|
||||
import paddle as paddle
|
||||
import copy
|
||||
from functools import reduce
|
||||
import rnn
|
||||
from pruningModel import pruningNoSenseTensor
|
||||
from fuseOps import opListFuse
|
||||
|
||||
|
||||
# 输入模型所在目录
|
||||
modelDir = None
|
||||
# 输入模型名
|
||||
modelName = None
|
||||
# 输入参数名,当且仅当所有模型参数被保存在一个单独的二进制文件中,它才需要被指定,若为分片模型,请设置为None
|
||||
paramsName = None
|
||||
# 是否打印模型信息
|
||||
enableLogModelInfo = False
|
||||
# 输出模型目录
|
||||
outputDir = None
|
||||
# 分片文件大小,单位:KB
|
||||
sliceDataSize = 4 * 1024
|
||||
# paddlepaddle运行程序实例
|
||||
program = None
|
||||
# 存放模型结构
|
||||
modelInfo = {"vars": {}, "ops": [], "chunkNum": 0, "dataLayout": "nchw", "feedShape": None}
|
||||
# 存放参数数值(未排序)
|
||||
paramValuesDict = {}
|
||||
|
||||
# 有一些后置算子适合在cpu中运行,所以单独统计
|
||||
postOps = []
|
||||
# 在转换过程中新生成的、需要添加到vars中的variable
|
||||
appendedVarList = []
|
||||
# rnn op索引列表
|
||||
rnnList = []
|
||||
|
||||
# 转换模型中需要过滤掉的参数
|
||||
needFilterAttributes = ['op_callstack', 'col', 'op_role', 'op_namescope', 'op_role_var',
|
||||
'data_format', 'is_test', 'use_mkldnn', 'use_cudnn', 'use_quantizer', 'workspace_size_MB',
|
||||
'mkldnn_data_type', 'op_device', '__@kernel_type_attr@__']
|
||||
|
||||
|
||||
class ObjDict(dict):
|
||||
"""
|
||||
Makes a dictionary behave like an object,with attribute-style access.
|
||||
"""
|
||||
def __getattr__(self,name):
|
||||
try:
|
||||
return self[name]
|
||||
except:
|
||||
raise AttributeError(name)
|
||||
def __setattr__(self,name,value):
|
||||
self[name]=value
|
||||
|
||||
def validateShape(shape, name):
|
||||
"""检验shape长度,超过4则截断"""
|
||||
if len(shape) > 4:
|
||||
newShape = shape[-4:]
|
||||
print('\033[31m ' + name + ' tensor shape length > 4, 处理为丢弃头部shape \033[0m')
|
||||
return newShape
|
||||
return shape
|
||||
|
||||
def splitLargeNum(x):
|
||||
"""将x拆分成两个因数相乘"""
|
||||
# 获取最小值
|
||||
num = math.floor(math.sqrt(x))
|
||||
while (num):
|
||||
if x % num == 0:
|
||||
return [num, int(x / num)]
|
||||
num -= 1
|
||||
|
||||
return [1, x]
|
||||
|
||||
def logModel(info):
|
||||
""" 打印信息 """
|
||||
if enableLogModelInfo:
|
||||
print(info)
|
||||
|
||||
def sortDict(oldDict, reverse=False):
|
||||
""" 对字典进行排序,返回有序字典,默认升序 """
|
||||
# 获得排序后的key list
|
||||
keys = sorted(oldDict.keys(), reverse=reverse)
|
||||
orderDict = collections.OrderedDict()
|
||||
# 遍历 key 列表
|
||||
for key in keys:
|
||||
orderDict[key] = oldDict[key]
|
||||
return orderDict
|
||||
|
||||
def dumpModelToJsonFile(outputDir):
|
||||
""" 导出模型数据到json文件 """
|
||||
print("Dumping model structure to json file...")
|
||||
if not os.path.exists(outputDir):
|
||||
os.makedirs(outputDir)
|
||||
outputModelPath = os.path.join(outputDir, "model.json")
|
||||
with open(outputModelPath, 'w') as outputFile:
|
||||
json.dump(modelInfo, outputFile, indent=4, separators=(", ", ": "), sort_keys=True)
|
||||
print("Dumping model structure to json file successfully")
|
||||
|
||||
def sliceDataToBinaryFile(paramValueList, outputDir):
|
||||
""" 将参数数据分片输出到文件,默认分片策略为按4M分片 """
|
||||
totalParamValuesCount = len(paramValueList)
|
||||
countPerSlice = int(sliceDataSize * 1024 / 4)
|
||||
|
||||
if not os.path.exists(outputDir):
|
||||
os.makedirs(outputDir)
|
||||
currentChunkIndex = 0
|
||||
currentParamDataIndex = 0
|
||||
|
||||
while currentParamDataIndex < totalParamValuesCount - 1:
|
||||
remainCount = totalParamValuesCount - currentParamDataIndex
|
||||
if remainCount < countPerSlice:
|
||||
countPerSlice = remainCount
|
||||
chunkPath = os.path.join(outputDir, 'chunk_%s.dat' % (currentChunkIndex + 1))
|
||||
file = open(chunkPath, 'wb')
|
||||
for i in paramValueList[currentParamDataIndex : currentParamDataIndex + countPerSlice]:
|
||||
byte = struct.pack('f', float(i))
|
||||
file.write(byte)
|
||||
file.close()
|
||||
currentParamDataIndex = currentParamDataIndex + countPerSlice
|
||||
currentChunkIndex = currentChunkIndex + 1
|
||||
print("Output No." + str(currentChunkIndex)+ " binary file, remain " + str(totalParamValuesCount - currentParamDataIndex) + " param values.")
|
||||
print("Slicing data to binary files successfully. (" + str(currentChunkIndex)+ " output files and " + str(currentParamDataIndex) + " param values)")
|
||||
|
||||
def reorderParamsValue():
|
||||
""" 对参数文件中的数值,按照variable.name字母序排序,返回排序后组合完成的value list """
|
||||
paramValuesOrderDict = sortDict(paramValuesDict)
|
||||
paramValues = []
|
||||
for value in paramValuesOrderDict.values():
|
||||
paramValues += value
|
||||
return paramValues
|
||||
|
||||
|
||||
def mapToPaddleJSTypeName(fluidOPName):
|
||||
""" 处理fluid的OP type与PaddleJS的OP type不对应情况 """
|
||||
if fluidOPName == "batch_norm":
|
||||
return "batchnorm"
|
||||
return fluidOPName
|
||||
|
||||
def excludeNegativeShape(shape):
|
||||
varShape = list(shape)
|
||||
varShapeExcludeNegativeOne = []
|
||||
for s in varShape:
|
||||
# 模型中 ?会自动转为 -1,需要单独处理成 1
|
||||
if s == -1:
|
||||
s = 1
|
||||
varShapeExcludeNegativeOne.append(s)
|
||||
return varShapeExcludeNegativeOne
|
||||
|
||||
def organizeModelVariableInfo(result):
|
||||
""" 组织参数信息 """
|
||||
print("Organizing model variables info...")
|
||||
index = 0
|
||||
# 存放var信息(未排序)
|
||||
varInfoDict = {}
|
||||
# 获取program中所有的var,遍历并获取所有未排序的var信息和参数数值
|
||||
vars = list(program.list_vars())
|
||||
for v in vars:
|
||||
# 跳过feed和fetch
|
||||
if "feed" == v.name:
|
||||
continue
|
||||
if "fetch" == v.name:
|
||||
continue
|
||||
|
||||
varShape = excludeNegativeShape(v.shape)
|
||||
# FIXME:end
|
||||
|
||||
# 存放variable信息,在dump成json时排序
|
||||
varInfo = {}
|
||||
varInfo["shape"] = varShape
|
||||
# 数据是否是持久化数据,如tensor为持久化数据,op的output不是持久化数据
|
||||
# 只输出持久化数据,paddlejs中也仅读取持久化数据
|
||||
varInfo["persistable"] = v.persistable
|
||||
varInfoDict[v.name] = varInfo
|
||||
|
||||
logModel("[Var index:" + str(index) + " name:" + v.name + "]")
|
||||
jsonDumpsIndentStr = json.dumps(varInfo, indent=2)
|
||||
logModel(jsonDumpsIndentStr)
|
||||
logModel("")
|
||||
index += 1
|
||||
|
||||
# persistable数据存入paramValuesDict,等待排序
|
||||
if v.persistable:
|
||||
tensor = np.array(fluid.global_scope().find_var(v.name).get_tensor())
|
||||
data = tensor.flatten().tolist()
|
||||
paramValuesDict[v.name] = data
|
||||
|
||||
# shape推断校正
|
||||
feed_target_names = result[1]
|
||||
fetch_targets = result[2]
|
||||
# 获取输入shape
|
||||
feedData = {}
|
||||
feeded_vars = [program.global_block().vars[varname] for varname in feed_target_names]
|
||||
for feedItem in feeded_vars:
|
||||
curShape = feedItem.shape
|
||||
feedName = feedItem.name
|
||||
feedData[feedName] = np.full(excludeNegativeShape(curShape), 1.0, "float32")
|
||||
|
||||
for v in program.list_vars():
|
||||
if not v.persistable:
|
||||
v.persistable = True
|
||||
exe.run(program, feed=feedData, fetch_list=fetch_targets, return_numpy=False)
|
||||
|
||||
for varKey in varInfoDict:
|
||||
var = fluid.global_scope().find_var(varKey)
|
||||
varData = np.array(var.get_tensor())
|
||||
varShape = list(varData.shape)
|
||||
varInfoDict[varKey]['shape'] = validateShape(varShape, varKey)
|
||||
|
||||
# vars追加
|
||||
vars = modelInfo['vars']
|
||||
for appendedVar in appendedVarList:
|
||||
appendedName = appendedVar['name']
|
||||
newName = appendedVar['new']
|
||||
for curVarKey in varInfoDict:
|
||||
if curVarKey == appendedName:
|
||||
newVar = copy.deepcopy(varInfoDict[curVarKey])
|
||||
varInfoDict[newName] = newVar
|
||||
break
|
||||
# 对var信息dict,按照key(var名)进行字母顺序排序
|
||||
varInfoOrderDict = sortDict(varInfoDict)
|
||||
# 将var信息按照顺序,添加到model info的vars中
|
||||
for key, value in varInfoOrderDict.items():
|
||||
value["name"] = key
|
||||
modelInfo["vars"][key] = value
|
||||
print("Organizing model variables info successfully.")
|
||||
|
||||
def organizeModelOpInfo():
|
||||
""" 组织模型OP结构信息 """
|
||||
print("Organizing model operators info...")
|
||||
ops = program.current_block().ops
|
||||
feedOutputName = None
|
||||
index = 0
|
||||
for op in ops:
|
||||
opInfo = {}
|
||||
|
||||
# 获取OP type,需要映射到PaddleJS的名字
|
||||
opInfo["type"] = mapToPaddleJSTypeName(op.type)
|
||||
|
||||
opInputs = op.input_names
|
||||
opOutputs = op.output_names
|
||||
|
||||
# 获取OP input
|
||||
inputs = {}
|
||||
for name in opInputs:
|
||||
value = op.input(name)
|
||||
if len(value) <= 0:
|
||||
continue
|
||||
if value[0] == feedOutputName:
|
||||
# FIXME:workaround,PaddleJSfeed 输入必须是image,且为单输入,这里修改feed后面的OP的input为image,建立前后关联
|
||||
inputs[name] = ["image"]
|
||||
else:
|
||||
inputs[name] = value
|
||||
opInfo["inputs"] = inputs
|
||||
|
||||
# 获取OP output
|
||||
outputs = {}
|
||||
# 将outputs转换为数组
|
||||
if op.type == 'density_prior_box' or op.type == 'prior_box' or op.type == 'box_coder':
|
||||
outputs['Out'] = []
|
||||
for name in opOutputs:
|
||||
value = op.output(name)
|
||||
if len(value) <= 0:
|
||||
continue
|
||||
outputs['Out'].append(value[0])
|
||||
else:
|
||||
for name in opOutputs:
|
||||
value = op.output(name)
|
||||
if len(value) <= 0:
|
||||
continue
|
||||
if op.type == "feed":
|
||||
# FIXME:workaround,PaddleJSfeed 输入必须是image,且为单输入,这里保存原始的输出名,以便映射
|
||||
feedOutputName = value[0]
|
||||
outputs[name] = ["image"]
|
||||
else:
|
||||
outputs[name] = value
|
||||
|
||||
opInfo["outputs"] = outputs
|
||||
|
||||
# 收敛outputs[name]
|
||||
if "Output" in opInfo["outputs"]:
|
||||
opInfo["outputs"]["Out"] = opInfo["outputs"]["Output"]
|
||||
del opInfo["outputs"]["Output"]
|
||||
|
||||
elif "Y" in opInfo["outputs"]:
|
||||
opInfo["outputs"]["Out"] = opInfo["outputs"]["Y"]
|
||||
del opInfo["outputs"]["Y"]
|
||||
|
||||
if "Out" not in opInfo["outputs"]:
|
||||
print("\033[31moutputs[name] not exist Out.\033[0m")
|
||||
sys.exit(1)
|
||||
|
||||
# 有的模型如人脸关键点,会出现两个算子合并的情况,如lmk_demo,elementwise_add后接了relu算子,relu的输入输出相等,兼容一下
|
||||
# inputs与outputs只有一个,名称相等,则,输入加后缀,改上一层算子。
|
||||
if 'X' in inputs and 'Out' in outputs:
|
||||
curInputs = inputs['X']
|
||||
curOutputs = outputs['Out']
|
||||
if len(curInputs) == 1 and len(curOutputs) == 1 and curInputs[0] == curOutputs[0] and index > 1:
|
||||
originName = curInputs[0]
|
||||
changedName = inputs['X'][0] = curInputs[0] = originName + '_changed'
|
||||
opInfo["inputs"]['X'] = curInputs
|
||||
# 获取上一层算子
|
||||
prevOpOutputs = modelInfo["ops"][index - 1]['outputs']
|
||||
for name in prevOpOutputs:
|
||||
values = prevOpOutputs[name]
|
||||
for i, curName in enumerate(values):
|
||||
if (curName == originName):
|
||||
modelInfo["ops"][index - 1]['outputs'][name][i] = changedName
|
||||
appendedVarList.append({'name': originName, 'new': changedName})
|
||||
|
||||
# 获取OP attribute
|
||||
attrs = {}
|
||||
for name in op.attr_names:
|
||||
# 过滤不需要的参数
|
||||
if name in needFilterAttributes:
|
||||
continue
|
||||
value = op.attr(name)
|
||||
attrs[name] = value
|
||||
opInfo["attrs"] = attrs
|
||||
|
||||
if (op.type == 'rnn'):
|
||||
global rnnList
|
||||
rnnList.append(index)
|
||||
|
||||
# multiclass_nms 单独处理
|
||||
if (op.type.startswith('multiclass_nms')):
|
||||
opInfo["type"] = 'multiclass_nms'
|
||||
postOps.append(opInfo)
|
||||
else:
|
||||
# 存入modelInfo
|
||||
modelInfo["ops"].append(opInfo)
|
||||
logModel("[OP index:" + str(index) + " type:" + op.type + "]")
|
||||
jsonDumpsIndentStr = json.dumps(opInfo, indent=2)
|
||||
logModel(jsonDumpsIndentStr)
|
||||
logModel("")
|
||||
index += 1
|
||||
print("Organizing model operators info successfully.")
|
||||
|
||||
def addChunkNumToJson(paramValueList):
|
||||
totalParamValuesCount = len(paramValueList)
|
||||
countPerSlice = int(sliceDataSize * 1024 / 4)
|
||||
count = totalParamValuesCount / countPerSlice
|
||||
modelInfo["chunkNum"] = math.ceil(count)
|
||||
print("Model chunkNum set successfully.")
|
||||
|
||||
def appendConnectOp(fetch_targets):
|
||||
targets = []
|
||||
inputNames = []
|
||||
totalShape = 0
|
||||
|
||||
# 从fetch_targets中提取输出算子信息
|
||||
for target in fetch_targets:
|
||||
name = target.name
|
||||
curVar = fluid.global_scope().find_var(name)
|
||||
curTensor = np.array(curVar.get_tensor())
|
||||
shape = list(curTensor.shape)
|
||||
totalShape += reduce(lambda x, y: x * y, shape)
|
||||
targets.append({'name': name, 'shape': excludeNegativeShape(shape)})
|
||||
inputNames.append(name)
|
||||
|
||||
# 构造connect算子
|
||||
op = {
|
||||
'attrs': {},
|
||||
'inputs': {'X': inputNames},
|
||||
'outputs': {'Out': ['connect_result']},
|
||||
'type': 'connect'
|
||||
}
|
||||
# 构造输出var
|
||||
outputVar = {'name': 'connect_result', 'shape': splitLargeNum(totalShape)}
|
||||
|
||||
ops = modelInfo['ops']
|
||||
vars = modelInfo['vars']
|
||||
|
||||
# 收集要删除的算子index
|
||||
delList = []
|
||||
for index, item in enumerate(ops):
|
||||
if item['type'] == 'fetch':
|
||||
delList.append(index)
|
||||
|
||||
# 去除fetch算子
|
||||
delCount = 0
|
||||
for delIndex in delList:
|
||||
del ops[delIndex - delCount]
|
||||
delCount += 1
|
||||
|
||||
fetchOp = {
|
||||
"attrs": {},
|
||||
"inputs": {
|
||||
"X": [
|
||||
"connect_result"
|
||||
]
|
||||
},
|
||||
"outputs": {
|
||||
"Out": [
|
||||
"fetch"
|
||||
]
|
||||
},
|
||||
"type": "fetch"
|
||||
}
|
||||
ops.append(op)
|
||||
ops.append(fetchOp)
|
||||
|
||||
vars['connect_result'] = outputVar
|
||||
modelInfo['multiOutputs'] = targets
|
||||
return targets
|
||||
|
||||
def genModelFeedShape(feed):
|
||||
if len(feed) != 1:
|
||||
print("\033[33;1mModel has more than one input feed.\033[0m")
|
||||
return
|
||||
|
||||
originFeedShape = modelInfo['vars'][feed[0]]['shape']
|
||||
feedShape = {}
|
||||
if len(originFeedShape) == 3:
|
||||
feedShape['fc'] = originFeedShape[0]
|
||||
feedShape['fh'] = originFeedShape[1]
|
||||
feedShape['fw'] = originFeedShape[2]
|
||||
elif len(originFeedShape) == 4:
|
||||
feedShape['fc'] = originFeedShape[1]
|
||||
feedShape['fh'] = originFeedShape[2]
|
||||
feedShape['fw'] = originFeedShape[3]
|
||||
elif len(originFeedShape) == 2:
|
||||
feedShape['fh'] = originFeedShape[0]
|
||||
feedShape['fw'] = originFeedShape[1]
|
||||
else:
|
||||
print("\033[33;1mFeedShape length is " + str(len(originFeedShape)) + ".\033[0m")
|
||||
return
|
||||
|
||||
modelInfo['feedShape'] = feedShape
|
||||
print("\033[32mModel FeedShape set successfully.\033[0m")
|
||||
|
||||
def convertToPaddleJSModel(modelDir, modelName, paramsName, outputDir, useGPUOpt):
|
||||
""" 转换fluid modle为paddleJS model """
|
||||
|
||||
|
||||
#In PaddlePaddle 2.x, we turn on dynamic graph mode by default, and 'load_inference_model()' is only supported in static graph mode. So call 'paddle.enable_static()' before this api to enter static graph mode.
|
||||
paddle.enable_static()
|
||||
|
||||
# 初始化fluid运行环境和配置
|
||||
global exe
|
||||
exe = fluid.Executor(fluid.CPUPlace())
|
||||
result = fluid.io.load_inference_model(dirname=modelDir, executor=exe, model_filename=modelName, params_filename=paramsName)
|
||||
global program
|
||||
program = result[0]
|
||||
fetch_targets = result[2]
|
||||
feed_target_names = result[1]
|
||||
|
||||
# 获取program中所有的op,按op顺序加入到model info
|
||||
organizeModelOpInfo()
|
||||
|
||||
# 获取program中所有的var,按照字母顺序加入到model info,同时读取参数数值
|
||||
organizeModelVariableInfo(result)
|
||||
|
||||
# 拆分rnn op
|
||||
if len(rnnList):
|
||||
for index in rnnList:
|
||||
rnn.splice_rnn_op(modelInfo, index)
|
||||
|
||||
if useGPUOpt:
|
||||
# 算子融合
|
||||
modelInfo['gpuOpt'] = True
|
||||
opListFuse(modelInfo['ops'])
|
||||
|
||||
# 对多输出模型追加connect算子
|
||||
if len(fetch_targets) > 1:
|
||||
appendConnectOp(fetch_targets)
|
||||
|
||||
if (postOps and len(postOps) > 0):
|
||||
for op in postOps:
|
||||
if (op['type'].startswith('multiclass_nms')):
|
||||
inputNames = []
|
||||
for input, value in op['inputs'].items():
|
||||
if len(value) <= 0:
|
||||
continue
|
||||
cur = ObjDict()
|
||||
cur.name = value[0]
|
||||
inputNames.append(cur)
|
||||
targets = appendConnectOp(inputNames)
|
||||
# op['inputs'] = targets
|
||||
keys = op['inputs'].keys()
|
||||
for i, val in enumerate(keys):
|
||||
op['inputs'][val] = targets[i]
|
||||
|
||||
|
||||
modelInfo['postOps'] = postOps
|
||||
|
||||
# 对参数数值dict,按照key(参数名)进行字母顺序排序,并组合到一起
|
||||
paramValues = reorderParamsValue()
|
||||
|
||||
# model.json 设置分片参数
|
||||
addChunkNumToJson(paramValues)
|
||||
|
||||
# model.json 设置 feedShape 输入 shape 信息
|
||||
genModelFeedShape(feed_target_names)
|
||||
|
||||
# 去掉无意义的 tensor 和对应 op
|
||||
pruningNoSenseTensor(modelInfo)
|
||||
|
||||
# 导出模型文件到json
|
||||
dumpModelToJsonFile(outputDir)
|
||||
|
||||
# 导出分片参数文件
|
||||
sliceDataToBinaryFile(paramValues, outputDir)
|
||||
|
||||
def main():
|
||||
|
||||
global sliceDataSize
|
||||
global enableLogModelInfo
|
||||
|
||||
try:
|
||||
p = argparse.ArgumentParser(description='模型转换参数解析')
|
||||
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数,且模型文件名必须为`__model__`', required=False)
|
||||
p.add_argument('--modelPath', help='fluid模型文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument('--paramPath', help='fluid参数文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument("--outputDir", help='paddleJS模型输出路径,必要参数', required=True)
|
||||
p.add_argument("--logModelInfo", type=int, default=0, help='是否输出模型结构信息,非必要参数,0为不输出,1为输出,默认不输出', required=False)
|
||||
p.add_argument("--sliceDataSize", type=int, default=4096, help='分片输出参数文件时,每片文件的大小,单位:KB,非必要参数,默认4096KB', required=False)
|
||||
p.add_argument('--useGPUOpt', help='转换模型是否执行GPU优化方法', required=False)
|
||||
|
||||
args = p.parse_args()
|
||||
modelDir = args.inputDir
|
||||
modelPath = args.modelPath
|
||||
paramPath = args.paramPath
|
||||
useGPUOpt = args.useGPUOpt
|
||||
|
||||
if not modelDir:
|
||||
modelDir, modelName = os.path.split(modelPath)
|
||||
paramDir, paramsName = os.path.split(paramPath)
|
||||
if paramDir != modelDir:
|
||||
print("\033[31mModel and param file should be put in a same directory!\033[0m")
|
||||
raise Exception()
|
||||
outputDir = args.outputDir
|
||||
sliceDataSize = args.sliceDataSize
|
||||
|
||||
if args.logModelInfo == 1:
|
||||
enableLogModelInfo = True
|
||||
|
||||
convertToPaddleJSModel(modelDir, modelName, paramsName, outputDir, useGPUOpt)
|
||||
|
||||
except Exception as identifier:
|
||||
print("\033[31mA fetal error occured. Failed to convert model.\033[0m")
|
||||
print(traceback.format_exc())
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@@ -0,0 +1,141 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
import sys
|
||||
import os
|
||||
import argparse
|
||||
import shutil
|
||||
import stat
|
||||
import traceback
|
||||
import copy
|
||||
|
||||
def cleanTempModel(optimizedModelTempDir):
|
||||
""" 清理opt优化完的临时模型文件 """
|
||||
if os.path.exists(optimizedModelTempDir):
|
||||
print("Cleaning optimized temporary model...")
|
||||
shutil.rmtree(optimizedModelTempDir, onerror=grantWritePermission)
|
||||
|
||||
def grantWritePermission(func, path, execinfo):
|
||||
""" 文件授权 """
|
||||
os.chmod(path, stat.S_IWRITE)
|
||||
func(path)
|
||||
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Example:
|
||||
'python convertToPaddleJSModel.py --modelPath=../infer_model/MobileNetV2/model --paramPath=../infer_model/MobileNetV2/params --outputDir=../jsmodel'
|
||||
"""
|
||||
try:
|
||||
p = argparse.ArgumentParser(description='转化为PaddleJS模型参数解析')
|
||||
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数,且模型文件名必须为`__model__`', required=False)
|
||||
p.add_argument('--modelPath', help='fluid模型文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument('--paramPath', help='fluid参数文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument("--outputDir", help='paddleJS模型输出路径,必要参数', required=True)
|
||||
p.add_argument("--disableOptimize", type=int, default=0, help='是否关闭模型优化,非必要参数,1为关闭优化,0为开启优化,默认开启优化', required=False)
|
||||
p.add_argument("--logModelInfo", type=int, default=0, help='是否输出模型结构信息,非必要参数,0为不输出,1为输出,默认不输出', required=False)
|
||||
p.add_argument("--sliceDataSize", type=int, default=4096, help='分片输出参数文件时,每片文件的大小,单位:KB,非必要参数,默认4096KB', required=False)
|
||||
p.add_argument('--useGPUOpt', help='转换模型是否执行GPU优化方法', required=False)
|
||||
|
||||
args = p.parse_args()
|
||||
# 获取当前用户使用的 python 解释器 bin 位置
|
||||
pythonCmd = sys.executable
|
||||
|
||||
# TODO: 由于PaddleLite和PaddlePaddle存在包冲突,因此将整个模型转换工具拆成两个python文件,由一个入口python文件通过命令行调用
|
||||
# 区分本地执行和命令行执行
|
||||
if os.path.exists("optimizeModel.py"):
|
||||
optimizeCmd = pythonCmd + " optimizeModel.py"
|
||||
else:
|
||||
optimizeCmd = "pdjsOptimizeModel"
|
||||
|
||||
if os.path.exists("convertModel.py"):
|
||||
convertCmd = pythonCmd + " convertModel.py"
|
||||
else:
|
||||
convertCmd = "pdjsConvertModel"
|
||||
|
||||
inputDir = args.inputDir
|
||||
modelPath = args.modelPath
|
||||
paramPath = args.paramPath
|
||||
outputDir = args.outputDir
|
||||
disableOptimization = args.disableOptimize
|
||||
args.disableOptimize = None
|
||||
enableLogModelInfo = args.logModelInfo
|
||||
sliceDataSize = args.sliceDataSize
|
||||
|
||||
optArgs = copy.deepcopy(args)
|
||||
|
||||
enableOptimization = 1 - disableOptimization
|
||||
optimizedModelTempDir = None
|
||||
if enableOptimization == 1:
|
||||
optimizedModelTempDir = os.path.join(outputDir, "optimize")
|
||||
optArgs.outputDir = optimizedModelTempDir
|
||||
if inputDir:
|
||||
args.inputDir = optimizedModelTempDir
|
||||
else:
|
||||
args.modelPath = os.path.join(optimizedModelTempDir, "model")
|
||||
args.paramPath = os.path.join(optimizedModelTempDir, "params")
|
||||
|
||||
|
||||
print("============Convert Model Args=============")
|
||||
if inputDir:
|
||||
print("inputDir: " + inputDir)
|
||||
else:
|
||||
print("modelPath: " + modelPath)
|
||||
print("paramPath: " + paramPath)
|
||||
print("outputDir: " + outputDir)
|
||||
print("enableOptimizeModel: " + str(enableOptimization))
|
||||
print("enableLogModelInfo: " + str(enableLogModelInfo))
|
||||
print("sliceDataSize:" + str(sliceDataSize))
|
||||
|
||||
|
||||
print("Starting...")
|
||||
if enableOptimization:
|
||||
print("Optimizing model...")
|
||||
for param in ["inputDir", "modelPath", "paramPath", "outputDir"]:
|
||||
if optArgs.__dict__[param]:
|
||||
# 用""框起命令参数值,解决路径中的空格问题
|
||||
optimizeCmd += " --" + param + "="+ '"' + str(optArgs.__dict__[param]) + '"'
|
||||
os.system(optimizeCmd)
|
||||
try:
|
||||
os.listdir(optimizedModelTempDir)
|
||||
except Exception as identifier:
|
||||
print("\n\033[31mOptimizing model failed.\033[0m")
|
||||
# restore inputDir or modelPath paramPath from optimize
|
||||
if inputDir:
|
||||
args.inputDir = inputDir
|
||||
else:
|
||||
args.modelPath = modelPath
|
||||
args.paramPath = paramPath
|
||||
else:
|
||||
print("\n\033[32mOptimizing model successfully.\033[0m")
|
||||
else:
|
||||
print("\033[33mYou choosed not to optimize model, consequently, optimizing model is skiped.\033[0m")
|
||||
|
||||
print("\nConverting model...")
|
||||
for param in args.__dict__:
|
||||
if args.__dict__[param]:
|
||||
# 用""框起参数,解决路径中的空格问题
|
||||
convertCmd += " --" + param + "=" + '"' + str(args.__dict__[param]) + '"'
|
||||
os.system(convertCmd)
|
||||
try:
|
||||
file = os.listdir(outputDir)
|
||||
if len(file) == 0:
|
||||
raise Exception
|
||||
except Exception as identifier:
|
||||
print("\033[31m============ALL DONE============\033[0m")
|
||||
else:
|
||||
if enableOptimization:
|
||||
cleanTempModel(optimizedModelTempDir)
|
||||
print("Temporary files has been deleted successfully.")
|
||||
print("\033[32mConverting model successfully.\033[0m")
|
||||
print("\033[32m============ALL DONE============\033[0m")
|
||||
|
||||
except Exception as identifier:
|
||||
print("\033[31mA fetal error occured. Failed to convert model.\033[0m")
|
||||
print(traceback.format_exc())
|
||||
exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
75
examples/application/js/web_demo/converter/fuseOps.py
Normal file
75
examples/application/js/web_demo/converter/fuseOps.py
Normal file
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
def opListFuse(ops):
|
||||
""" 算子融合 """
|
||||
fuseOpList = [
|
||||
'relu',
|
||||
'relu6',
|
||||
'leaky_relu',
|
||||
'scale',
|
||||
'sigmoid',
|
||||
'hard_sigmoid',
|
||||
'pow',
|
||||
'sqrt',
|
||||
'tanh',
|
||||
'hard_swish',
|
||||
'dropout'
|
||||
]
|
||||
|
||||
# 判断op是否为单节点
|
||||
def opExistSingleNode(opName):
|
||||
name = opName
|
||||
if name:
|
||||
nodeNum = 0
|
||||
for i in range(len(ops)):
|
||||
op = ops[i]
|
||||
if 'X' not in op['inputs']:
|
||||
continue
|
||||
|
||||
inputName = op['inputs']['X']
|
||||
for x in inputName:
|
||||
if x == name:
|
||||
nodeNum = nodeNum + 1
|
||||
|
||||
return True if nodeNum == 1 else False
|
||||
|
||||
else:
|
||||
return False
|
||||
|
||||
|
||||
for index in reversed(range(len(ops))):
|
||||
if index > 0:
|
||||
op = ops[index]
|
||||
|
||||
# 兼容paddlelite 算子融合字段
|
||||
if 'act_type' in op['attrs']:
|
||||
name = op['attrs']['act_type']
|
||||
op['attrs']['fuse_opt'] = {}
|
||||
op['attrs']['fuse_opt'][name] = {}
|
||||
|
||||
if name == 'hard_swish':
|
||||
op['attrs']['fuse_opt'][name]['offset'] = op['attrs']['hard_swish_offset']
|
||||
op['attrs']['fuse_opt'][name]['scale'] = op['attrs']['hard_swish_scale']
|
||||
op['attrs']['fuse_opt'][name]['threshold'] = op['attrs']['hard_swish_threshold']
|
||||
|
||||
if name == 'relu6':
|
||||
op['attrs']['fuse_opt'][name]['threshold'] = op['attrs']['fuse_brelu_threshold']
|
||||
|
||||
for fuse in fuseOpList:
|
||||
if op['type'] == fuse:
|
||||
prevOp = ops[index - 1]
|
||||
|
||||
if opExistSingleNode(prevOp['outputs']['Out'][0]) and len(prevOp['outputs']['Out']) == 1 :
|
||||
prevOp['attrs']['fuse_opt'] = {}
|
||||
if 'fuse_opt' in op['attrs']:
|
||||
prevOp['attrs']['fuse_opt'] = op['attrs']['fuse_opt']
|
||||
del op['attrs']['fuse_opt']
|
||||
|
||||
prevOp['attrs']['fuse_opt'][fuse] = op['attrs']
|
||||
prevOp['outputs']['Out'] = op['outputs']['Out']
|
||||
|
||||
del ops[index]
|
||||
|
||||
|
||||
|
53
examples/application/js/web_demo/converter/optimizeModel.py
Normal file
53
examples/application/js/web_demo/converter/optimizeModel.py
Normal file
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
import collections
|
||||
import argparse
|
||||
import traceback
|
||||
from paddlejslite import lite
|
||||
import pkg_resources
|
||||
from packaging import version
|
||||
|
||||
lite_version = pkg_resources.get_distribution("paddlelite").version
|
||||
|
||||
def optimizeModel(inputDir, modelPath, paramPath, outputDir):
|
||||
""" 使用opt python接口执行模型优化 """
|
||||
opt = lite.Opt()
|
||||
if inputDir:
|
||||
# 分片参数文件优化
|
||||
opt.set_model_dir(inputDir)
|
||||
else:
|
||||
# 合并参数文件优化
|
||||
opt.set_model_file(modelPath)
|
||||
opt.set_param_file(paramPath)
|
||||
|
||||
opt.set_valid_places("arm")
|
||||
opt.set_model_type("protobuf")
|
||||
opt.set_optimize_out(outputDir)
|
||||
opt.run()
|
||||
|
||||
|
||||
def main():
|
||||
try:
|
||||
p = argparse.ArgumentParser('模型优化参数解析')
|
||||
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数,且模型文件名必须为`__model__`', required=False)
|
||||
p.add_argument('--modelPath', help='fluid模型文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument('--paramPath', help='fluid参数文件所在路径,使用合并参数文件时使用该参数', required=False)
|
||||
p.add_argument("--outputDir", help='优化后fluid模型目录,必要参数', required=True)
|
||||
|
||||
args = p.parse_args()
|
||||
inputDir = args.inputDir
|
||||
modelPath = args.modelPath
|
||||
paramPath = args.paramPath
|
||||
outputDir = args.outputDir
|
||||
|
||||
optimizeModel(inputDir, modelPath, paramPath, outputDir)
|
||||
|
||||
except Exception as identifier:
|
||||
print("\033[31mA fetal error occured. Failed to optimize model.\033[0m")
|
||||
print(traceback.format_exc())
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
64
examples/application/js/web_demo/converter/pruningModel.py
Normal file
64
examples/application/js/web_demo/converter/pruningModel.py
Normal file
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
|
||||
# pruning op tensor and relatad op with no sense, like ShapeTensor and OutSize
|
||||
def pruningNoSenseTensor(model):
|
||||
global ops
|
||||
ops = model["ops"]
|
||||
global vars
|
||||
vars = model["vars"]
|
||||
for op in ops[:]:
|
||||
shapeTensor = op["inputs"].get("ShapeTensor")
|
||||
outSizeTensor = op["inputs"].get("OutSize")
|
||||
|
||||
noSenseTensor = shapeTensor or outSizeTensor
|
||||
if not noSenseTensor:
|
||||
continue
|
||||
|
||||
print(noSenseTensor)
|
||||
if shapeTensor:
|
||||
del op["inputs"]["ShapeTensor"]
|
||||
if outSizeTensor:
|
||||
del op["inputs"]["OutSize"]
|
||||
|
||||
for tensorId in noSenseTensor:
|
||||
delLeafOpWithoutChildren(tensorId)
|
||||
|
||||
|
||||
# delete leaf op which has no child
|
||||
def delLeafOpWithoutChildren(tensorId):
|
||||
# judge if there is an op which used the tensor
|
||||
for op in ops[:]:
|
||||
inputsTensor = op["inputs"]
|
||||
input = inputsTensor.get("Input") or inputsTensor.get("X")
|
||||
|
||||
if input and (tensorId in input):
|
||||
return
|
||||
|
||||
op = getOpByOutputTensor(tensorId)
|
||||
if not op:
|
||||
return
|
||||
|
||||
# del op
|
||||
ops.remove(op)
|
||||
# del vars
|
||||
del vars[tensorId]
|
||||
|
||||
# upward recursion
|
||||
delOpinputsTensor = op["inputs"]
|
||||
input = delOpinputsTensor.get("Input") or delOpinputsTensor.get("X")
|
||||
if not input:
|
||||
return
|
||||
for inputTensorId in input:
|
||||
delLeafOpWithoutChildren(inputTensorId)
|
||||
|
||||
|
||||
|
||||
# find op by output tensor id
|
||||
def getOpByOutputTensor(tensorId):
|
||||
for op in ops[:]:
|
||||
outputTensor = op["outputs"]
|
||||
out = outputTensor.get("Out") or outputTensor.get("Output") or outputTensor.get("Y")
|
||||
if out[0] == tensorId:
|
||||
return op
|
35
examples/application/js/web_demo/converter/publish.sh
Normal file
35
examples/application/js/web_demo/converter/publish.sh
Normal file
@@ -0,0 +1,35 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
shopt -s extglob
|
||||
|
||||
|
||||
|
||||
if [ $# == 0 ];then
|
||||
ENV="test"
|
||||
else
|
||||
ENV=$1
|
||||
fi
|
||||
|
||||
# clear dist
|
||||
rm -rf dist
|
||||
|
||||
# build
|
||||
python3 setup.py sdist bdist_wheel
|
||||
|
||||
|
||||
# publish
|
||||
|
||||
if [ ${ENV} == 'production' ];then
|
||||
echo "publish to https://pypi.org"
|
||||
python3 -m twine upload dist/*
|
||||
else
|
||||
echo "publish to https://test.pypi.org/"
|
||||
python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*
|
||||
fi
|
||||
|
||||
# del egg-info and build
|
||||
rm -rf paddlejsconverter.egg-info
|
||||
rm -rf build
|
||||
|
273
examples/application/js/web_demo/converter/rnn.py
Normal file
273
examples/application/js/web_demo/converter/rnn.py
Normal file
@@ -0,0 +1,273 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
|
||||
def splice_rnn_op(model_info, rnn_index):
|
||||
|
||||
global input_shape
|
||||
global weight_0_shape
|
||||
global weight_1_shape
|
||||
global rnn_input_name
|
||||
ops = model_info['ops']
|
||||
vars = model_info['vars']
|
||||
op = ops[rnn_index]
|
||||
|
||||
rnn_input_name = op['inputs']['Input'][0]
|
||||
rnn_output_name = op['outputs']['Out'][0]
|
||||
|
||||
is_bidirec = 2 if op['attrs']['is_bidirec'] else 1
|
||||
num_layers = op['attrs']['num_layers']
|
||||
hidden_size = op['attrs']['hidden_size']
|
||||
layer_num = num_layers * is_bidirec
|
||||
# concat input最大值
|
||||
max_concat_num = 15
|
||||
|
||||
def concat_mul(index, list, num):
|
||||
global rnn_input_name
|
||||
end = len(list)
|
||||
|
||||
if end < max_concat_num:
|
||||
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(num) + '.tmp_concat'
|
||||
# 非最后一层遍历,将concat作为下一层输入
|
||||
if index < is_bidirec * num_layers - 1:
|
||||
rnn_input_name = concat_output_name
|
||||
|
||||
# 最后一层遍历,将rnn_output_name赋给最后一个concat
|
||||
else:
|
||||
concat_output_name = rnn_output_name
|
||||
|
||||
concat_op = {
|
||||
'attrs': {
|
||||
'axis': 0
|
||||
},
|
||||
'inputs': {
|
||||
'X': []
|
||||
},
|
||||
'outputs': {'Out': [concat_output_name]},
|
||||
'type': 'concat'
|
||||
}
|
||||
|
||||
concat_output_shape = 0
|
||||
|
||||
for x in range(0, end):
|
||||
x_input_name = 'lstm_' + str(index - 1) + '_' + str(list[x]) + '.tmp_concat'
|
||||
concat_op['inputs']['X'].append(x_input_name)
|
||||
concat_output_shape += vars[x_input_name]['shape'][0]
|
||||
|
||||
concat_var = {
|
||||
'name': concat_output_name,
|
||||
'persistable': False,
|
||||
'shape': [concat_output_shape, 1, weight_1_shape[1] * 2]
|
||||
}
|
||||
|
||||
ops.append(concat_op)
|
||||
|
||||
if index < is_bidirec * num_layers - 1:
|
||||
vars[concat_output_name] = concat_var
|
||||
return
|
||||
|
||||
# concat新列表
|
||||
new_list = []
|
||||
|
||||
for i in range(0, end, max_concat_num):
|
||||
if i + max_concat_num > end:
|
||||
for n in range(i, end):
|
||||
new_list.append(list[n])
|
||||
break
|
||||
|
||||
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(num) + '.tmp_concat'
|
||||
# concat_list长度为max_concat_num && 最后一层遍历,将rnn_output_name赋给最后一个concat
|
||||
if end == max_concat_num and index == is_bidirec * num_layers - 1:
|
||||
concat_output_name = rnn_output_name
|
||||
|
||||
concat_op = {
|
||||
'attrs': {
|
||||
'axis': 0
|
||||
},
|
||||
'inputs': {
|
||||
'X': []
|
||||
},
|
||||
'outputs': {'Out': [concat_output_name]},
|
||||
'type': 'concat'
|
||||
}
|
||||
|
||||
concat_output_shape = 0
|
||||
|
||||
for x in range(0, max_concat_num):
|
||||
x_input_name = 'lstm_' + str(index - 1) + '_' + str(list[i + x]) + '.tmp_concat'
|
||||
concat_op['inputs']['X'].append(x_input_name)
|
||||
concat_output_shape += vars[x_input_name]['shape'][0]
|
||||
|
||||
concat_var = {
|
||||
'name': concat_output_name,
|
||||
'persistable': False,
|
||||
'shape': [concat_output_shape, 1, weight_1_shape[1] * 2]
|
||||
}
|
||||
|
||||
ops.append(concat_op)
|
||||
vars[concat_output_name] = concat_var
|
||||
new_list.append(num)
|
||||
|
||||
# 若concat_list长度为max_concat_num,在下一次递归时直接修改rnn_input_name,结束递归,num无需+1
|
||||
if end != max_concat_num:
|
||||
num += 1
|
||||
|
||||
concat_mul(index, new_list, num)
|
||||
|
||||
for index in range(layer_num):
|
||||
last_hidden = op['inputs']['PreState'][0]
|
||||
last_cell = op['inputs']['PreState'][1]
|
||||
weight_list_0 = op['inputs']['WeightList'][index * 2]
|
||||
weight_list_1 = op['inputs']['WeightList'][index * 2 + 1]
|
||||
weight_list_2 = op['inputs']['WeightList'][(index + num_layers * is_bidirec) * 2]
|
||||
weight_list_3 = op['inputs']['WeightList'][(index + num_layers * is_bidirec) * 2 + 1]
|
||||
output_name = 'rnn_origin_' + str(index)
|
||||
input_shape = vars[rnn_input_name]['shape']
|
||||
batch = input_shape[0]
|
||||
|
||||
if vars[weight_list_0]:
|
||||
weight_0_shape = vars[weight_list_0]['shape']
|
||||
|
||||
if vars[weight_list_1]:
|
||||
weight_1_shape = vars[weight_list_1]['shape']
|
||||
|
||||
if batch == 0:
|
||||
continue
|
||||
|
||||
origin_op = {
|
||||
'attrs': {
|
||||
'state_axis': index
|
||||
},
|
||||
'inputs': {
|
||||
'Input': [rnn_input_name],
|
||||
'PreState': [last_hidden],
|
||||
'WeightList': [
|
||||
weight_list_0,
|
||||
weight_list_1,
|
||||
weight_list_2,
|
||||
weight_list_3
|
||||
]
|
||||
},
|
||||
'outputs': {'Out': [output_name]},
|
||||
'type': 'rnn_origin'
|
||||
}
|
||||
|
||||
origin_var = {
|
||||
'name': output_name,
|
||||
'persistable': False,
|
||||
'shape': [input_shape[0], input_shape[1], weight_0_shape[0]]
|
||||
}
|
||||
ops.append(origin_op)
|
||||
vars[output_name] = origin_var
|
||||
|
||||
for bat in range(batch):
|
||||
matmul_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_matmul'
|
||||
cell_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_c'
|
||||
hidden_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_h'
|
||||
|
||||
matmul_op = {
|
||||
'attrs': {
|
||||
'input_axis': bat,
|
||||
'state_axis': index if bat == 0 else 0,
|
||||
'batch': batch,
|
||||
'reverse': False if index % 2 == 0 else True
|
||||
},
|
||||
'inputs': {
|
||||
'Input': [output_name],
|
||||
'PreState': [last_hidden],
|
||||
'WeightList': [weight_list_1]
|
||||
},
|
||||
'outputs': {'Out': [matmul_output_name]},
|
||||
'type': 'rnn_matmul'
|
||||
}
|
||||
|
||||
matmul_var = {
|
||||
'name': matmul_output_name,
|
||||
'persistable': False,
|
||||
'shape': [1, 1, weight_0_shape[0]]
|
||||
}
|
||||
|
||||
ops.append(matmul_op)
|
||||
vars[matmul_output_name] = matmul_var
|
||||
|
||||
cell_op = {
|
||||
'attrs': {
|
||||
'state_axis': index if bat == 0 else 0,
|
||||
'hidden_size': hidden_size
|
||||
},
|
||||
'inputs': {
|
||||
'X': [matmul_output_name],
|
||||
'Y': [last_cell]
|
||||
},
|
||||
'outputs': {'Out': [cell_output_name]},
|
||||
'type': 'rnn_cell'
|
||||
}
|
||||
|
||||
cell_var = {
|
||||
'name': cell_output_name,
|
||||
'persistable': False,
|
||||
'shape': [1, 1, weight_1_shape[1]]
|
||||
}
|
||||
|
||||
ops.append(cell_op)
|
||||
vars[cell_output_name] = cell_var
|
||||
|
||||
hidden_op = {
|
||||
'attrs': {
|
||||
'state_axis': index if bat == 0 else 0,
|
||||
'hidden_size': hidden_size
|
||||
},
|
||||
'inputs': {
|
||||
'X': [matmul_output_name],
|
||||
'Y': [last_cell]
|
||||
},
|
||||
'outputs': {'Out': [hidden_output_name]},
|
||||
'type': 'rnn_hidden'
|
||||
}
|
||||
|
||||
hidden_var = {
|
||||
'name': hidden_output_name,
|
||||
'persistable': False,
|
||||
'shape': [1, 1, weight_1_shape[1]]
|
||||
}
|
||||
|
||||
ops.append(hidden_op)
|
||||
vars[hidden_output_name] = hidden_var
|
||||
|
||||
last_hidden = hidden_output_name
|
||||
last_cell = cell_output_name
|
||||
|
||||
# concat
|
||||
if index % 2 == 1:
|
||||
|
||||
concat_list = []
|
||||
concat_num = 0
|
||||
# concat forword and backword
|
||||
for bat in range(batch):
|
||||
x_input_name_0 = 'lstm_' + str(index - 1) + '_' + str(bat) + '.tmp_h'
|
||||
x_input_name_1 = 'lstm_' + str(index) + '_' + str(batch - bat - 1) + '.tmp_h'
|
||||
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(bat) + '.tmp_concat'
|
||||
concat_op = {
|
||||
'attrs': {
|
||||
'axis': 2
|
||||
},
|
||||
'inputs': {
|
||||
'X': [x_input_name_0, x_input_name_1]
|
||||
},
|
||||
'outputs': {'Out': [concat_output_name]},
|
||||
'type': 'concat'
|
||||
}
|
||||
|
||||
concat_var = {
|
||||
'name': concat_output_name,
|
||||
'persistable': False,
|
||||
'shape': [1, 1, weight_1_shape[1] * 2]
|
||||
}
|
||||
ops.append(concat_op)
|
||||
vars[concat_output_name] = concat_var
|
||||
concat_list.append(bat)
|
||||
concat_num += 1
|
||||
|
||||
concat_mul(index, concat_list, concat_num)
|
||||
|
||||
# 删除rnn op
|
||||
del ops[rnn_index]
|
37
examples/application/js/web_demo/converter/setup.py
Normal file
37
examples/application/js/web_demo/converter/setup.py
Normal file
@@ -0,0 +1,37 @@
|
||||
import setuptools
|
||||
|
||||
PY_MODILES = ["convertToPaddleJSModel", "convertModel", "optimizeModel", "pruningModel", "rnn", "fuseOps"]
|
||||
|
||||
with open("README.md", "r") as fh:
|
||||
long_description = fh.read()
|
||||
|
||||
setuptools.setup(
|
||||
name="paddlejsconverter",
|
||||
version="1.0.7",
|
||||
author="paddlejs",
|
||||
author_email="382248373@qq.com",
|
||||
description="Paddlejs model converter",
|
||||
long_description=long_description,
|
||||
long_description_content_type="text/markdown",
|
||||
url="https://github.com/PaddlePaddle/Paddle.js",
|
||||
py_modules=PY_MODILES,
|
||||
classifiers=[
|
||||
"Programming Language :: Python",
|
||||
"Programming Language :: Python :: 3",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Operating System :: OS Independent",
|
||||
],
|
||||
python_requires='>=3.6',
|
||||
install_requires=[
|
||||
"paddlepaddle >= 2.0.0",
|
||||
"paddlejslite >= 0.0.2",
|
||||
"numpy"
|
||||
],
|
||||
entry_points={
|
||||
"console_scripts": [
|
||||
"paddlejsconverter = convertToPaddleJSModel:main",
|
||||
"pdjsConvertModel = convertModel:main",
|
||||
"pdjsOptimizeModel = optimizeModel:main"
|
||||
]
|
||||
}
|
||||
)
|
3
examples/application/js/web_demo/demo/.commitlintrc.js
Normal file
3
examples/application/js/web_demo/demo/.commitlintrc.js
Normal file
@@ -0,0 +1,3 @@
|
||||
module.exports = {
|
||||
extends: ['@commitlint/config-conventional'], // 使用预设的配置 https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/config-conventional/index.js
|
||||
}
|
15
examples/application/js/web_demo/demo/.eslintrc.cjs
Normal file
15
examples/application/js/web_demo/demo/.eslintrc.cjs
Normal file
@@ -0,0 +1,15 @@
|
||||
/* eslint-env node */
|
||||
require("@rushstack/eslint-patch/modern-module-resolution");
|
||||
|
||||
module.exports = {
|
||||
root: true,
|
||||
extends: [
|
||||
"plugin:vue/vue3-essential",
|
||||
"eslint:recommended",
|
||||
"@vue/eslint-config-typescript",
|
||||
"@vue/eslint-config-prettier",
|
||||
],
|
||||
parserOptions: {
|
||||
ecmaVersion: "latest",
|
||||
},
|
||||
};
|
31
examples/application/js/web_demo/demo/.gitignore
vendored
Normal file
31
examples/application/js/web_demo/demo/.gitignore
vendored
Normal file
@@ -0,0 +1,31 @@
|
||||
# Logs
|
||||
logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
lerna-debug.log*
|
||||
|
||||
node_modules
|
||||
.DS_Store
|
||||
dist
|
||||
dist-ssr
|
||||
coverage
|
||||
*.local
|
||||
|
||||
/cypress/videos/
|
||||
/cypress/screenshots/
|
||||
|
||||
# Editor directories and files
|
||||
.vscode/*
|
||||
!.vscode/extensions.json
|
||||
.idea
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
|
||||
.yalc
|
||||
yalc.lock
|
1
examples/application/js/web_demo/demo/.prettierrc.json
Normal file
1
examples/application/js/web_demo/demo/.prettierrc.json
Normal file
@@ -0,0 +1 @@
|
||||
{}
|
81
examples/application/js/web_demo/demo/README.md
Normal file
81
examples/application/js/web_demo/demo/README.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Paddle.js-demo
|
||||
|
||||
该分支是 Paddle.js 应用的Demo集合;可以查看各个 Demo 的实现代码。
|
||||
|
||||
## Demo 目录
|
||||
|
||||
| 分类 | 名称 | 目录 |
|
||||
|:----:| :--------------- | -------------------------------------------------------- |
|
||||
| CV | 人像扣图 | /src/views/cv/segmentation/PortraitMatting |
|
||||
| CV | 人像分割背景替换 | /src/views/cv/segmentation/PortraitBackgroundReplacement |
|
||||
| CV | 手势识别AI猜丁壳 | /src/views/cv/recognition/GestureRecognition |
|
||||
| CV | 1000种物品识别 | /src/views/cv/recognition/ItemIdentification |
|
||||
| CV | 酒瓶识别 | /src/views/cv/recognition/WineBottleIdentification |
|
||||
| CV | 文本检测 | /src/views/cv/ocr/TextDetection |
|
||||
| CV | 文本识别 | /src/views/cv/ocr/TextRecognition |
|
||||
|
||||
## 开发简介
|
||||
|
||||
### 安装依赖
|
||||
|
||||
```sh
|
||||
npm install
|
||||
```
|
||||
|
||||
### 开发
|
||||
|
||||
```sh
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### 构建
|
||||
|
||||
```sh
|
||||
npm run build
|
||||
```
|
||||
|
||||
### [ESLint](https://eslint.org/) 格式化
|
||||
|
||||
```sh
|
||||
npm run lint
|
||||
```
|
||||
|
||||
### 工程风格
|
||||
|
||||
1. 项目使用TypeScript
|
||||
2. 推荐使用 Vue 的组合式 API,可以根据 'src/views/ExampleFile.vue' 模板创建新的组件
|
||||
3. CSS 使用 Less
|
||||
4. eslint 使用的是 Vue 推荐的,一般情况请尽量符合对应的要求
|
||||
5. store 使用的是 [Pinia](https://pinia.web3doc.top/)
|
||||
6. router 使用的是 [vue-router](https://router.vuejs.org/zh/)
|
||||
|
||||
### src 目录简介
|
||||
|
||||
```text
|
||||
├─assets 资源文件
|
||||
├─components 全局组件
|
||||
├─router 路由
|
||||
├─stores 存储库
|
||||
└─views 视图
|
||||
└─cv cv相关demo
|
||||
├─ocr ocr相关demo
|
||||
│ ├─TextDetection
|
||||
│ └─TextRecognition
|
||||
├─...
|
||||
├─recognition 识别相关demo
|
||||
│ ├─GestureRecognition
|
||||
│ ├─ItemIdentification
|
||||
│ ├─...
|
||||
│ └─WineBottleIdentification
|
||||
└─segmentation 分割相关demo
|
||||
├─PortraitBackgroundReplacement
|
||||
├─...
|
||||
└─PortraitMatting
|
||||
|
||||
```
|
||||
新增组件在对应类别下新增即可,可以参考模板 'src/views/ExampleFile.vue'
|
||||
|
||||
### 提交相关
|
||||
|
||||
1. 提交前会自动进行全部文件的 ESLint 格式化
|
||||
2. commit 格式需要遵守 `<type>(<scope>): <subject>`,会自动进行检查
|
5
examples/application/js/web_demo/demo/auto-imports.d.ts
vendored
Normal file
5
examples/application/js/web_demo/demo/auto-imports.d.ts
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
// Generated by 'unplugin-auto-import'
|
||||
export {}
|
||||
declare global {
|
||||
|
||||
}
|
35
examples/application/js/web_demo/demo/components.d.ts
vendored
Normal file
35
examples/application/js/web_demo/demo/components.d.ts
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
// generated by unplugin-vue-components
|
||||
// We suggest you to commit this file into source control
|
||||
// Read more: https://github.com/vuejs/core/pull/3399
|
||||
import '@vue/runtime-core'
|
||||
|
||||
export {}
|
||||
|
||||
declare module '@vue/runtime-core' {
|
||||
export interface GlobalComponents {
|
||||
ElAside: typeof import('element-plus/es')['ElAside']
|
||||
ElButton: typeof import('element-plus/es')['ElButton']
|
||||
ElCard: typeof import('element-plus/es')['ElCard']
|
||||
ElCol: typeof import('element-plus/es')['ElCol']
|
||||
ElContainer: typeof import('element-plus/es')['ElContainer']
|
||||
ElDialog: typeof import('element-plus/es')['ElDialog']
|
||||
ElFooter: typeof import('element-plus/es')['ElFooter']
|
||||
ElHeader: typeof import('element-plus/es')['ElHeader']
|
||||
ElInput: typeof import('element-plus/es')['ElInput']
|
||||
ElMain: typeof import('element-plus/es')['ElMain']
|
||||
ElMenu: typeof import('element-plus/es')['ElMenu']
|
||||
ElMenuItem: typeof import('element-plus/es')['ElMenuItem']
|
||||
ElRow: typeof import('element-plus/es')['ElRow']
|
||||
ElSubMenu: typeof import('element-plus/es')['ElSubMenu']
|
||||
HelloWorld: typeof import('./src/components/HelloWorld.vue')['default']
|
||||
IconCommunity: typeof import('./src/components/icons/IconCommunity.vue')['default']
|
||||
IconDocumentation: typeof import('./src/components/icons/IconDocumentation.vue')['default']
|
||||
IconEcosystem: typeof import('./src/components/icons/IconEcosystem.vue')['default']
|
||||
IconSupport: typeof import('./src/components/icons/IconSupport.vue')['default']
|
||||
IconTooling: typeof import('./src/components/icons/IconTooling.vue')['default']
|
||||
RouterLink: typeof import('vue-router')['RouterLink']
|
||||
RouterView: typeof import('vue-router')['RouterView']
|
||||
TheWelcome: typeof import('./src/components/TheWelcome.vue')['default']
|
||||
WelcomeItem: typeof import('./src/components/WelcomeItem.vue')['default']
|
||||
}
|
||||
}
|
1
examples/application/js/web_demo/demo/env.d.ts
vendored
Normal file
1
examples/application/js/web_demo/demo/env.d.ts
vendored
Normal file
@@ -0,0 +1 @@
|
||||
/// <reference types="vite/client" />
|
52
examples/application/js/web_demo/demo/package.json
Normal file
52
examples/application/js/web_demo/demo/package.json
Normal file
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"name": "paddle-js-demo",
|
||||
"version": "1.0.0",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "run-p type-check build-only",
|
||||
"preview": "vite preview --port 4173",
|
||||
"build-only": "vite build",
|
||||
"type-check": "vue-tsc --noEmit",
|
||||
"lint": "eslint . --ext .vue,.js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --fix --ignore-path .gitignore"
|
||||
},
|
||||
"dependencies": {
|
||||
"@antv/x6": "^1.34.2",
|
||||
"@paddle-js-models/detect": "^3.0.0",
|
||||
"@paddle-js-models/facedetect": "^3.0.0",
|
||||
"@paddle-js-models/gesture": "^3.0.0",
|
||||
"@paddle-js-models/humanseg": "^3.0.0",
|
||||
"@paddle-js-models/humanseg_gpu": "^3.0.0",
|
||||
"@paddle-js-models/mobilenet": "^3.0.0",
|
||||
"@paddle-js-models/ocr": "3.1.0",
|
||||
"@paddle-js-models/ocrdet": "3.1.0",
|
||||
"@paddlejs/paddlejs-backend-webgl": "^1.2.9",
|
||||
"@paddlejs/paddlejs-core": "^2.2.0",
|
||||
"element-plus": "^2.2.16",
|
||||
"less": "^4.1.3",
|
||||
"less-loader": "^11.0.0",
|
||||
"pinia": "^2.0.21",
|
||||
"vue": "^3.2.38",
|
||||
"vue-router": "^4.1.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@commitlint/cli": "^17.1.2",
|
||||
"@commitlint/config-conventional": "^17.1.0",
|
||||
"@rushstack/eslint-patch": "^1.1.4",
|
||||
"@types/node": "^16.11.56",
|
||||
"@vitejs/plugin-vue": "^3.0.3",
|
||||
"@vue/eslint-config-prettier": "^7.0.0",
|
||||
"@vue/eslint-config-typescript": "^11.0.0",
|
||||
"@vue/tsconfig": "^0.1.3",
|
||||
"commitlint": "^17.1.2",
|
||||
"eslint": "8.5.0",
|
||||
"eslint-plugin-vue": "^9.3.0",
|
||||
"husky": "^8.0.1",
|
||||
"npm-run-all": "^4.1.5",
|
||||
"prettier": "^2.7.1",
|
||||
"typescript": "~4.7.4",
|
||||
"unplugin-auto-import": "^0.11.2",
|
||||
"unplugin-vue-components": "^0.22.4",
|
||||
"vite": "^3.0.9",
|
||||
"vue-tsc": "^0.40.7"
|
||||
}
|
||||
}
|
BIN
examples/application/js/web_demo/demo/public/favicon.ico
Normal file
BIN
examples/application/js/web_demo/demo/public/favicon.ico
Normal file
Binary file not shown.
After Width: | Height: | Size: 4.2 KiB |
74
examples/application/js/web_demo/demo/src/assets/base.css
Normal file
74
examples/application/js/web_demo/demo/src/assets/base.css
Normal file
@@ -0,0 +1,74 @@
|
||||
/* color palette from <https://github.com/vuejs/theme> */
|
||||
:root {
|
||||
--vt-c-white: #ffffff;
|
||||
--vt-c-white-soft: #f8f8f8;
|
||||
--vt-c-white-mute: #f2f2f2;
|
||||
|
||||
--vt-c-black: #181818;
|
||||
--vt-c-black-soft: #222222;
|
||||
--vt-c-black-mute: #282828;
|
||||
|
||||
--vt-c-indigo: #2c3e50;
|
||||
|
||||
--vt-c-divider-light-1: rgba(60, 60, 60, 0.29);
|
||||
--vt-c-divider-light-2: rgba(60, 60, 60, 0.12);
|
||||
--vt-c-divider-dark-1: rgba(84, 84, 84, 0.65);
|
||||
--vt-c-divider-dark-2: rgba(84, 84, 84, 0.48);
|
||||
|
||||
--vt-c-text-light-1: var(--vt-c-indigo);
|
||||
--vt-c-text-light-2: rgba(60, 60, 60, 0.66);
|
||||
--vt-c-text-dark-1: var(--vt-c-white);
|
||||
--vt-c-text-dark-2: rgba(235, 235, 235, 0.64);
|
||||
}
|
||||
|
||||
/* semantic color variables for this project */
|
||||
:root {
|
||||
--color-background: var(--vt-c-white);
|
||||
--color-background-soft: var(--vt-c-white-soft);
|
||||
--color-background-mute: var(--vt-c-white-mute);
|
||||
|
||||
--color-border: var(--vt-c-divider-light-2);
|
||||
--color-border-hover: var(--vt-c-divider-light-1);
|
||||
|
||||
--color-heading: var(--vt-c-text-light-1);
|
||||
--color-text: var(--vt-c-text-light-1);
|
||||
|
||||
--section-gap: 160px;
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--color-background: var(--vt-c-black);
|
||||
--color-background-soft: var(--vt-c-black-soft);
|
||||
--color-background-mute: var(--vt-c-black-mute);
|
||||
|
||||
--color-border: var(--vt-c-divider-dark-2);
|
||||
--color-border-hover: var(--vt-c-divider-dark-1);
|
||||
|
||||
--color-heading: var(--vt-c-text-dark-1);
|
||||
--color-text: var(--vt-c-text-dark-2);
|
||||
}
|
||||
}
|
||||
|
||||
*,
|
||||
*::before,
|
||||
*::after {
|
||||
box-sizing: border-box;
|
||||
margin: 0;
|
||||
position: relative;
|
||||
font-weight: normal;
|
||||
}
|
||||
|
||||
body {
|
||||
min-height: 100vh;
|
||||
color: var(--color-text);
|
||||
background: var(--color-background);
|
||||
transition: color 0.5s, background-color 0.5s;
|
||||
line-height: 1.6;
|
||||
font-family: Inter, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu,
|
||||
Cantarell, 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
|
||||
font-size: 15px;
|
||||
text-rendering: optimizeLegibility;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
}
|
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 261.76 226.69" xmlns:v="https://vecta.io/nano"><path d="M161.096.001l-30.225 52.351L100.647.001H-.005l130.877 226.688L261.749.001z" fill="#41b883"/><path d="M161.096.001l-30.225 52.351L100.647.001H52.346l78.526 136.01L209.398.001z" fill="#34495e"/></svg>
|
After Width: | Height: | Size: 308 B |
@@ -0,0 +1,8 @@
|
||||
@import "./base.css";
|
||||
|
||||
#app {
|
||||
height: 100vh;
|
||||
width: 100%;
|
||||
margin: 0 auto;
|
||||
padding: 2rem;
|
||||
}
|
@@ -0,0 +1,41 @@
|
||||
<script setup lang="ts">
|
||||
defineProps<{
|
||||
msg: string;
|
||||
}>();
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="greetings">
|
||||
<h1 class="green">{{ msg }}</h1>
|
||||
<h3>
|
||||
You’ve successfully created a project with
|
||||
<a href="https://vitejs.dev/" target="_blank" rel="noopener">Vite</a> +
|
||||
<a href="https://vuejs.org/" target="_blank" rel="noopener">Vue 3</a>.
|
||||
What's next?
|
||||
</h3>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
h1 {
|
||||
font-weight: 500;
|
||||
font-size: 2.6rem;
|
||||
top: -10px;
|
||||
}
|
||||
|
||||
h3 {
|
||||
font-size: 1.2rem;
|
||||
}
|
||||
|
||||
.greetings h1,
|
||||
.greetings h3 {
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
@media (min-width: 1024px) {
|
||||
.greetings h1,
|
||||
.greetings h3 {
|
||||
text-align: left;
|
||||
}
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,121 @@
|
||||
<script setup lang="ts">
|
||||
import WelcomeItem from "./WelcomeItem.vue";
|
||||
import DocumentationIcon from "./icons/IconDocumentation.vue";
|
||||
import ToolingIcon from "./icons/IconTooling.vue";
|
||||
import EcosystemIcon from "./icons/IconEcosystem.vue";
|
||||
import CommunityIcon from "./icons/IconCommunity.vue";
|
||||
import SupportIcon from "./icons/IconSupport.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<WelcomeItem>
|
||||
<template #icon>
|
||||
<DocumentationIcon />
|
||||
</template>
|
||||
<template #heading>Documentation</template>
|
||||
|
||||
Vue’s
|
||||
<a href="https://vuejs.org/" target="_blank" rel="noopener"
|
||||
>official documentation</a
|
||||
>
|
||||
provides you with all information you need to get started.
|
||||
</WelcomeItem>
|
||||
|
||||
<WelcomeItem>
|
||||
<template #icon>
|
||||
<ToolingIcon />
|
||||
</template>
|
||||
<template #heading>Tooling</template>
|
||||
|
||||
This project is served and bundled with
|
||||
<a
|
||||
href="https://vitejs.dev/guide/features.html"
|
||||
target="_blank"
|
||||
rel="noopener"
|
||||
>Vite</a
|
||||
>. The recommended IDE setup is
|
||||
<a href="https://code.visualstudio.com/" target="_blank" rel="noopener"
|
||||
>VSCode</a
|
||||
>
|
||||
+
|
||||
<a
|
||||
href="https://github.com/johnsoncodehk/volar"
|
||||
target="_blank"
|
||||
rel="noopener"
|
||||
>Volar</a
|
||||
>. If you need to test your components and web pages, check out
|
||||
<a href="https://www.cypress.io/" target="_blank" rel="noopener">Cypress</a>
|
||||
and
|
||||
<a href="https://on.cypress.io/component" target="_blank"
|
||||
>Cypress Component Testing</a
|
||||
>.
|
||||
|
||||
<br />
|
||||
|
||||
More instructions are available in <code>README.md</code>.
|
||||
</WelcomeItem>
|
||||
|
||||
<WelcomeItem>
|
||||
<template #icon>
|
||||
<EcosystemIcon />
|
||||
</template>
|
||||
<template #heading>Ecosystem</template>
|
||||
|
||||
Get official tools and libraries for your project:
|
||||
<a href="https://pinia.vuejs.org/" target="_blank" rel="noopener">Pinia</a>,
|
||||
<a href="https://router.vuejs.org/" target="_blank" rel="noopener"
|
||||
>Vue Router</a
|
||||
>,
|
||||
<a href="https://test-utils.vuejs.org/" target="_blank" rel="noopener"
|
||||
>Vue Test Utils</a
|
||||
>, and
|
||||
<a href="https://github.com/vuejs/devtools" target="_blank" rel="noopener"
|
||||
>Vue Dev Tools</a
|
||||
>. If you need more resources, we suggest paying
|
||||
<a
|
||||
href="https://github.com/vuejs/awesome-vue"
|
||||
target="_blank"
|
||||
rel="noopener"
|
||||
>Awesome Vue</a
|
||||
>
|
||||
a visit.
|
||||
</WelcomeItem>
|
||||
|
||||
<WelcomeItem>
|
||||
<template #icon>
|
||||
<CommunityIcon />
|
||||
</template>
|
||||
<template #heading>Community</template>
|
||||
|
||||
Got stuck? Ask your question on
|
||||
<a href="https://chat.vuejs.org" target="_blank" rel="noopener">Vue Land</a
|
||||
>, our official Discord server, or
|
||||
<a
|
||||
href="https://stackoverflow.com/questions/tagged/vue.js"
|
||||
target="_blank"
|
||||
rel="noopener"
|
||||
>StackOverflow</a
|
||||
>. You should also subscribe to
|
||||
<a href="https://news.vuejs.org" target="_blank" rel="noopener"
|
||||
>our mailing list</a
|
||||
>
|
||||
and follow the official
|
||||
<a href="https://twitter.com/vuejs" target="_blank" rel="noopener"
|
||||
>@vuejs</a
|
||||
>
|
||||
twitter account for latest news in the Vue world.
|
||||
</WelcomeItem>
|
||||
|
||||
<WelcomeItem>
|
||||
<template #icon>
|
||||
<SupportIcon />
|
||||
</template>
|
||||
<template #heading>Support Vue</template>
|
||||
|
||||
As an independent project, Vue relies on community backing for its
|
||||
sustainability. You can help us by
|
||||
<a href="https://vuejs.org/sponsor/" target="_blank" rel="noopener"
|
||||
>becoming a sponsor</a
|
||||
>.
|
||||
</WelcomeItem>
|
||||
</template>
|
@@ -0,0 +1,86 @@
|
||||
<template>
|
||||
<div class="item">
|
||||
<i>
|
||||
<slot name="icon"></slot>
|
||||
</i>
|
||||
<div class="details">
|
||||
<h3>
|
||||
<slot name="heading"></slot>
|
||||
</h3>
|
||||
<slot></slot>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.item {
|
||||
margin-top: 2rem;
|
||||
display: flex;
|
||||
}
|
||||
|
||||
.details {
|
||||
flex: 1;
|
||||
margin-left: 1rem;
|
||||
}
|
||||
|
||||
i {
|
||||
display: flex;
|
||||
place-items: center;
|
||||
place-content: center;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
|
||||
color: var(--color-text);
|
||||
}
|
||||
|
||||
h3 {
|
||||
font-size: 1.2rem;
|
||||
font-weight: 500;
|
||||
margin-bottom: 0.4rem;
|
||||
color: var(--color-heading);
|
||||
}
|
||||
|
||||
@media (min-width: 1024px) {
|
||||
.item {
|
||||
margin-top: 0;
|
||||
padding: 0.4rem 0 1rem calc(var(--section-gap) / 2);
|
||||
}
|
||||
|
||||
i {
|
||||
top: calc(50% - 25px);
|
||||
left: -26px;
|
||||
position: absolute;
|
||||
border: 1px solid var(--color-border);
|
||||
background: var(--color-background);
|
||||
border-radius: 8px;
|
||||
width: 50px;
|
||||
height: 50px;
|
||||
}
|
||||
|
||||
.item:before {
|
||||
content: " ";
|
||||
border-left: 1px solid var(--color-border);
|
||||
position: absolute;
|
||||
left: 0;
|
||||
bottom: calc(50% + 25px);
|
||||
height: calc(50% - 25px);
|
||||
}
|
||||
|
||||
.item:after {
|
||||
content: " ";
|
||||
border-left: 1px solid var(--color-border);
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: calc(50% + 25px);
|
||||
height: calc(50% - 25px);
|
||||
}
|
||||
|
||||
.item:first-of-type:before {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.item:last-of-type:after {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,12 @@
|
||||
<template>
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
width="20"
|
||||
height="20"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="M15 4a1 1 0 1 0 0 2V4zm0 11v-1a1 1 0 0 0-1 1h1zm0 4l-.707.707A1 1 0 0 0 16 19h-1zm-4-4l.707-.707A1 1 0 0 0 11 14v1zm-4.707-1.293a1 1 0 0 0-1.414 1.414l1.414-1.414zm-.707.707l-.707-.707.707.707zM9 11v-1a1 1 0 0 0-.707.293L9 11zm-4 0h1a1 1 0 0 0-1-1v1zm0 4H4a1 1 0 0 0 1.707.707L5 15zm10-9h2V4h-2v2zm2 0a1 1 0 0 1 1 1h2a3 3 0 0 0-3-3v2zm1 1v6h2V7h-2zm0 6a1 1 0 0 1-1 1v2a3 3 0 0 0 3-3h-2zm-1 1h-2v2h2v-2zm-3 1v4h2v-4h-2zm1.707 3.293l-4-4-1.414 1.414 4 4 1.414-1.414zM11 14H7v2h4v-2zm-4 0c-.276 0-.525-.111-.707-.293l-1.414 1.414C5.42 15.663 6.172 16 7 16v-2zm-.707 1.121l3.414-3.414-1.414-1.414-3.414 3.414 1.414 1.414zM9 12h4v-2H9v2zm4 0a3 3 0 0 0 3-3h-2a1 1 0 0 1-1 1v2zm3-3V3h-2v6h2zm0-6a3 3 0 0 0-3-3v2a1 1 0 0 1 1 1h2zm-3-3H3v2h10V0zM3 0a3 3 0 0 0-3 3h2a1 1 0 0 1 1-1V0zM0 3v6h2V3H0zm0 6a3 3 0 0 0 3 3v-2a1 1 0 0 1-1-1H0zm3 3h2v-2H3v2zm1-1v4h2v-4H4zm1.707 4.707l.586-.586-1.414-1.414-.586.586 1.414 1.414z"
|
||||
/>
|
||||
</svg>
|
||||
</template>
|
@@ -0,0 +1,12 @@
|
||||
<template>
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
width="20"
|
||||
height="17"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="M11 2.253a1 1 0 1 0-2 0h2zm-2 13a1 1 0 1 0 2 0H9zm.447-12.167a1 1 0 1 0 1.107-1.666L9.447 3.086zM1 2.253L.447 1.42A1 1 0 0 0 0 2.253h1zm0 13H0a1 1 0 0 0 1.553.833L1 15.253zm8.447.833a1 1 0 1 0 1.107-1.666l-1.107 1.666zm0-14.666a1 1 0 1 0 1.107 1.666L9.447 1.42zM19 2.253h1a1 1 0 0 0-.447-.833L19 2.253zm0 13l-.553.833A1 1 0 0 0 20 15.253h-1zm-9.553-.833a1 1 0 1 0 1.107 1.666L9.447 14.42zM9 2.253v13h2v-13H9zm1.553-.833C9.203.523 7.42 0 5.5 0v2c1.572 0 2.961.431 3.947 1.086l1.107-1.666zM5.5 0C3.58 0 1.797.523.447 1.42l1.107 1.666C2.539 2.431 3.928 2 5.5 2V0zM0 2.253v13h2v-13H0zm1.553 13.833C2.539 15.431 3.928 15 5.5 15v-2c-1.92 0-3.703.523-5.053 1.42l1.107 1.666zM5.5 15c1.572 0 2.961.431 3.947 1.086l1.107-1.666C9.203 13.523 7.42 13 5.5 13v2zm5.053-11.914C11.539 2.431 12.928 2 14.5 2V0c-1.92 0-3.703.523-5.053 1.42l1.107 1.666zM14.5 2c1.573 0 2.961.431 3.947 1.086l1.107-1.666C18.203.523 16.421 0 14.5 0v2zm3.5.253v13h2v-13h-2zm1.553 12.167C18.203 13.523 16.421 13 14.5 13v2c1.573 0 2.961.431 3.947 1.086l1.107-1.666zM14.5 13c-1.92 0-3.703.523-5.053 1.42l1.107 1.666C11.539 15.431 12.928 15 14.5 15v-2z"
|
||||
/>
|
||||
</svg>
|
||||
</template>
|
@@ -0,0 +1,12 @@
|
||||
<template>
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
width="18"
|
||||
height="20"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="M11.447 8.894a1 1 0 1 0-.894-1.789l.894 1.789zm-2.894-.789a1 1 0 1 0 .894 1.789l-.894-1.789zm0 1.789a1 1 0 1 0 .894-1.789l-.894 1.789zM7.447 7.106a1 1 0 1 0-.894 1.789l.894-1.789zM10 9a1 1 0 1 0-2 0h2zm-2 2.5a1 1 0 1 0 2 0H8zm9.447-5.606a1 1 0 1 0-.894-1.789l.894 1.789zm-2.894-.789a1 1 0 1 0 .894 1.789l-.894-1.789zm2 .789a1 1 0 1 0 .894-1.789l-.894 1.789zm-1.106-2.789a1 1 0 1 0-.894 1.789l.894-1.789zM18 5a1 1 0 1 0-2 0h2zm-2 2.5a1 1 0 1 0 2 0h-2zm-5.447-4.606a1 1 0 1 0 .894-1.789l-.894 1.789zM9 1l.447-.894a1 1 0 0 0-.894 0L9 1zm-2.447.106a1 1 0 1 0 .894 1.789l-.894-1.789zm-6 3a1 1 0 1 0 .894 1.789L.553 4.106zm2.894.789a1 1 0 1 0-.894-1.789l.894 1.789zm-2-.789a1 1 0 1 0-.894 1.789l.894-1.789zm1.106 2.789a1 1 0 1 0 .894-1.789l-.894 1.789zM2 5a1 1 0 1 0-2 0h2zM0 7.5a1 1 0 1 0 2 0H0zm8.553 12.394a1 1 0 1 0 .894-1.789l-.894 1.789zm-1.106-2.789a1 1 0 1 0-.894 1.789l.894-1.789zm1.106 1a1 1 0 1 0 .894 1.789l-.894-1.789zm2.894.789a1 1 0 1 0-.894-1.789l.894 1.789zM8 19a1 1 0 1 0 2 0H8zm2-2.5a1 1 0 1 0-2 0h2zm-7.447.394a1 1 0 1 0 .894-1.789l-.894 1.789zM1 15H0a1 1 0 0 0 .553.894L1 15zm1-2.5a1 1 0 1 0-2 0h2zm12.553 2.606a1 1 0 1 0 .894 1.789l-.894-1.789zM17 15l.447.894A1 1 0 0 0 18 15h-1zm1-2.5a1 1 0 1 0-2 0h2zm-7.447-5.394l-2 1 .894 1.789 2-1-.894-1.789zm-1.106 1l-2-1-.894 1.789 2 1 .894-1.789zM8 9v2.5h2V9H8zm8.553-4.894l-2 1 .894 1.789 2-1-.894-1.789zm.894 0l-2-1-.894 1.789 2 1 .894-1.789zM16 5v2.5h2V5h-2zm-4.553-3.894l-2-1-.894 1.789 2 1 .894-1.789zm-2.894-1l-2 1 .894 1.789 2-1L8.553.106zM1.447 5.894l2-1-.894-1.789-2 1 .894 1.789zm-.894 0l2 1 .894-1.789-2-1-.894 1.789zM0 5v2.5h2V5H0zm9.447 13.106l-2-1-.894 1.789 2 1 .894-1.789zm0 1.789l2-1-.894-1.789-2 1 .894 1.789zM10 19v-2.5H8V19h2zm-6.553-3.894l-2-1-.894 1.789 2 1 .894-1.789zM2 15v-2.5H0V15h2zm13.447 1.894l2-1-.894-1.789-2 1 .894 1.789zM18 15v-2.5h-2V15h2z"
|
||||
/>
|
||||
</svg>
|
||||
</template>
|
@@ -0,0 +1,12 @@
|
||||
<template>
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
width="20"
|
||||
height="20"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="M10 3.22l-.61-.6a5.5 5.5 0 0 0-7.666.105 5.5 5.5 0 0 0-.114 7.665L10 18.78l8.39-8.4a5.5 5.5 0 0 0-.114-7.665 5.5 5.5 0 0 0-7.666-.105l-.61.61z"
|
||||
/>
|
||||
</svg>
|
||||
</template>
|
@@ -0,0 +1,19 @@
|
||||
<!-- This icon is from <https://github.com/Templarian/MaterialDesign>, distributed under Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0) license-->
|
||||
<template>
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
aria-hidden="true"
|
||||
role="img"
|
||||
class="iconify iconify--mdi"
|
||||
width="24"
|
||||
height="24"
|
||||
preserveAspectRatio="xMidYMid meet"
|
||||
viewBox="0 0 24 24"
|
||||
>
|
||||
<path
|
||||
d="M20 18v-4h-3v1h-2v-1H9v1H7v-1H4v4h16M6.33 8l-1.74 4H7v-1h2v1h6v-1h2v1h2.41l-1.74-4H6.33M9 5v1h6V5H9m12.84 7.61c.1.22.16.48.16.8V18c0 .53-.21 1-.6 1.41c-.4.4-.85.59-1.4.59H4c-.55 0-1-.19-1.4-.59C2.21 19 2 18.53 2 18v-4.59c0-.32.06-.58.16-.8L4.5 7.22C4.84 6.41 5.45 6 6.33 6H7V5c0-.55.18-1 .57-1.41C7.96 3.2 8.44 3 9 3h6c.56 0 1.04.2 1.43.59c.39.41.57.86.57 1.41v1h.67c.88 0 1.49.41 1.83 1.22l2.34 5.39z"
|
||||
fill="currentColor"
|
||||
></path>
|
||||
</svg>
|
||||
</template>
|
@@ -0,0 +1,15 @@
|
||||
<template>
|
||||
<div class="about">
|
||||
<h1>This is an about page</h1>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style>
|
||||
@media (min-width: 1024px) {
|
||||
.about {
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
}
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,7 @@
|
||||
<template>
|
||||
<div></div>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts"></script>
|
||||
|
||||
<style scoped lang="less"></style>
|
@@ -0,0 +1,9 @@
|
||||
<script setup lang="ts">
|
||||
import TheWelcome from "../components/TheWelcome.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<main>
|
||||
<TheWelcome />
|
||||
</main>
|
||||
</template>
|
5
examples/application/js/web_demo/demo/src/pages/auto-imports.d.ts
vendored
Normal file
5
examples/application/js/web_demo/demo/src/pages/auto-imports.d.ts
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
// Generated by 'unplugin-auto-import'
|
||||
export {}
|
||||
declare global {
|
||||
|
||||
}
|
27
examples/application/js/web_demo/demo/src/pages/components.d.ts
vendored
Normal file
27
examples/application/js/web_demo/demo/src/pages/components.d.ts
vendored
Normal file
@@ -0,0 +1,27 @@
|
||||
// generated by unplugin-vue-components
|
||||
// We suggest you to commit this file into source control
|
||||
// Read more: https://github.com/vuejs/core/pull/3399
|
||||
import '@vue/runtime-core'
|
||||
|
||||
export {}
|
||||
|
||||
declare module '@vue/runtime-core' {
|
||||
export interface GlobalComponents {
|
||||
ElAside: typeof import('element-plus/es')['ElAside']
|
||||
ElButton: typeof import('element-plus/es')['ElButton']
|
||||
ElCard: typeof import('element-plus/es')['ElCard']
|
||||
ElCol: typeof import('element-plus/es')['ElCol']
|
||||
ElContainer: typeof import('element-plus/es')['ElContainer']
|
||||
ElDialog: typeof import('element-plus/es')['ElDialog']
|
||||
ElFooter: typeof import('element-plus/es')['ElFooter']
|
||||
ElHeader: typeof import('element-plus/es')['ElHeader']
|
||||
ElInput: typeof import('element-plus/es')['ElInput']
|
||||
ElMain: typeof import('element-plus/es')['ElMain']
|
||||
ElMenu: typeof import('element-plus/es')['ElMenu']
|
||||
ElMenuItem: typeof import('element-plus/es')['ElMenuItem']
|
||||
ElRow: typeof import('element-plus/es')['ElRow']
|
||||
ElSubMenu: typeof import('element-plus/es')['ElSubMenu']
|
||||
RouterLink: typeof import('vue-router')['RouterLink']
|
||||
RouterView: typeof import('vue-router')['RouterView']
|
||||
}
|
||||
}
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import FaceDetection from "@/pages/cv/detection/FaceDetection/FaceDetection.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<FaceDetection></FaceDetection>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,142 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>上传人脸图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>人脸区域检测</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始检测</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="canvas" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import { FaceDetector } from "@paddle-js-models/facedetect";
|
||||
import { onMounted, ref } from "vue";
|
||||
|
||||
interface TransformData {
|
||||
left: number;
|
||||
width: number;
|
||||
top: number;
|
||||
height: number;
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
const faceDet = new FaceDetector();
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
await faceDet.init();
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await faceDet.detect(img);
|
||||
console.log(res);
|
||||
drawBox(res);
|
||||
};
|
||||
|
||||
const drawBox = (data: TransformData[]) => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const imgHeight = img.height;
|
||||
const imgWidth = img.width;
|
||||
canvas.value.width = imgWidth;
|
||||
canvas.value.height = imgHeight;
|
||||
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
|
||||
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
|
||||
data.forEach((item: TransformData) => {
|
||||
// 开始一个新的绘制路径
|
||||
ctx.beginPath();
|
||||
// 设置线条颜色为蓝色
|
||||
ctx.strokeStyle = "red";
|
||||
// 获取检测框选坐标
|
||||
const x = item.left * imgWidth;
|
||||
const y = item.top * imgHeight;
|
||||
const w = item.width * imgWidth;
|
||||
const h = item.height * imgHeight;
|
||||
ctx.beginPath();
|
||||
// 绘制检测框选矩形
|
||||
ctx.rect(x, y, w, h);
|
||||
// 绘制label
|
||||
ctx.fillText(item.confidence.toFixed(6), x, y);
|
||||
ctx.stroke();
|
||||
});
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import ScrewDetection from "@/pages/cv/detection/ScrewDetection/ScrewDetection.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<ScrewDetection></ScrewDetection>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,138 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>上传图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>螺丝螺母区域检测</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始检测</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="canvas" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as det from "@paddle-js-models/detect";
|
||||
import { onMounted, ref } from "vue";
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
const label = new Map([
|
||||
[0, "default"],
|
||||
[1, "screw"],
|
||||
[2, "nut"],
|
||||
]);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
await det.init();
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await det.detect(img);
|
||||
console.log(res);
|
||||
drawBox(res);
|
||||
};
|
||||
|
||||
const drawBox = (points: number[][]) => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const imgHeight = img.height;
|
||||
const imgWidth = img.width;
|
||||
canvas.value.width = imgWidth;
|
||||
canvas.value.height = imgHeight;
|
||||
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
|
||||
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
|
||||
points.forEach((point: number[]) => {
|
||||
// 开始一个新的绘制路径
|
||||
ctx.beginPath();
|
||||
// 设置线条颜色为蓝色
|
||||
ctx.strokeStyle = "red";
|
||||
// 获取检测框选坐标
|
||||
const left = Math.floor(point[2] * imgWidth);
|
||||
const top = Math.floor(point[3] * imgHeight);
|
||||
const right = Math.floor(point[4] * imgWidth);
|
||||
const bottom = Math.floor(point[5] * imgHeight);
|
||||
ctx.beginPath();
|
||||
// 绘制检测框选矩形
|
||||
ctx.rect(left, top, right - left, bottom - top);
|
||||
// 绘制label
|
||||
ctx.fillText(label.get(point[0])!, left + 10, top + 10);
|
||||
ctx.stroke();
|
||||
});
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import TextDetection from "@/pages/cv/ocr/TextDetection/TextDetection.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<TextDetection></TextDetection>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,126 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>上传文本图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="12">
|
||||
<el-row class="small-title">
|
||||
<h2>文字区域检测</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始检测</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="canvas" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as ocrDet from "@paddle-js-models/ocrdet";
|
||||
import { onMounted, ref } from "vue";
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
await ocrDet.load("https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json");
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await ocrDet.detect(img);
|
||||
console.log(res);
|
||||
drawBox(res);
|
||||
};
|
||||
|
||||
const drawBox = (points: number[][][]) => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
canvas.value.width = img.naturalWidth;
|
||||
canvas.value.height = img.naturalHeight;
|
||||
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
|
||||
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
|
||||
points.forEach((point) => {
|
||||
// 开始一个新的绘制路径
|
||||
ctx.beginPath();
|
||||
// 设置线条颜色为蓝色
|
||||
ctx.strokeStyle = "blue";
|
||||
// 设置路径起点坐标
|
||||
ctx.moveTo(point[0][0], point[0][1]);
|
||||
ctx.lineTo(point[1][0], point[1][1]);
|
||||
ctx.lineTo(point[2][0], point[2][1]);
|
||||
ctx.lineTo(point[3][0], point[3][1]);
|
||||
ctx.closePath();
|
||||
ctx.stroke();
|
||||
});
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import TextRecognition from "@/pages/cv/ocr/TextRecognition/TextRecognition.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<TextRecognition></TextRecognition>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,121 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>上传文本图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>文字区域检测</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始识别</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="canvas" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>识别结果展示</h2>
|
||||
</el-row>
|
||||
<span v-html="result"></span>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as ocr from "@paddle-js-models/ocr";
|
||||
import { onMounted, onUnmounted, ref } from "vue";
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const result = ref("");
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
|
||||
await ocr.init("https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json", "https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_rec_infer_js/model.json");
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
onUnmounted(() => {
|
||||
console.log("delete rec");
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await ocr.recognize(img, { canvas: canvas.value });
|
||||
console.log(res);
|
||||
if (res.text?.length) {
|
||||
// 页面展示识别内容
|
||||
result.value = res.text.reduce((total, cur) => total + `<p>${cur}</p>`);
|
||||
}
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import GestureRecognition from "@/pages/cv/recognition/GestureRecognition/GestureRecognition.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<GestureRecognition></GestureRecognition>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,168 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>上传手势图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>手势检测</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始识别</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="canvas" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>手势识别结果</h2>
|
||||
</el-row>
|
||||
<span v-html="result"></span>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as gestureRec from "@paddle-js-models/gesture";
|
||||
import { onMounted, onUnmounted, ref } from "vue";
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const result = ref("");
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
|
||||
await gestureRec.load();
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
onUnmounted(() => {
|
||||
console.log("delete rec");
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await gestureRec.classify(img);
|
||||
console.log(res);
|
||||
if (res.box && res.box.length) {
|
||||
result.value = res.type;
|
||||
drawBox(res.box as number[][]);
|
||||
} else {
|
||||
result.value = "未识别到手";
|
||||
}
|
||||
};
|
||||
|
||||
const drawBox = (box: number[][]) => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
canvas.value.width = img.naturalWidth;
|
||||
canvas.value.height = img.naturalHeight;
|
||||
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
|
||||
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
|
||||
|
||||
/**
|
||||
* 计算缩放比率,得到实际绘制框的坐标
|
||||
*/
|
||||
let offsetX = 0;
|
||||
let offsetY = 0;
|
||||
|
||||
if (img.width < img.height) {
|
||||
offsetX = img.height - img.width;
|
||||
}
|
||||
|
||||
if (img.width > img.height) {
|
||||
offsetY = img.width - img.height;
|
||||
}
|
||||
|
||||
const widthRatio = (img.width + offsetX) / 256;
|
||||
const heightRatio = (img.height + offsetY) / 256;
|
||||
const points: number[][] = [];
|
||||
|
||||
box.forEach((item) => {
|
||||
const tmpPonit = [];
|
||||
tmpPonit[0] = item[0] * widthRatio - offsetX / 2;
|
||||
tmpPonit[1] = item[1] * heightRatio - offsetY / 2;
|
||||
points.push(tmpPonit);
|
||||
});
|
||||
|
||||
// 开始一个新的绘制路径
|
||||
ctx.beginPath();
|
||||
// 设置线条颜色为蓝色
|
||||
ctx.strokeStyle = "blue";
|
||||
// 设置路径起点坐标
|
||||
ctx.moveTo(points[0][0], points[0][1]);
|
||||
ctx.lineTo(points[1][0], points[1][1]);
|
||||
ctx.lineTo(points[2][0], points[2][1]);
|
||||
ctx.lineTo(points[3][0], points[3][1]);
|
||||
ctx.closePath();
|
||||
ctx.stroke();
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import ItemIdentification from "@/pages/cv/recognition/ItemIdentification/ItemIdentification.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<ItemIdentification></ItemIdentification>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,115 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="16">
|
||||
<el-row class="small-title">
|
||||
<h2>上传图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>预测结果</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="predict">开始检测</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
{{ itemName }}
|
||||
</el-row>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as mobilenet from "@paddle-js-models/mobilenet";
|
||||
import { onMounted, ref } from "vue";
|
||||
import map from "./map.json";
|
||||
|
||||
const fileName = ref(null);
|
||||
|
||||
const canvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
|
||||
const itemName = ref(null);
|
||||
|
||||
onMounted(async () => {
|
||||
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
|
||||
await mobilenet.load(
|
||||
{
|
||||
path: "https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/mobileNetV2_fuse_activation/model.json",
|
||||
mean: [0.485, 0.456, 0.406],
|
||||
std: [0.229, 0.224, 0.225],
|
||||
},
|
||||
map
|
||||
);
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const predict = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await mobilenet.classify(img);
|
||||
console.log(res);
|
||||
itemName.value = res;
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
@@ -0,0 +1,9 @@
|
||||
import { createApp } from "vue";
|
||||
|
||||
import App from "./App.vue";
|
||||
|
||||
import "../../../../assets/main.css";
|
||||
|
||||
const app = createApp(App);
|
||||
|
||||
app.mount("#app");
|
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,7 @@
|
||||
<template>
|
||||
<div></div>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts"></script>
|
||||
|
||||
<style scoped lang="less"></style>
|
@@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import HumanSeg from "@/pages/cv/segmentation/HumanSeg/HumanSeg.vue";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-container>
|
||||
<el-header>Header</el-header>
|
||||
<el-main>
|
||||
<HumanSeg></HumanSeg>
|
||||
</el-main>
|
||||
<el-footer>Footer</el-footer>
|
||||
</el-container>
|
||||
</template>
|
||||
|
||||
<style scoped lang="less">
|
||||
.el-container {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,229 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
v-model="isLoadingModel"
|
||||
title="提示"
|
||||
width="30%"
|
||||
center
|
||||
:lock-scroll="true"
|
||||
:show-close="false"
|
||||
:close-on-click-modal="false"
|
||||
:close-on-press-escape="false"
|
||||
>
|
||||
<span>正在加载模型,请稍等。</span>
|
||||
</el-dialog>
|
||||
<el-row :gutter="20">
|
||||
<el-col :span="8">
|
||||
<el-row class="small-title">
|
||||
<h2>上传图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="show-img" class="show-area" />
|
||||
<!-- 用于存放真实图片进行文字识别 -->
|
||||
<img id="raw-img" style="display: none" />
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="4">
|
||||
<el-row class="small-title">
|
||||
<h2>上传替换背景图片</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-input
|
||||
type="file"
|
||||
v-model="newBackgroundImgFileName"
|
||||
@change="uploadNewBackgroundImg"
|
||||
></el-input>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<!-- 用于展示图片 -->
|
||||
<img id="new-background-img" class="show-area" />
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="background" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="4">
|
||||
<el-row class="small-title">
|
||||
<h2>背景替换</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="backgroundReplace">
|
||||
替换背景
|
||||
</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="replace-background" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="4">
|
||||
<el-row class="small-title">
|
||||
<h2>背景虚化</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="blurBackground">背景虚化</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="blur" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
<el-col :span="4">
|
||||
<el-row class="small-title">
|
||||
<h2>人形遮罩</h2>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<el-button type="primary" @click="drawHumanoidMask">
|
||||
绘制人形遮罩
|
||||
</el-button>
|
||||
</el-row>
|
||||
<el-row>
|
||||
<canvas id="mask" class="show-area"></canvas>
|
||||
</el-row>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</template>
|
||||
|
||||
<script setup lang="ts">
|
||||
import * as humanSeg from "@paddle-js-models/humanseg";
|
||||
import { onMounted, ref } from "vue";
|
||||
|
||||
const fileName = ref(null);
|
||||
const newBackgroundImgFileName = ref(null);
|
||||
|
||||
const backgroundCanvas = ref(null as unknown as HTMLCanvasElement);
|
||||
const replaceBackgroundCanvas = ref(null as unknown as HTMLCanvasElement);
|
||||
const blurCanvas = ref(null as unknown as HTMLCanvasElement);
|
||||
const maskCanvas = ref(null as unknown as HTMLCanvasElement);
|
||||
|
||||
const isLoadingModel = ref(true);
|
||||
const modelPredictDone = ref(false);
|
||||
|
||||
/**
|
||||
* 存储模型分割后的像素 alpha 值
|
||||
*/
|
||||
let garyData: number[];
|
||||
|
||||
onMounted(async () => {
|
||||
backgroundCanvas.value = document.getElementById(
|
||||
"background"
|
||||
) as HTMLCanvasElement;
|
||||
replaceBackgroundCanvas.value = document.getElementById(
|
||||
"replace-background"
|
||||
) as HTMLCanvasElement;
|
||||
blurCanvas.value = document.getElementById("blur") as HTMLCanvasElement;
|
||||
maskCanvas.value = document.getElementById("mask") as HTMLCanvasElement;
|
||||
await humanSeg.load();
|
||||
isLoadingModel.value = false;
|
||||
});
|
||||
|
||||
const uploadImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById("show-img") as HTMLImageElement;
|
||||
// 用于识别
|
||||
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[0]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
rawImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const uploadNewBackgroundImg = () => {
|
||||
/**
|
||||
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
|
||||
*/
|
||||
const reader = new FileReader();
|
||||
// 用于展示
|
||||
const showImg = document.getElementById(
|
||||
"new-background-img"
|
||||
) as HTMLImageElement;
|
||||
// 获取背景图片的 input
|
||||
const inputElement = document
|
||||
.getElementsByClassName("el-input")[1]
|
||||
.getElementsByTagName("input")[0];
|
||||
|
||||
try {
|
||||
const file = inputElement.files![0];
|
||||
reader.onload = () => {
|
||||
showImg.src = URL.createObjectURL(file);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
};
|
||||
|
||||
const backgroundReplace = async () => {
|
||||
if (!modelPredictDone.value) {
|
||||
await seg();
|
||||
modelPredictDone.value = true;
|
||||
}
|
||||
|
||||
const showImg = document.getElementById(
|
||||
"new-background-img"
|
||||
) as HTMLImageElement;
|
||||
backgroundCanvas.value.width = showImg.naturalWidth;
|
||||
backgroundCanvas.value.height = showImg.naturalHeight;
|
||||
backgroundCanvas.value
|
||||
.getContext("2d")!
|
||||
.drawImage(showImg, 0, 0, showImg.naturalWidth, showImg.naturalHeight);
|
||||
humanSeg.drawHumanSeg(
|
||||
garyData,
|
||||
replaceBackgroundCanvas.value,
|
||||
backgroundCanvas.value
|
||||
);
|
||||
};
|
||||
|
||||
const blurBackground = async () => {
|
||||
if (!modelPredictDone.value) {
|
||||
await seg();
|
||||
modelPredictDone.value = true;
|
||||
}
|
||||
humanSeg.blurBackground(garyData, blurCanvas.value);
|
||||
};
|
||||
|
||||
const drawHumanoidMask = async () => {
|
||||
if (!modelPredictDone.value) {
|
||||
await seg();
|
||||
modelPredictDone.value = true;
|
||||
}
|
||||
humanSeg.drawMask(garyData, maskCanvas.value, backgroundCanvas.value);
|
||||
};
|
||||
|
||||
const seg = async () => {
|
||||
const img = document.getElementById("raw-img") as HTMLImageElement;
|
||||
const res = await humanSeg.getGrayValue(img);
|
||||
console.log(res);
|
||||
garyData = res.data;
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped lang="less">
|
||||
.small-title {
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.show-area {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.el-row {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<script>var Module;</script>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/favicon.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Vite App</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="main.ts"></script>
|
||||
</body>
|
||||
</html>
|
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user