[Model] add Paddle.js web demo (#392)

* add application include paddle.js web demo and xcx

* cp PR #5

* add readme

* fix comments and link

* fix xcx readme

* fix Task 1

* fix bugs

* refine readme

* delete ocrxcx readme

* refine readme

* fix bugs

* delete old readme

* 200px to 300px

* revert 200px to 300px

Co-authored-by: Jason <jiangjiajun@baidu.com>
This commit is contained in:
Double_V
2022-10-20 17:01:38 +08:00
committed by GitHub
parent 587ffd4caf
commit 0a22979c35
302 changed files with 63930 additions and 0 deletions

View File

@@ -0,0 +1,35 @@
# 前端AI应用
人工智能技术的快速发展带动了计算机视觉、自然语言处理领域的产业升级。另外随着PC和移动设备上算力的稳步增强、模型压缩技术迭代更新以及各种创新需求的不断催生在浏览器中部署AI模型实现前端智能已经具备了良好的基础条件。
针对前端部署AI深度学习模型困难的问题百度开源了Paddle.js前端深度学习模型部署框架可以很容易的将深度学习模型部署到前端项目中。
## Paddle.js简介
[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)是百度`PaddlePaddle`的web方向子项目是一个运行在浏览器中的开源深度学习框架。`Paddle.js`可以加载`PaddlePaddle`动转静的模型,经过`Paddle.js`的模型转换工具`paddlejs-converter`转换成浏览器友好的模型,易于在线推理预测使用。`Paddle.js`支持`WebGL/WebGPU/WebAssembly`的浏览器中运行,也可以在百度小程序和微信小程序环境下运行。
简言之利用Paddle.js我们可以在浏览器、小程序等前端应用场景上线AI功能包括但不限于目标检测图像分割OCR物品分类等AI能力。
## Web Demo使用
在浏览器中直接运行官方demo参考[文档](./web_demo/README.md)
|demo名称|web demo目录|可视化|
|-|-|-|
|目标检测|[ScrewDetection/FaceDetection](./web_demo/demo/src/pages/cv/detection/)| <img src="https://user-images.githubusercontent.com/26592129/196874536-b7fa2c0a-d71f-4271-8c40-f9088bfad3c9.png" height="200px">|
|人像分割背景替换|[HumanSeg](./web_demo//demo/src/pages/cv/segmentation/HumanSeg)|<img src="https://user-images.githubusercontent.com/26592129/196874452-4ef2e770-fbb3-4a35-954b-f871716d6669.png" height="200px">|
|物体识别|[GestureRecognition/ItemIdentification](./web_demo//demo/src/pages/cv/recognition/)|<img src="https://user-images.githubusercontent.com/26592129/196874416-454e6bb0-4ebd-4b51-a88a-8c40614290ae.png" height="200px">|
|OCR|[TextDetection/TextRecognition](./web_demo//demo/src/pages/cv/ocr/)|<img src="https://user-images.githubusercontent.com/26592129/196874354-1b5eecb0-f273-403c-aa6c-4463bf6d78db.png" height="200px">|
## 微信小程序Demo使用
在微信小程序运行官方demo参考[文档](./mini_program/README.md)
|名称|目录|
|-|-|
|OCR文本检测| [ocrdetecXcx](./mini_program/ocrdetectXcx/) |
|OCR文本识别| [ocrXcx](./mini_program/ocrXcx/) |
|目标检测| coming soon |
|图像分割| coming soon |
|物品分类| coming soon |

View File

@@ -0,0 +1,126 @@
# Paddle.js微信小程序Demo
- [1.简介](#1)
- [2. 项目启动](#2)
* [2.1 准备工作](#21)
* [2.2 启动步骤](#22)
* [2.3 效果展示](#23)
- [3. 模型推理pipeline](#3)
- [4. 常见问题](#4)
<a name="1"></a>
## 1.简介
本目录下包含文本检测、文本识别小程序demo通过使用 [Paddle.js](https://github.com/PaddlePaddle/Paddle.js) 以及 [Paddle.js微信小程序插件](https://mp.weixin.qq.com/wxopen/plugindevdoc?appid=wx7138a7bb793608c3&token=956931339&lang=zh_CN) 完成在小程序上利用用户终端算力实现文本检测框选效果。
<a name="2"></a>
## 2. 项目启动
<a name="21"></a>
### 2.1 准备工作
* [申请微信小程序账号](https://mp.weixin.qq.com/)
* [微信小程序开发者工具](https://developers.weixin.qq.com/miniprogram/dev/devtools/download.html)
* 前端开发环境准备node、npm
* 小程序管理后台配置服务器域名,或打开开发者工具【不校验合法域名】
详情参考:https://mp.weixin.qq.com/wxamp/devprofile/get_profile?token=1132303404&lang=zh_CN)
<a name="22"></a>
### 2.2 启动步骤
#### **1. 克隆Demo代码**
```sh
git clone https://github.com/PaddlePaddle/FastDeploy
cd FastDeploy/examples/application/js/mini_program
```
#### **2. 进入小程序目录,安装依赖**
```sh
# 运行文本识别demo进入到ocrXcx目录
cd ./ocrXcx && npm install
# 运行文本检测demo进入到ocrdetectXcx目录
# cd ./ocrdetectXcx && npm install
```
#### **3. 微信小程序导入代码**
打开微信开发者工具 --> 导入 --> 选定目录,输入相关信息
#### **4. 添加 Paddle.js微信小程序插件**
小程序管理界面 --> 设置 --> 第三方设置 --> 插件管理 --> 添加插件 --> 搜索 `wx7138a7bb793608c3` 并添加
[参考文档](https://developers.weixin.qq.com/miniprogram/dev/framework/plugin/using.html)
#### **5. 构建依赖**
点击开发者工具中的菜单栏:工具 --> 构建 npm
原因node_modules 目录不会参与编译、上传和打包中,小程序想要使用 npm 包必须走一遍“构建 npm”的过程构建完成会生成一个 miniprogram_npm 目录,里面会存放构建打包后的 npm 包,也就是小程序真正使用的 npm 包。*
[参考文档](https://developers.weixin.qq.com/miniprogram/dev/devtools/npm.html)
<a name="23"></a>
### 2.3 效果展示
<img src="https://user-images.githubusercontent.com/43414102/157648579-cdbbee61-9866-4364-9edd-a97ac0eda0c1.png" width="300px">
<a name="3"></a>
## 3. 模型推理pipeline
```typescript
// 引入 paddlejs 和 paddlejs-plugin注册小程序环境变量和合适的 backend
import * as paddlejs from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
const plugin = requirePlugin('paddlejs-plugin');
plugin.register(paddlejs, wx);
// 初始化推理引擎
const runner = new paddlejs.Runner({modelPath, feedShape, mean, std});
await runner.init();
// 获取图像信息
wx.canvasGetImageData({
canvasId: canvasId,
x: 0,
y: 0,
width: canvas.width,
height: canvas.height,
success(res) {
// 推理预测
runner.predict({
data: res.data,
width: canvas.width,
height: canvas.height,
}, function (data) {
// 获取推理结果
console.log(data)
});
}
});
```
<a name="4"></a>
## 4. 常见问题
### 4.1 出现报错 `Invalid context type [webgl2] for Canvas#getContext`
可以不管不影响正常代码运行和demo功能
### 4.2 预览看不到结果
建议尝试真机调试
### 4.3 微信开发者工具出现黑屏,然后出现超多报错
重启微信开发者工具
### 4.4 模拟和真机调试结果不一致;模拟检测不到文本等
可以以真机为准;
模拟检测不到文本等可以尝试随意改动下代码(增删换行等)再点击编译
### 4.5 手机调试或运行时出现 长时间无反应等提示
请继续等待,模型推理需要一定时间

View File

@@ -0,0 +1,12 @@
/* global wx, App */
import * as paddlejs from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
// eslint-disable-next-line no-undef
const plugin = requirePlugin('paddlejs-plugin');
plugin.register(paddlejs, wx);
App({
globalData: {
Paddlejs: paddlejs.Runner
}
});

View File

@@ -0,0 +1,12 @@
{
"pages": [
"pages/index/index"
],
"plugins": {
"paddlejs-plugin": {
"version": "2.0.1",
"provider": "wx7138a7bb793608c3"
}
},
"sitemapLocation": "sitemap.json"
}

View File

@@ -0,0 +1,72 @@
{
"name": "paddlejs-demo",
"version": "0.0.1",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "paddlejs-demo",
"version": "0.0.1",
"license": "ISC",
"dependencies": {
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
"@paddlejs/paddlejs-core": "^2.1.18",
"d3-polygon": "2.0.0",
"js-clipper": "1.0.1",
"number-precision": "1.5.2"
}
},
"node_modules/@paddlejs/paddlejs-backend-webgl": {
"version": "1.2.9",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.9.tgz",
"integrity": "sha512-cVDa0/Wbw2EyfsYqdYUPhFeqKsET79keEUWjyhYQmQkJfWg8j1qdR6yp7g6nx9qAGrqFvwuj1s0EqkYA1dok6A=="
},
"node_modules/@paddlejs/paddlejs-core": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.2.0.tgz",
"integrity": "sha512-P3rPkF9fFHtq8uSte5gA7fJQwBNl9Ytsvj6aTcfQSsirnBO/HxMNu0gJyh7+lItvEtF92PR15eI0eOwJYfZDhQ=="
},
"node_modules/d3-polygon": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
},
"node_modules/js-clipper": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
"integrity": "sha512-0XYAS0ZoCki5K0fWwj8j8ug4mgxHXReW3ayPbVqr4zXPJuIs2pyvemL1sALadsEiAywZwW5Ify1XfU4bNJvokg=="
},
"node_modules/number-precision": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
}
},
"dependencies": {
"@paddlejs/paddlejs-backend-webgl": {
"version": "1.2.9",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.9.tgz",
"integrity": "sha512-cVDa0/Wbw2EyfsYqdYUPhFeqKsET79keEUWjyhYQmQkJfWg8j1qdR6yp7g6nx9qAGrqFvwuj1s0EqkYA1dok6A=="
},
"@paddlejs/paddlejs-core": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.2.0.tgz",
"integrity": "sha512-P3rPkF9fFHtq8uSte5gA7fJQwBNl9Ytsvj6aTcfQSsirnBO/HxMNu0gJyh7+lItvEtF92PR15eI0eOwJYfZDhQ=="
},
"d3-polygon": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
},
"js-clipper": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
"integrity": "sha512-0XYAS0ZoCki5K0fWwj8j8ug4mgxHXReW3ayPbVqr4zXPJuIs2pyvemL1sALadsEiAywZwW5Ify1XfU4bNJvokg=="
},
"number-precision": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
}
}
}

View File

@@ -0,0 +1,19 @@
{
"name": "paddlejs-demo",
"version": "0.0.1",
"description": "",
"main": "app.js",
"dependencies": {
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
"@paddlejs/paddlejs-core": "^2.1.18",
"d3-polygon": "2.0.0",
"js-clipper": "1.0.1",
"number-precision": "1.5.2"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC"
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 222 KiB

View File

@@ -0,0 +1,578 @@
/* global wx, Page */
import * as paddlejs from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
import clipper from 'js-clipper';
import { divide, enableBoundaryChecking, plus } from 'number-precision';
import { recDecode } from 'recPostprocess.js';
// eslint-disable-next-line no-undef
const plugin = requirePlugin('paddlejs-plugin');
const Polygon = require('d3-polygon');
global.wasm_url = 'pages/index/wasm/opencv_js.wasm.br';
const CV = require('./wasm/opencv.js');
plugin.register(paddlejs, wx);
let DETSHAPE = 960;
let RECWIDTH = 320;
const RECHEIGHT = 32;
// 声明后续图像变换要用到的canvas此时未绑定
let canvas_det;
let canvas_rec;
let my_canvas;
let my_canvas_ctx;
const imgList = [
'https://paddlejs.bj.bcebos.com/xcx/ocr.png'
];
// eslint-disable-next-line max-lines-per-function
const outputBox = (res) => {
const thresh = 0.3;
const box_thresh = 0.5;
const max_candidates = 1000;
const min_size = 3;
const width = 960;
const height = 960;
const pred = res;
const segmentation = [];
pred.forEach(item => {
segmentation.push(item > thresh ? 255 : 0);
});
function get_mini_boxes(contour) {
// 生成最小外接矩形
const bounding_box = CV.minAreaRect(contour);
const points = [];
const mat = new CV.Mat();
// 获取矩形的四个顶点坐标
CV.boxPoints(bounding_box, mat);
for (let i = 0; i < mat.data32F.length; i += 2) {
const arr = [];
arr[0] = mat.data32F[i];
arr[1] = mat.data32F[i + 1];
points.push(arr);
}
function sortNumber(a, b) {
return a[0] - b[0];
}
points.sort(sortNumber);
let index_1 = 0;
let index_2 = 1;
let index_3 = 2;
let index_4 = 3;
if (points[1][1] > points[0][1]) {
index_1 = 0;
index_4 = 1;
}
else {
index_1 = 1;
index_4 = 0;
}
if (points[3][1] > points[2][1]) {
index_2 = 2;
index_3 = 3;
}
else {
index_2 = 3;
index_3 = 2;
}
const box = [
points[index_1],
points[index_2],
points[index_3],
points[index_4]
];
const side = Math.min(bounding_box.size.height, bounding_box.size.width);
mat.delete();
return {
points: box,
side
};
}
function box_score_fast(bitmap, _box) {
const h = height;
const w = width;
const box = JSON.parse(JSON.stringify(_box));
const x = [];
const y = [];
box.forEach(item => {
x.push(item[0]);
y.push(item[1]);
});
// clip这个函数将将数组中的元素限制在a_min, a_max之间大于a_max的就使得它等于 a_max小于a_min,的就使得它等于a_min。
const xmin = clip(Math.floor(Math.min(...x)), 0, w - 1);
const xmax = clip(Math.ceil(Math.max(...x)), 0, w - 1);
const ymin = clip(Math.floor(Math.min(...y)), 0, h - 1);
const ymax = clip(Math.ceil(Math.max(...y)), 0, h - 1);
// eslint-disable-next-line new-cap
const mask = new CV.Mat.zeros(ymax - ymin + 1, xmax - xmin + 1, CV.CV_8UC1);
box.forEach(item => {
item[0] = Math.max(item[0] - xmin, 0);
item[1] = Math.max(item[1] - ymin, 0);
});
const npts = 4;
const point_data = new Uint8Array(box.flat());
const points = CV.matFromArray(npts, 1, CV.CV_32SC2, point_data);
const pts = new CV.MatVector();
pts.push_back(points);
const color = new CV.Scalar(255);
// 多个多边形填充
CV.fillPoly(mask, pts, color, 1);
const sliceArr = [];
for (let i = ymin; i < ymax + 1; i++) {
sliceArr.push(...bitmap.slice(960 * i + xmin, 960 * i + xmax + 1));
}
const mean = num_mean(sliceArr, mask.data);
mask.delete();
points.delete();
pts.delete();
return mean;
}
function clip(data, min, max) {
return data < min ? min : data > max ? max : data;
}
function unclip(box) {
const unclip_ratio = 1.6;
const area = Math.abs(Polygon.polygonArea(box));
const length = Polygon.polygonLength(box);
const distance = area * unclip_ratio / length;
const tmpArr = [];
box.forEach(item => {
const obj = {
X: 0,
Y: 0
};
obj.X = item[0];
obj.Y = item[1];
tmpArr.push(obj);
});
const offset = new clipper.ClipperOffset();
offset.AddPath(tmpArr, clipper.JoinType.jtRound, clipper.EndType.etClosedPolygon);
const expanded = [];
offset.Execute(expanded, distance);
let expandedArr = [];
expanded[0] && expanded[0].forEach(item => {
expandedArr.push([item.X, item.Y]);
});
expandedArr = [].concat(...expandedArr);
return expandedArr;
}
function num_mean(data, mask) {
let sum = 0;
let length = 0;
for (let i = 0; i < data.length; i++) {
if (mask[i]) {
sum = plus(sum, data[i]);
length++;
}
}
return divide(sum, length);
}
// eslint-disable-next-line new-cap
const src = new CV.matFromArray(960, 960, CV.CV_8UC1, segmentation);
const contours = new CV.MatVector();
const hierarchy = new CV.Mat();
// 获取轮廓
CV.findContours(src, contours, hierarchy, CV.RETR_LIST, CV.CHAIN_APPROX_SIMPLE);
const num_contours = Math.min(contours.size(), max_candidates);
const boxes = [];
const scores = [];
const arr = [];
for (let i = 0; i < num_contours; i++) {
const contour = contours.get(i);
let {
points,
side
} = get_mini_boxes(contour);
if (side < min_size) {
continue;
}
const score = box_score_fast(pred, points);
if (box_thresh > score) {
continue;
}
let box = unclip(points);
// eslint-disable-next-line new-cap
const boxMap = new CV.matFromArray(box.length / 2, 1, CV.CV_32SC2, box);
const resultObj = get_mini_boxes(boxMap);
box = resultObj.points;
side = resultObj.side;
if (side < min_size + 2) {
continue;
}
box.forEach(item => {
item[0] = clip(Math.round(item[0]), 0, 960);
item[1] = clip(Math.round(item[1]), 0, 960);
});
boxes.push(box);
scores.push(score);
arr.push(i);
boxMap.delete();
}
src.delete();
contours.delete();
hierarchy.delete();
return {
boxes,
scores
};
};
const sorted_boxes = (box) => {
function sortNumber(a, b) {
return a[0][1] - b[0][1];
}
const boxes = box.sort(sortNumber);
const num_boxes = boxes.length;
for (let i = 0; i < num_boxes - 1; i++) {
if (Math.abs(boxes[i + 1][0][1] - boxes[i][0][1]) < 10
&& boxes[i + 1][0][0] < boxes[i][0][0]) {
const tmp = boxes[i];
boxes[i] = boxes[i + 1];
boxes[i + 1] = tmp;
}
}
return boxes;
}
function flatten(arr) {
return arr.toString().split(',').map(item => +item);
}
function int(num) {
return num > 0 ? Math.floor(num) : Math.ceil(num);
}
function clip(data, min, max) {
return data < min ? min : data > max ? max : data;
}
function get_rotate_crop_image(img, points) {
const img_crop_width = int(Math.max(
linalg_norm(points[0], points[1]),
linalg_norm(points[2], points[3])
));
const img_crop_height = int(Math.max(
linalg_norm(points[0], points[3]),
linalg_norm(points[1], points[2])
));
const pts_std = [
[0, 0],
[img_crop_width, 0],
[img_crop_width, img_crop_height],
[0, img_crop_height]
];
const srcTri = CV.matFromArray(4, 1, CV.CV_32FC2, flatten(points));
const dstTri = CV.matFromArray(4, 1, CV.CV_32FC2, flatten(pts_std));
// 获取到目标矩阵
const M = CV.getPerspectiveTransform(srcTri, dstTri);
const src = CV.imread(img);
const dst = new CV.Mat();
const dsize = new CV.Size(img_crop_width, img_crop_height);
// 透视转换
CV.warpPerspective(src, dst, M, dsize, CV.INTER_CUBIC, CV.BORDER_REPLICATE, new CV.Scalar());
const dst_img_height = dst.rows;
const dst_img_width = dst.cols;
let dst_rot;
// 图像旋转
if (dst_img_height / dst_img_width >= 1.5) {
dst_rot = new CV.Mat();
const dsize_rot = new CV.Size(dst.rows, dst.cols);
const center = new CV.Point(dst.cols / 2, dst.cols / 2);
const M = CV.getRotationMatrix2D(center, 90, 1);
CV.warpAffine(dst, dst_rot, M, dsize_rot, CV.INTER_CUBIC, CV.BORDER_REPLICATE, new CV.Scalar());
}
const dst_resize = new CV.Mat();
const dsize_resize = new CV.Size(0, 0);
let scale;
if (dst_rot) {
scale = RECHEIGHT / dst_rot.rows;
CV.resize(dst_rot, dst_resize, dsize_resize, scale, scale, CV.INTER_AREA);
dst_rot.delete();
}
else {
scale = RECHEIGHT / dst_img_height;
CV.resize(dst, dst_resize, dsize_resize, scale, scale, CV.INTER_AREA);
}
canvas_det.width = dst_resize.cols;
canvas_det.height = dst_resize.rows;
canvas_det.getContext('2d').clearRect(0, 0, canvas_det.width, canvas_det.height);
CV.imshow(canvas_det, dst_resize);
src.delete();
dst.delete();
dst_resize.delete();
srcTri.delete();
dstTri.delete();
}
function linalg_norm(x, y) {
return Math.sqrt(Math.pow(x[0] - y[0], 2) + Math.pow(x[1] - y[1], 2));
}
function resize_norm_img_splice(
image,
origin_width,
origin_height,
index = 0
) {
canvas_rec.width = RECWIDTH;
canvas_rec.height = RECHEIGHT;
const ctx = canvas_rec.getContext('2d');
ctx.fillStyle = '#fff';
ctx.clearRect(0, 0, canvas_rec.width, canvas_rec.height);
// ctx.drawImage(image, -index * RECWIDTH, 0, origin_width, origin_height);
ctx.putImageData(image, -index * RECWIDTH, 0);
}
// 声明检测和识别Runner未初始化
let detectRunner;
let recRunner;
Page({
data: {
photo_src:'',
imgList: imgList,
imgInfo: {},
result: '',
select_mode: false,
loaded: false
},
switch_choose(){
this.setData({
select_mode: true
})
},
switch_example(){
this.setData({
select_mode: false
})
},
chose_photo:function(evt){
let _this = this
wx.chooseImage({
count: 1,
sizeType: ['original', 'compressed'],
sourceType: ['album', 'camera'],
success(res) {
console.log(res.tempFilePaths) //一个数组每个元素都是“http://...”图片地址
_this.setData({
photo_src: res.tempFilePaths[0]
})
}
})
},
reselect:function(evt){
let _this = this
wx.chooseImage({
count: 1,
sizeType: ['original', 'compressed'],
sourceType: ['album', 'camera'],
success(res) {
_this.setData({
photo_src: res.tempFilePaths[0]
})
}
})
},
photo_preview:function(evt){
let _this = this;
let imgs = [];
imgs.push(_this.data.photo_src);
wx.previewImage({
urls:imgs
})
},
predect_choose_img() {
console.log(this.data.photo_src)
this.getImageInfo(this.data.photo_src);
},
onLoad() {
enableBoundaryChecking(false);
// 绑定canvas该操作是异步因此最好加延迟保证后续使用时已完成绑定
wx.createSelectorQuery()
.select('#canvas_det')
.fields({ node: true, size: true })
.exec(async(res) => {
canvas_det = res[0].node;
});
wx.createSelectorQuery()
.select('#canvas_rec')
.fields({ node: true, size: true })
.exec(async(res) => {
canvas_rec = res[0].node;
});
wx.createSelectorQuery()
.select('#myCanvas')
.fields({ node: true, size: true })
.exec((res) => {
my_canvas = res[0].node;
my_canvas_ctx = my_canvas.getContext('2d');
});
const me = this;
// 初始化Runner
detectRunner = new paddlejs.Runner({
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_det_infer_js_960/model.json',
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225],
bgr: true,
webglFeedProcess: true
});
recRunner = new paddlejs.Runner({
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_rec_infer_js/model.json',
fill: '#000',
mean: [0.5, 0.5, 0.5],
std: [0.5, 0.5, 0.5],
bgr: true,
webglFeedProcess: true
});
// 等待模型数据全部加载完成
Promise.all([detectRunner.init(), recRunner.init()]).then(_ => {
me.setData({
loaded: true
});
});
},
selectImage(event) {
const imgPath = this.data.imgList[event.target.dataset.index];
this.getImageInfo(imgPath);
},
getImageInfo(imgPath) {
const me = this;
wx.getImageInfo({
src: imgPath,
success: (imgInfo) => {
const {
path,
width,
height
} = imgInfo;
const canvasPath = imgPath.includes('http') ? path : imgPath;
let sw = 960;
let sh = 960;
let x = 0;
let y = 0;
if (height / width >= 1) {
sw = Math.round(sh * width / height);
x = Math.floor((960 - sw) / 2);
}
else {
sh = Math.round(sw * height / width);
y = Math.floor((960 - sh) / 2);
}
my_canvas.width = sw;
my_canvas.height = sh;
// 微信上canvas输入图片
const image = my_canvas.createImage();
image.src = canvasPath;
image.onload = () => {
my_canvas_ctx.clearRect(0, 0, my_canvas.width, my_canvas.height);
my_canvas_ctx.drawImage(image, x, y, sw, sh);
const imageData = my_canvas_ctx.getImageData(0, 0, sw, sh);
// 开始识别
me.recognize({
data: imageData.data,
width: 960,
height: 960
}, {canvasPath, sw, sh, x, y});
}
}
});
},
async recognize(res, img) {
const me = this;
// 文本框选坐标点
let points;
await detectRunner.predict(res, function (detectRes) {
points = outputBox(detectRes);
});
// 绘制文本框
me.drawCanvasPoints(img, points.boxes);
// 排序,使得最后结果输出尽量按照从上到下的顺序
const boxes = sorted_boxes(points.boxes);
const text_list = [];
for (let i = 0; i < boxes.length; i++) {
const tmp_box = JSON.parse(JSON.stringify(boxes[i]));
// 获取tmp_box对应图片到canvas_det
get_rotate_crop_image(res, tmp_box);
// 这里是计算要识别文字的图片片段是否大于识别模型要求的输入宽度;超过了的话会分成多次识别,再拼接结果
const width_num = Math.ceil(canvas_det.width / RECWIDTH);
let text_list_tmp = '';
for (let j = 0; j < width_num; j++) {
// 根据原图的宽度进行裁剪拼接,超出指定宽度会被截断;然后再次识别,最后拼接起来
resize_norm_img_splice(canvas_det.getContext('2d').getImageData(0, 0, canvas_det.width, canvas_det.height), canvas_det.width, canvas_det.height, j);
const imgData = canvas_rec.getContext('2d').getImageData(0, 0, canvas_rec.width, canvas_rec.height);
await recRunner.predict(imgData, function(output){
// 将输出向量转化为idx再传化为对应字符
const text = recDecode(output);
text_list_tmp = text_list_tmp.concat(text.text);
});
}
text_list.push(text_list_tmp);
}
me.setData({
result: JSON.stringify(boxes) + JSON.stringify(text_list)
});
},
drawCanvasPoints(img, points) {
// 设置线条
my_canvas_ctx.strokeStyle = 'blue';
my_canvas_ctx.lineWidth = 5;
// 先绘制图片
const image = my_canvas.createImage();
image.src = img.canvasPath;
image.onload = () => {
my_canvas_ctx.clearRect(0, 0, my_canvas_ctx.width, my_canvas_ctx.height);
my_canvas_ctx.drawImage(image, img.x, img.y, img.sw, img.sh);
// 绘制线框
points.length && points.forEach(point => {
my_canvas_ctx.beginPath();
// 设置路径起点坐标
my_canvas_ctx.moveTo(point[0][0], point[0][1]);
my_canvas_ctx.lineTo(point[1][0], point[1][1]);
my_canvas_ctx.lineTo(point[2][0], point[2][1]);
my_canvas_ctx.lineTo(point[3][0], point[3][1]);
my_canvas_ctx.lineTo(point[0][0], point[0][1]);
my_canvas_ctx.stroke();
my_canvas_ctx.closePath();
});
}
}
});

View File

@@ -0,0 +1,2 @@
{
}

View File

@@ -0,0 +1,56 @@
<view>
<view>
<button bindtap="switch_choose" wx:if="{{!select_mode}}">切换选择本地图片模式</button>
<button bindtap="switch_example" wx:else>切换示例图片模式</button>
</view>
<view class="photo_box" wx:if="{{select_mode}}">
<view wx:if="{{photo_src != ''}}" class="photo_preview">
<image src="{{photo_src}}" mode="aspectFit" bindtap="photo_preview"></image>
<view class="reselect" bindtap="reselect">重新选择</view>
</view>
<view wx:else class="photo_text" bindtap="chose_photo">点击拍照或上传本地照片</view>
<button bindtap="predect_choose_img">检测</button>
</view>
<view wx:else>
<text class="title">点击图片进行预测</text>
<scroll-view class="imgWrapper" scroll-x="true">
<image
class="img {{selectedIndex == index ? 'selected' : ''}}"
wx:for="{{imgList}}"
wx:key="index"
src="{{item}}"
mode="aspectFit"
bindtap="selectImage"
data-index="{{index}}"
></image>
</scroll-view>
</view>
<view class="img-view">
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
<canvas
id="myCanvas"
type="2d"
style="width: 960px; height: 960px;"
></canvas>
</scroll-view>
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
<canvas
id="canvas_det"
type="2d"
style="width: 960px; height: 960px;"
></canvas>
</scroll-view>
<scroll-view class="imgWrapper" scroll-x="true" style="width: 960px; height: 960px;">
<canvas
id="canvas_rec"
type="2d"
style="width: 960px; height: 960px;"
></canvas>
</scroll-view>
<text class="result" wx:if="{{result}}" style="height: 300rpx;">文本框选坐标:{{result}}</text>
</view>
</view>
<view class="mask" wx:if="{{!loaded}}">
<text class="loading">loading…</text>
</view>

View File

@@ -0,0 +1,78 @@
.photo_box{
width: 750rpx;
border: 1px solid #cccccc;
box-sizing: border-box;
}
.photo_text{
width: 100%;
line-height: 500rpx;
text-align: center;
}
.photo_preview image{
width: 750rpx;
}
.photo_preview .reselect{
width: 750rpx;
height: 100rpx;
background-color: #3F8EFF;
text-align: center;
line-height: 100rpx;
border-top: 1px solid #cccccc;
}
text {
display: block;
}
.title {
margin-top: 10px;
font-size: 16px;
line-height: 32px;
font-weight: bold;
}
.imgWrapper {
margin: 10px 10px 0;
white-space: nowrap;
}
.img {
width: 960px;
height: 960px;
border: 1px solid #f1f1f1;
}
.result {
margin-top: 5px;
}
.selected {
border: 1px solid #999;
}
.select-btn {
margin-top: 20px;
width: 60%;
}
.mask {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: rgba(0, 0, 0, .7);
}
.loading {
color: #fff;
font-size: 20px;
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
}
.img-view {
padding-bottom: 20px;
border-bottom: 1px solid #f1f1f1;
}

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,63 @@
import { character } from 'ppocr_keys_v1.js';
const ocr_character = character;
let preds_idx = [];
let preds_prob = [];
function init(preds) {
preds_idx = [];
preds_prob = [];
// preds: [1, ?, 6625]
const pred_len = 6625;
for (let i = 0; i < preds.length; i += pred_len) {
const tmpArr = preds.slice(i, i + pred_len - 1);
const tmpMax = Math.max(...tmpArr);
const tmpIdx = tmpArr.indexOf(tmpMax);
preds_prob.push(tmpMax);
preds_idx.push(tmpIdx);
}
}
function get_ignored_tokens() {
return [0];
}
function decode(text_index, text_prob, is_remove_duplicate = false) {
const ignored_tokens = get_ignored_tokens();
const char_list = [];
const conf_list = [];
for (let idx = 0; idx < text_index.length; idx++) {
if (text_index[idx] in ignored_tokens) {
continue;
}
if (is_remove_duplicate) {
if (idx > 0 && text_index[idx - 1] === text_index[idx]) {
continue;
}
}
char_list.push(ocr_character[text_index[idx] - 1]);
if (text_prob) {
conf_list.push(text_prob[idx]);
}
else {
conf_list.push(1);
}
}
let text = '';
let mean = 0;
if (char_list.length) {
text = char_list.join('');
let sum = 0;
conf_list.forEach(item => {
sum += item;
});
mean = sum / conf_list.length;
}
return { text, mean };
}
export function recDecode(preds) {
init(preds);
return decode(preds_idx, preds_prob, true);
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,58 @@
{
"description": "项目配置文件详见文档https://developers.weixin.qq.com/miniprogram/dev/devtools/projectconfig.html",
"packOptions": {
"ignore": [],
"include": []
},
"setting": {
"urlCheck": false,
"es6": true,
"enhance": true,
"postcss": true,
"preloadBackgroundData": false,
"minified": true,
"newFeature": false,
"coverView": true,
"nodeModules": true,
"autoAudits": false,
"showShadowRootInWxmlPanel": true,
"scopeDataCheck": false,
"uglifyFileName": false,
"checkInvalidKey": true,
"checkSiteMap": true,
"uploadWithSourceMap": true,
"compileHotReLoad": false,
"lazyloadPlaceholderEnable": false,
"useMultiFrameRuntime": true,
"useApiHook": true,
"useApiHostProcess": true,
"babelSetting": {
"ignore": [],
"disablePlugins": [],
"outputPath": ""
},
"enableEngineNative": false,
"useIsolateContext": true,
"userConfirmedBundleSwitch": false,
"packNpmManually": false,
"packNpmRelationList": [],
"minifyWXSS": true,
"disableUseStrict": false,
"minifyWXML": true,
"showES6CompileOption": false,
"useCompilerPlugins": false,
"useStaticServer": true,
"ignoreUploadUnusedFiles": false
},
"compileType": "miniprogram",
"libVersion": "2.22.1",
"appid": "wx78461a9c81d1234c",
"projectname": "mobilenet",
"simulatorType": "wechat",
"simulatorPluginLibVersion": {},
"condition": {},
"editorSetting": {
"tabIndent": "insertSpaces",
"tabSize": 2
}
}

View File

@@ -0,0 +1,8 @@
{
"projectname": "ocrXcx",
"setting": {
"compileHotReLoad": true
},
"description": "项目私有配置文件。此文件中的内容将覆盖 project.config.json 中的相同字段。项目的改动优先同步到此文件中。详见文档https://developers.weixin.qq.com/miniprogram/dev/devtools/projectconfig.html",
"libVersion": "2.23.4"
}

View File

@@ -0,0 +1,7 @@
{
"desc": "关于本文件的更多信息,请参考文档 https://developers.weixin.qq.com/miniprogram/dev/framework/sitemap.html",
"rules": [{
"action": "allow",
"page": "*"
}]
}

View File

@@ -0,0 +1,12 @@
/* global wx, App */
import * as paddlejs from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
// eslint-disable-next-line no-undef
const plugin = requirePlugin('paddlejs-plugin');
plugin.register(paddlejs, wx);
App({
globalData: {
Paddlejs: paddlejs.Runner
}
});

View File

@@ -0,0 +1,12 @@
{
"pages": [
"pages/index/index"
],
"plugins": {
"paddlejs-plugin": {
"version": "2.0.1",
"provider": "wx7138a7bb793608c3"
}
},
"sitemapLocation": "sitemap.json"
}

View File

@@ -0,0 +1,33 @@
{
"name": "paddlejs-demo",
"version": "0.0.1",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"@paddlejs/paddlejs-backend-webgl": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-backend-webgl/-/paddlejs-backend-webgl-1.2.0.tgz",
"integrity": "sha512-bKJKJkGldC3NPOuJyk+372z0XW1dd1D9lR0f9OHqWQboY0Mkah+gX+8tkerrNg+QjYz88IW0iJaRKB0jm+6d9g=="
},
"@paddlejs/paddlejs-core": {
"version": "2.1.18",
"resolved": "https://registry.npmjs.org/@paddlejs/paddlejs-core/-/paddlejs-core-2.1.18.tgz",
"integrity": "sha512-QrXxwaHm4llp1sxbUq/oCCqlYx4ciXanBn/Lfq09UqR4zkYi5SptacQlIxgJ70HOO6RWIxjWN4liQckMwa2TkA=="
},
"d3-polygon": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-2.0.0.tgz",
"integrity": "sha512-MsexrCK38cTGermELs0cO1d79DcTsQRN7IWMJKczD/2kBjzNXxLUWP33qRF6VDpiLV/4EI4r6Gs0DAWQkE8pSQ=="
},
"js-clipper": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/js-clipper/-/js-clipper-1.0.1.tgz",
"integrity": "sha1-TWsHQ0pECOfBKeMiAc5m0hR07SE="
},
"number-precision": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/number-precision/-/number-precision-1.5.2.tgz",
"integrity": "sha512-q7C1ZW3FyjsJ+IpGB6ykX8OWWa5+6M+hEY0zXBlzq1Sq1IPY9GeI3CQ9b2i6CMIYoeSuFhop2Av/OhCxClXqag=="
}
}
}

View File

@@ -0,0 +1,19 @@
{
"name": "paddlejs-demo",
"version": "0.0.1",
"description": "",
"main": "app.js",
"dependencies": {
"@paddlejs/paddlejs-backend-webgl": "^1.2.0",
"@paddlejs/paddlejs-core": "^2.1.18",
"d3-polygon": "2.0.0",
"js-clipper": "1.0.1",
"number-precision": "1.5.2"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC"
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 222 KiB

View File

@@ -0,0 +1,337 @@
/* global wx, Page */
import * as paddlejs from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
import clipper from 'js-clipper';
import { divide, enableBoundaryChecking, plus } from 'number-precision';
// eslint-disable-next-line no-undef
const plugin = requirePlugin('paddlejs-plugin');
const Polygon = require('d3-polygon');
global.wasm_url = 'pages/index/wasm/opencv_js.wasm.br';
const CV = require('./wasm/opencv.js');
plugin.register(paddlejs, wx);
const imgList = [
'https://paddlejs.bj.bcebos.com/xcx/ocr.png',
'./img/width.png'
];
// eslint-disable-next-line max-lines-per-function
const outputBox = res => {
const thresh = 0.3;
const box_thresh = 0.5;
const max_candidates = 1000;
const min_size = 3;
const width = 960;
const height = 960;
const pred = res;
const segmentation = [];
pred.forEach(item => {
segmentation.push(item > thresh ? 255 : 0);
});
function get_mini_boxes(contour) {
// 生成最小外接矩形
const bounding_box = CV.minAreaRect(contour);
const points = [];
const mat = new CV.Mat();
// 获取矩形的四个顶点坐标
CV.boxPoints(bounding_box, mat);
for (let i = 0; i < mat.data32F.length; i += 2) {
const arr = [];
arr[0] = mat.data32F[i];
arr[1] = mat.data32F[i + 1];
points.push(arr);
}
function sortNumber(a, b) {
return a[0] - b[0];
}
points.sort(sortNumber);
let index_1 = 0;
let index_2 = 1;
let index_3 = 2;
let index_4 = 3;
if (points[1][1] > points[0][1]) {
index_1 = 0;
index_4 = 1;
}
else {
index_1 = 1;
index_4 = 0;
}
if (points[3][1] > points[2][1]) {
index_2 = 2;
index_3 = 3;
}
else {
index_2 = 3;
index_3 = 2;
}
const box = [
points[index_1],
points[index_2],
points[index_3],
points[index_4]
];
const side = Math.min(bounding_box.size.height, bounding_box.size.width);
mat.delete();
return {
points: box,
side
};
}
function box_score_fast(bitmap, _box) {
const h = height;
const w = width;
const box = JSON.parse(JSON.stringify(_box));
const x = [];
const y = [];
box.forEach(item => {
x.push(item[0]);
y.push(item[1]);
});
// clip这个函数将将数组中的元素限制在a_min, a_max之间大于a_max的就使得它等于 a_max小于a_min,的就使得它等于a_min。
const xmin = clip(Math.floor(Math.min(...x)), 0, w - 1);
const xmax = clip(Math.ceil(Math.max(...x)), 0, w - 1);
const ymin = clip(Math.floor(Math.min(...y)), 0, h - 1);
const ymax = clip(Math.ceil(Math.max(...y)), 0, h - 1);
// eslint-disable-next-line new-cap
const mask = new CV.Mat.zeros(ymax - ymin + 1, xmax - xmin + 1, CV.CV_8UC1);
box.forEach(item => {
item[0] = Math.max(item[0] - xmin, 0);
item[1] = Math.max(item[1] - ymin, 0);
});
const npts = 4;
const point_data = new Uint8Array(box.flat());
const points = CV.matFromArray(npts, 1, CV.CV_32SC2, point_data);
const pts = new CV.MatVector();
pts.push_back(points);
const color = new CV.Scalar(255);
// 多个多边形填充
CV.fillPoly(mask, pts, color, 1);
const sliceArr = [];
for (let i = ymin; i < ymax + 1; i++) {
sliceArr.push(...bitmap.slice(960 * i + xmin, 960 * i + xmax + 1));
}
const mean = num_mean(sliceArr, mask.data);
mask.delete();
points.delete();
pts.delete();
return mean;
}
function clip(data, min, max) {
return data < min ? min : data > max ? max : data;
}
function unclip(box) {
const unclip_ratio = 1.6;
const area = Math.abs(Polygon.polygonArea(box));
const length = Polygon.polygonLength(box);
const distance = area * unclip_ratio / length;
const tmpArr = [];
box.forEach(item => {
const obj = {
X: 0,
Y: 0
};
obj.X = item[0];
obj.Y = item[1];
tmpArr.push(obj);
});
const offset = new clipper.ClipperOffset();
offset.AddPath(tmpArr, clipper.JoinType.jtRound, clipper.EndType.etClosedPolygon);
const expanded = [];
offset.Execute(expanded, distance);
let expandedArr = [];
expanded[0] && expanded[0].forEach(item => {
expandedArr.push([item.X, item.Y]);
});
expandedArr = [].concat(...expandedArr);
return expandedArr;
}
function num_mean(data, mask) {
let sum = 0;
let length = 0;
for (let i = 0; i < data.length; i++) {
if (mask[i]) {
sum = plus(sum, data[i]);
length++;
}
}
return divide(sum, length);
}
// eslint-disable-next-line new-cap
const src = new CV.matFromArray(960, 960, CV.CV_8UC1, segmentation);
const contours = new CV.MatVector();
const hierarchy = new CV.Mat();
// 获取轮廓
CV.findContours(src, contours, hierarchy, CV.RETR_LIST, CV.CHAIN_APPROX_SIMPLE);
const num_contours = Math.min(contours.size(), max_candidates);
const boxes = [];
const scores = [];
const arr = [];
for (let i = 0; i < num_contours; i++) {
const contour = contours.get(i);
let {
points,
side
} = get_mini_boxes(contour);
if (side < min_size) {
continue;
}
const score = box_score_fast(pred, points);
if (box_thresh > score) {
continue;
}
let box = unclip(points);
// eslint-disable-next-line new-cap
const boxMap = new CV.matFromArray(box.length / 2, 1, CV.CV_32SC2, box);
const resultObj = get_mini_boxes(boxMap);
box = resultObj.points;
side = resultObj.side;
if (side < min_size + 2) {
continue;
}
box.forEach(item => {
item[0] = clip(Math.round(item[0]), 0, 960);
item[1] = clip(Math.round(item[1]), 0, 960);
});
boxes.push(box);
scores.push(score);
arr.push(i);
boxMap.delete();
}
src.delete();
contours.delete();
hierarchy.delete();
return {
boxes,
scores
};
};
let detectRunner;
Page({
data: {
imgList: imgList,
imgInfo: {},
result: '',
loaded: false
},
onLoad() {
enableBoundaryChecking(false);
const me = this;
detectRunner = new paddlejs.Runner({
modelPath: 'https://paddleocr.bj.bcebos.com/PaddleJS/PP-OCRv3/ch/ch_PP-OCRv3_det_infer_js_960/model.json',
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225],
bgr: true,
webglFeedProcess: true
});
detectRunner.init().then(_ => {
me.setData({
loaded: true
});
});
},
selectImage(event) {
const imgPath = this.data.imgList[event.target.dataset.index];
this.getImageInfo(imgPath);
},
getImageInfo(imgPath) {
const me = this;
wx.getImageInfo({
src: imgPath,
success: imgInfo => {
const {
path,
width,
height
} = imgInfo;
const canvasPath = imgPath.includes('http') ? path : imgPath;
const canvasId = 'myCanvas';
const ctx = wx.createCanvasContext(canvasId);
let sw = 960;
let sh = 960;
let x = 0;
let y = 0;
if (height / width >= 1) {
sw = Math.round(sh * width / height);
x = Math.floor((960 - sw) / 2);
}
else {
sh = Math.round(sw * height / width);
y = Math.floor((960 - sh) / 2);
}
ctx.drawImage(canvasPath, x, y, sw, sh);
ctx.draw(false, () => {
// API 1.9.0 获取图像数据
wx.canvasGetImageData({
canvasId: canvasId,
x: 0,
y: 0,
width: 960,
height: 960,
success(res) {
me.predict({
data: res.data,
width: 960,
height: 960
}, {
canvasPath,
sw,
sh,
x,
y
});
}
});
});
}
});
},
predict(res, img) {
const me = this;
detectRunner.predict(res, function (data) {
// 获取坐标
const points = outputBox(data);
me.drawCanvasPoints(img, points.boxes);
me.setData({
result: JSON.stringify(points.boxes)
});
});
},
drawCanvasPoints(img, points) {
const canvasId = 'result';
const ctx = wx.createCanvasContext(canvasId);
ctx.drawImage(img.canvasPath, img.x, img.y, img.sw, img.sh);
points.length && points.forEach(point => {
// 开始一个新的绘制路径
ctx.beginPath();
// 设置线条颜色为蓝色
ctx.strokeStyle = 'blue';
// 设置路径起点坐标
ctx.moveTo(point[0][0], point[0][1]);
ctx.lineTo(point[1][0], point[1][1]);
ctx.lineTo(point[2][0], point[2][1]);
ctx.lineTo(point[3][0], point[3][1]);
ctx.closePath();
ctx.stroke();
});
ctx.draw();
}
});

View File

@@ -0,0 +1,29 @@
<view>
<view class="img-view">
<text class="title">点击图片进行预测</text>
<scroll-view class="imgWrapper" scroll-x="true">
<image
class="img {{selectedIndex == index ? 'selected' : ''}}"
wx:for="{{imgList}}"
wx:key="index"
src="{{item}}"
mode="aspectFit"
bindtap="selectImage"
data-index="{{index}}"
></image>
<canvas
canvas-id="myCanvas"
style="width: 960px; height: 960px; position: absolute; z-index: -1"
></canvas>
<canvas
canvas-id="result"
style="width: 960px; height: 960px;"
></canvas>
</scroll-view>
<text class="result" wx:if="{{result}}" style="height: 300rpx;">文本框选坐标:{{result}}</text>
</view>
</view>
<view class="mask" wx:if="{{!loaded}}">
<text class="loading">loading…</text>
</view>

View File

@@ -0,0 +1,56 @@
text {
display: block;
}
.title {
margin-top: 10px;
font-size: 16px;
line-height: 32px;
font-weight: bold;
}
.imgWrapper {
margin: 10px 10px 0;
white-space: nowrap;
}
.img {
width: 960px;
height: 960px;
border: 1px solid #f1f1f1;
}
.result {
margin-top: 5px;
}
.selected {
border: 1px solid #999;
}
.select-btn {
margin-top: 20px;
width: 60%;
}
.mask {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: rgba(0, 0, 0, .7);
}
.loading {
color: #fff;
font-size: 20px;
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
}
.img-view {
padding-bottom: 20px;
border-bottom: 1px solid #f1f1f1;
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,75 @@
{
"description": "项目配置文件",
"packOptions": {
"ignore": []
},
"setting": {
"urlCheck": false,
"es6": true,
"enhance": true,
"postcss": true,
"preloadBackgroundData": false,
"minified": true,
"newFeature": false,
"coverView": true,
"nodeModules": true,
"autoAudits": false,
"showShadowRootInWxmlPanel": true,
"scopeDataCheck": false,
"uglifyFileName": false,
"checkInvalidKey": true,
"checkSiteMap": true,
"uploadWithSourceMap": true,
"compileHotReLoad": false,
"lazyloadPlaceholderEnable": false,
"useMultiFrameRuntime": true,
"useApiHook": true,
"useApiHostProcess": true,
"babelSetting": {
"ignore": [],
"disablePlugins": [],
"outputPath": ""
},
"enableEngineNative": false,
"useIsolateContext": true,
"userConfirmedBundleSwitch": false,
"packNpmManually": false,
"packNpmRelationList": [],
"minifyWXSS": true,
"disableUseStrict": false,
"minifyWXML": true,
"showES6CompileOption": false,
"useCompilerPlugins": false
},
"compileType": "miniprogram",
"libVersion": "2.22.1",
"appid": "wxc43cbd2fafe0aec2",
"projectname": "mobilenet",
"debugOptions": {
"hidedInDevtools": []
},
"scripts": {},
"isGameTourist": false,
"simulatorType": "wechat",
"simulatorPluginLibVersion": {},
"condition": {
"search": {
"list": []
},
"conversation": {
"list": []
},
"game": {
"list": []
},
"plugin": {
"list": []
},
"gamePlugin": {
"list": []
},
"miniprogram": {
"list": []
}
}
}

View File

@@ -0,0 +1,7 @@
{
"desc": "关于本文件的更多信息,请参考文档 https://developers.weixin.qq.com/miniprogram/dev/framework/sitemap.html",
"rules": [{
"action": "allow",
"page": "*"
}]
}

View File

@@ -0,0 +1,172 @@
# Web Demo介绍
- [简介](#0)
- [1. 快速开始](#1)
- [2. npm包调用](#2)
- [3. 模型替换](#3)
- [4. 自定义前后处理参数](#4)
- [5. 其他](#5)
<a name="0"></a>
## 简介
本项目基于[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)在浏览器中实现目标检测人像分割OCR物品分类等计算机视觉任务。
|demo名称|web demo组件|源码目录|npm包|
|-|-|-|-|
|人脸检测|[FaceDetection](./demo/src/pages/cv/detection/FaceDetection/)| [facedetect](./packages/paddlejs-models/facedetect)|[@paddle-js-models/facedetect](https://www.npmjs.com/package/@paddle-js-models/facedetect)|
|螺丝钉检测|[ScrewDetection](./demo/src/pages/cv/detection/ScrewDetection)| [detect](./packages/paddlejs-models/detect)|[@paddle-js-models/detect](https://www.npmjs.com/package/@paddle-js-models/detect)|
|人像分割背景替换|[HumanSeg](./demo/src/pages/cv/segmentation/HumanSeg)|[humanseg](./packages/paddlejs-models/humanseg)|[@paddle-js-models/humanseg](https://www.npmjs.com/package/@paddle-js-models/humanseg)|
|手势识别AI猜丁壳|[GestureRecognition](./demo/src/pages/cv/recognition/GestureRecognition)|[gesture](./packages/paddlejs-models/gesture)|[@paddle-js-models/gesture](https://www.npmjs.com/package/@paddle-js-models/gesture)|
|1000种物品识别|[ItemIdentification](./demo/src/pages/cv/recognition/ItemIdentification)|[mobilenet](./packages/paddlejs-models/mobilenet)|[@paddle-js-models/mobilenet](https://www.npmjs.com/package/@paddle-js-models/mobilenet)|
|文本检测|[TextDetection](./demo/src/pages/cv/ocr/TextDetection)|[ocrdetection](./packages/paddlejs-models/ocrdetection)|[@paddle-js-models/ocrdet](https://www.npmjs.com/package/@paddle-js-models/ocrdet)|
|文本识别|[TextRecognition](./demo/src/pages/cv/ocr/TextRecognition)|[ocr](./packages/paddlejs-models/ocr)|[@paddle-js-models/ocr](https://www.npmjs.com/package/@paddle-js-models/ocr)|
<a name="1"></a>
## 1. 快速开始
本节介绍如何在浏览器中直接运行官方demo。
**1. 安装Node.js**
`Node.js`官网https://nodejs.org/en/download/ 下载适合自己平台的`Node.js`安装包并安装。
**2. 安装demo依赖并启动**
`./web_demo/demo`目录下执行如下指令:
```
# 安装依赖
npm install
# 启动demo
npm run dev
```
在浏览器中打开网址 `http://localhost:5173/main/index.html` 即可快速体验在浏览器中运行计算机视觉任务。
![22416f4a3e7d63f950b838be3cd11e80](https://user-images.githubusercontent.com/26592129/196685868-93ab53bd-cb2e-44ff-a56b-50c1781b8679.jpg)
<a name="2"></a>
## 2. npm包调用
本节介绍npm包的使用方式每个demo均提供简单易用的接口用户只需初始化上传图片即可获得结果使用步骤如下
1. 调用模块
2. 初始化模型
3. 传入输入,执行预测
以 OCR 为例,在前端项目中,`@paddle-js-models/ocr`包的使用方式如下:
```
// 1. 调用ocr模块
import * as ocr from '@paddle-js-models/ocr';
// 2. 初始化ocr模型
await ocr.init();
// 3. 传入HTMLImageElement类型的图像作为输入并获得结果
const res = await ocr.recognize(img);
// 打印OCR模型得到的文本坐标以及文本内容
console.log(res.text);
console.log(res.points);
```
<a name="3"></a>
## 3. 模型替换
由于前端环境和计算资源限制在前端部署深度学习模型时我们对模型的性能有着更严格的要求简单来说模型需要足够轻量化。理论上模型的输入shape越小、模型大小越小则对应的模型的flops越小在前端运行也能更流畅。经验总结使用`Paddle.js`部署的模型存储尽量不超过*5M*,实际情况根据硬件和计算资源情况决定。
在实际应用中常常根据垂类的场景定制化模型官方的demo支持修改传入参数替换模型。
以OCR demo为例[ocr.init()函数](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo/packages/paddlejs-models/ocr/src/index.ts#L52)中,包含默认初始化的模型链接,如果要替换模型参考下述步骤。
步骤1将模型转成js格式
```
# 安装paddlejsconverter
pip3 install paddlejsconverter
# 转换模型格式输入模型为inference模型
paddlejsconverter --modelPath=./inference.pdmodel --paramPath=./inference.pdiparams --outputDir=./ --useGPUOpt=True
# 注意useGPUOpt 选项默认不开启,如果模型用在 gpu backendwebgl/webgpu则开启 useGPUOpt如果模型运行在wasm/plain js则不要开启。
```
导出成功后,本地目录下会出现 `model.json chunk_1.dat`等文件分别是对应js模型的网络结构、模型参数二进制文件。
步骤2将导出的js模型上传到支持跨域访问的服务器服务器的CORS配置参考下图
![image](https://user-images.githubusercontent.com/26592129/196612669-5233137a-969c-49eb-b8c7-71bef5088686.png)
步骤3修改代码替换默认的模型。以OCR demo为例修改OCR web demo中[模型初始化代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/js/web_demo/demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L64),即
```
await ocr.init();
修改为:
await ocr.init({modelPath: "https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json"}); # 第一个参数传入新的文本检测字典类型参数
```
重新在demo目录下执行下述命令即可体验新的模型效果。
```
npm run dev
```
<a name="4"></a>
## 4. 自定义前后处理参数
**自定义前处理参数**
在不同计算机视觉任务中不同的模型可能有不同的预处理参数比如mean,std,keep_ratio等参数替换模型后也需要对预处理参数进行修改。paddle.js发布的npm包中提供了自定义预处理参数的简单方案。只需要在调用模型初始化函数时传入自定义的参数即可。
```
# 默认参数初始化
await model.init();
自定义参数初始化
const Config = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5], keepratio: false};
await model.init(Config);
```
以OCR文本检测demo为例修改模型前处理的mean和std参数只需要在模型初始化时传入自定义的mean和std参数。
```
await ocr.init();
修改为:
const detConfig = {mean: [0.5, 0.5, 0.5], std: [0.5, 0.5, 0.5]};
await ocr.init(detConfig); # 第一个参数传入新的文本检测模型链接
```
**自定义后处理参数**
同理paddle.js发布的npm包也提供了后处理参数的自定义方案。
```
# 默认参数运行
await model.predict();
# 自定义后处理参数
const postConfig = {thresh: 0.5};
await model.predict(Config);
```
以OCR文本检测 demo为例修改文本检测后处理的参数实现扩大文本检测框的效果修改OCR web demo中执行[模型预测代码](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/application/web_demo/demo/src/pages/cv/ocr/TextRecognition/TextRecognition.vue#L99),即:
```
const res = await ocr.recognize(img, { canvas: canvas.value });
修改为:
// 定义超参数将unclip_ratio参数从1.5 增大为3.5
const detConfig = {shape: 960, thresh: 0.3, box_thresh: 0.6, unclip_ratio:3.5};
const res = await ocr.recognize(img, { canvas: canvas.value }, detConfig);
```
不同的任务有不同的后处理参数详细参数参考npm包中的API。
<a name="5"></a>
## 5. 其他
`Paddle.js`转换后的模型不仅支持浏览器中使用,也可以在百度小程序和微信小程序环境下运行。
|名称|目录|
|-|-|
|OCR文本检测| [ocrdetecXcx](../mini_program/ocrdetectXcx/) |
|OCR文本识别| [ocrXcx](../mini_program/ocrXcx/) |
|目标检测| coming soon |
|图像分割| coming soon |
|物品分类| coming soon |

View File

@@ -0,0 +1,73 @@
[中文版](./DEVELOPMENT_cn.md)
# paddlejs-converter
paddlejs-converter is a model transformation tool for Paddle.js. Its role is to convert PaddlePaddle models (also known as fluid models) into a browser-friendly format that Paddle.js can use to load and predict usage in browsers as well as other environments. In addition, paddlejs-converter provides powerful model optimization capabilities to help developers optimize the model structure and improve runtime performance.
## 1. Tutorial
### 1.1. Environment Construction
#### Python Version
Confirm whether the python environment and version of the running platform meet the requirements. If Python 3 is used, you may need to change the `python` in subsequent commands to `python3`:
- Python3 3.5.1+ / 3.6 / 3.7
- Python2 2.7.15+
#### Install Virtual Environment
*Since the development environment may have multiple versions of Python installed, there may be different versions of dependent packages. In order to avoid conflicts, it is strongly recommended to use Python virtual environment to execute the commands required by the conversion tool to avoid various problems. If you are not using a virtual environment or if you have a virtual environment installed, you can skip this step.*
Take Anaconda as an example
Go to [Anaconda](https://www.anaconda.com/) main pageSelect the corresponding platform and python version of anaconda and install it according to the official prompts
After installation, execute the following command on the command line to create a python virtual environment:
``` bash
conda create --name <your_env_name>
```
Execute the following command to switch to the virtual environment
``` bash
# Linux or macOS
source activate <your_env_name>
# Windows
activate <your_env_name>
```
#### Installation Dependency
- If you don't need to optimize model, execute the command
``` bash
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
```
- Otherwiseexecute the command
``` bash
python -m pip install paddlepaddle paddlelite==2.6.0 -i https://mirror.baidu.com/pypi/simple
```
### 1.2. Get Start
- If the weight file of fluid model to be converted is merged format which means one model corresponds to one weight file, then execute:
``` bash
python convertToPaddleJSModel.py --modelPath=<fluid_model_file_path> --paramPath=<fluid_param_file_path> --outputDir=<paddlejs_model_directory>
```
- Otherwiseexecute
``` bash
# Note that in this way, you need to ensure that the model file name '__ model__ ' in the inputDir
python convertToPaddleJSModel.py --inputDir=<fluid_model_directory> --outputDir=<paddlejs_model_directory>
````
The model converter generates the following two types of files for Paddle.js:
- model.json (Contains the model structure and parameter list)
- chunk_\*.dat (The collection of binary weight files)
## 2. Detailed Documentation
Parameter | description
:-: | :-:
--inputDir | The fluid model directory, If and only if weight files are not merged format, `modelPath` and `paramPath` below will be ignoredand the model file name should be `__model__`.
--modelPath | The model file path, used when the weight file is merged.
--paramPath | The weight file pathused when the weight file is merged.
--outputDir | `Necessary`, the output model directory generated after converting.
--disableOptimize | Whether to disable optimize model, `1`is to disable, `0`is use optimize(need to install PaddleLite), default 0.
--logModelInfo | Whether to print model structure information `0` means not to print, `1` means to print, default 0.
--sliceDataSize | Shard size (in KB) of each weight file. Default size is 4096.
--useGPUOpt | Whether to use gpu opt, default is False.
## 3. Other information
If the model to be converted is in `tensorflow / Cafe / onnx` format, there is [X2Paddle](https://github.com/PaddlePaddle/X2Paddle) tool in PaddlePaddle program for converting other models with different formats to fluid model, and then you can use paddlejs-converter to get a Paddle.js model.

View File

@@ -0,0 +1,73 @@
[English](./README.md)
# paddlejs-converter
paddlejs-converter 是适用于 Paddle.js 的模型转换工具,其作用是将 PaddlePaddle 模型(或称为 fluid 模型转化为浏览器友好的格式以供Paddle.js在浏览器等环境中加载预测使用。此外paddlejs-converter 还提供了强大的模型优化能力,帮助开发者对模型结构进行优化,提高运行时性能。
## 1. 使用教程
### 1.1. 环境搭建
#### Python 版本确认
确认运行平台的 Python 环境与版本是否满足要求,若使用 Python3 ,则可能需要将后续命令中的 `python` 换成 `python3`
- Python3 3.5.1+ / 3.6 / 3.7
- Python2 2.7.15+
#### 安装虚拟环境
*由于开发环境可能安装了多个版本的 Python相关依赖包可能存在不同的版本为避免产生冲突**强烈建议**使用 Python 虚拟环境执行转换工具所需的各项命令,以免产生各种问题。若不使用虚拟环境或已安装虚拟环境,可跳过该步骤。*
以 Anaconda 为例:
前往 [Anaconda](https://www.anaconda.com/) 主页选择对应平台、Python 版本的 Anaconda 按照官方提示,进行安装;
安装完毕后在命令行执行以下命令创建Python 虚拟环境:
``` bash
conda create --name <your_env_name>
```
执行以下命令,切换至虚拟环境
``` bash
# Linux 或 macOS下请执行
source activate <your_env_name>
# Windows 下请执行
activate <your_env_name>
```
#### 安装依赖
- 如果`不需要`使用优化模型的能力,执行命令:
``` bash
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
```
- 如果`需要`使用优化模型的能力,执行命令:
``` bash
python -m pip install paddlepaddle paddlelite==2.6.0 -i https://mirror.baidu.com/pypi/simple
```
### 1.2. 快速上手
- 如果待转换的 fluid 模型为`合并参数文件`,即一个模型对应一个参数文件:
``` bash
python convertToPaddleJSModel.py --modelPath=<fluid_model_file_path> --paramPath=<fluid_param_file_path> --outputDir=<paddlejs_model_directory>
```
- 如果待转换的 fluid 模型为`分片参数文件`,即一个模型文件对应多个参数文件:
``` bash
# 注意,使用这种方式调用转换器,需要保证 inputDir 中,模型文件名为'__model__'
python convertToPaddleJSModel.py --inputDir=<fluid_model_directory> --outputDir=<paddlejs_model_directory>
````
模型转换器将生成以下两种类型的文件以供 Paddle.js 使用:
- model.json (模型结构与参数清单)
- chunk_\*.dat (二进制参数文件集合)
## 2. 详细文档
参数 | 描述
:-: | :-:
--inputDir | fluid 模型所在目录,当且仅当使用分片参数文件时使用该参数,将忽略 `modelPath` 和 `paramPath` 参数,且模型文件名必须为`__model__`
--modelPath | fluid 模型文件所在路径,使用合并参数文件时使用该参数
--paramPath | fluid 参数文件所在路径,使用合并参数文件时使用该参数
--outputDir | `必要参数` Paddle.js 模型输出路径
--disableOptimize | 是否关闭模型优化, `1` 为关闭优化,`0` 为开启优化(需安装 PaddleLite ),默认执行优化
--logModelInfo | 是否打印模型结构信息, `0` 为不打印, `1` 为打印,默认不打印
--sliceDataSize | 分片输出 Paddle.js 参数文件时每片文件的大小单位KB默认 4096
--useGPUOpt | 是否开启模型 GPU 优化,默认不开启(当模型准备运行在 webgl/webgpu 计算方案时,可以设置为 True 开启,在 wasm/plainjs 方案,则不用开启)
## 3. 其他信息
若需要转换的模型为 `TensorFlow/Caffe/ONNX` 格式,可使用 PaddlePaddle 项目下的 `X2Paddle`工具,将其他格式的模型转为 fluid 模型后,再使用本工具转化为 Paddle.js 模型。
详细请参考 [X2Paddle 项目](https://github.com/PaddlePaddle/X2Paddle)

View File

@@ -0,0 +1,29 @@
# PaddleJsConverter
## Installation
System Requirements:
* paddlepaddle >= 2.0.0
* paddlejslite >= 0.0.2
* Python3 3.5.1+ / 3.6 / 3.7
* Python2 2.7.15+
#### Install PaddleJsConverter
<img src="https://img.shields.io/pypi/v/paddlejsconverter" alt="version">
```shell
pip install paddlejsconverter
# or
pip3 install paddlejsconverter
```
## Usage
```shell
paddlejsconverter --modelPath=user_model_path --paramPath=user_model_params_path --outputDir=model_saved_path --useGPUOpt=True
```
注意useGPUOpt 选项默认不开启,如果模型用在 gpu backendwebgl/webgpu则开启 useGPUOpt如果模型运行在wasm/plain js则不要开启。

View File

@@ -0,0 +1,82 @@
# RNN算子计算过程
## 一、RNN理解
**RNN** 是循环神经网络,由输入层、隐藏层和输出层组成,擅长对序列数据进行处理。
![RNN](https://user-images.githubusercontent.com/43414102/144739164-d6c4b9ff-d885-4812-8d05-5bf045d3a11b.png)
paddle官网文档https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/RNN_cn.html#rnn
paddle源码实现https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rnn_op.h#L812
##二、RNN计算方式
t 时刻,输入层为 ![图片](https://paddlejs.bj.bcebos.com/doc/xt.svg) ,隐藏层为 ![图片](https://paddlejs.bj.bcebos.com/doc/st.svg) ,输出层为 ![图片](https://paddlejs.bj.bcebos.com/doc/ot.svg) 。由上图可知,![图片](https://paddlejs.bj.bcebos.com/doc/st.svg) 的值不仅仅取决于 ![图片](https://paddlejs.bj.bcebos.com/doc/xt.svg) ,还取决于 ![图片](https://paddlejs.bj.bcebos.com/doc/st1.svg) 。计算公式如下:
![RNN公式](https://user-images.githubusercontent.com/43414102/144739185-92724c8c-25f7-4559-9b1d-f1d76e65d965.jpeg)
## 三、pdjs中RNN算子实现
因为 RNN 有梯度消失问题,不能获取更多上下文信息,所以 CRNN 中使用的是 **LSTMLong Short Term Memory**LSTM 是一种特殊的 RNN能够保存长期的依赖关系。
基于图像的序列,两个方向的上下文是相互有用且互补的。由于 LSTM 是单向的,所以将两个 LSTM一个向前和一个向后组合到一个**双向 LSTM** 中。此外,可以堆叠多层双向 LSTM。ch_PP-OCRv2_rec_infer 识别模型就是使用的双层双向 LSTM 结构。计算过程如下图所示:
#### 以ch_ppocr_mobile_v2.0_rec_infer 模型 rnn算子为例
```javascript
{
Attr: {
mode: 'LSTM'
// 是否双向为true则正向反向都需要遍历
is_bidirec: true
// 隐藏层层数,代表循环次数
num_layers: 2
}
Input: [
transpose_1.tmp_0[25, 1, 288]
]
PreState: [
fill_constant_batch_size_like_0.tmp_0[4, 1, 48],
fill_constant_batch_size_like_1.tmp_0[4, 1, 48]
]
WeightList: [
lstm_cell_0.w_0[192, 288], lstm_cell_0.w_1[192, 48],
lstm_cell_1.w_0[192, 288], lstm_cell_1.w_1[192, 48],
lstm_cell_2.w_0[192, 96], lstm_cell_2.w_1[192, 48],
lstm_cell_3.w_0[192, 96], lstm_cell_3.w_1[192, 48],
lstm_cell_0.b_0[192], lstm_cell_0.b_1[192],
lstm_cell_1.b_0[192], lstm_cell_1.b_1[192],
lstm_cell_2.b_0[192], lstm_cell_2.b_1[192],
lstm_cell_3.b_0[192], lstm_cell_3.b_1[192]
]
Output: [
lstm_0.tmp_0[25, 1, 96]
]
}
```
#### 整体计算过程
![LSTM计算过程](https://user-images.githubusercontent.com/43414102/144739246-daf839ad-1d96-4e1d-8f34-38ed0bc5f288.png)
#### rnn 计算中新增op
1rnn_origin
计算公式: blas.MatMul(Input, WeightList_ih, blas_ih) + blas.MatMul(PreState, WeightList_hh, blas_hh)
2rnn_matmul
计算公式rnn_matmul = rnn_origin + Matmul( $ S_{t-1} $, WeightList_hh)
3rnn_cell
计算方式将rnn_matmul op输出结果分割成4份每份执行不同激活函数计算最后输出lstm_x_y.tmp_c[1, 1, 48]。x∈[0, 3]y∈[0, 24]。
详见算子实现:[rnn_cell](../paddlejs-backend-webgl/src/ops/shader/rnn/rnn_cell.ts)
)
4rnn_hidden
计算方式将rnn_matmul op输出结果分割成4份每份执行不同激活函数计算最后输出lstm_x_y.tmp_h[1, 1, 48]。x∈[0, 3]y∈[0, 24]。
详见算子实现:[rnn_hidden](../paddlejs-backend-webgl/src/ops/shader/rnn/rnn_hidden.ts)

View File

@@ -0,0 +1,558 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import json
import collections
import math
import sys
import os
import struct
import argparse
import shutil
import stat
import traceback
import numpy as np
import paddle.fluid as fluid
import paddle as paddle
import copy
from functools import reduce
import rnn
from pruningModel import pruningNoSenseTensor
from fuseOps import opListFuse
# 输入模型所在目录
modelDir = None
# 输入模型名
modelName = None
# 输入参数名当且仅当所有模型参数被保存在一个单独的二进制文件中它才需要被指定若为分片模型请设置为None
paramsName = None
# 是否打印模型信息
enableLogModelInfo = False
# 输出模型目录
outputDir = None
# 分片文件大小单位KB
sliceDataSize = 4 * 1024
# paddlepaddle运行程序实例
program = None
# 存放模型结构
modelInfo = {"vars": {}, "ops": [], "chunkNum": 0, "dataLayout": "nchw", "feedShape": None}
# 存放参数数值(未排序)
paramValuesDict = {}
# 有一些后置算子适合在cpu中运行所以单独统计
postOps = []
# 在转换过程中新生成的、需要添加到vars中的variable
appendedVarList = []
# rnn op索引列表
rnnList = []
# 转换模型中需要过滤掉的参数
needFilterAttributes = ['op_callstack', 'col', 'op_role', 'op_namescope', 'op_role_var',
'data_format', 'is_test', 'use_mkldnn', 'use_cudnn', 'use_quantizer', 'workspace_size_MB',
'mkldnn_data_type', 'op_device', '__@kernel_type_attr@__']
class ObjDict(dict):
"""
Makes a dictionary behave like an object,with attribute-style access.
"""
def __getattr__(self,name):
try:
return self[name]
except:
raise AttributeError(name)
def __setattr__(self,name,value):
self[name]=value
def validateShape(shape, name):
"""检验shape长度超过4则截断"""
if len(shape) > 4:
newShape = shape[-4:]
print('\033[31m ' + name + ' tensor shape length > 4, 处理为丢弃头部shape \033[0m')
return newShape
return shape
def splitLargeNum(x):
"""将x拆分成两个因数相乘"""
# 获取最小值
num = math.floor(math.sqrt(x))
while (num):
if x % num == 0:
return [num, int(x / num)]
num -= 1
return [1, x]
def logModel(info):
""" 打印信息 """
if enableLogModelInfo:
print(info)
def sortDict(oldDict, reverse=False):
""" 对字典进行排序,返回有序字典,默认升序 """
# 获得排序后的key list
keys = sorted(oldDict.keys(), reverse=reverse)
orderDict = collections.OrderedDict()
# 遍历 key 列表
for key in keys:
orderDict[key] = oldDict[key]
return orderDict
def dumpModelToJsonFile(outputDir):
""" 导出模型数据到json文件 """
print("Dumping model structure to json file...")
if not os.path.exists(outputDir):
os.makedirs(outputDir)
outputModelPath = os.path.join(outputDir, "model.json")
with open(outputModelPath, 'w') as outputFile:
json.dump(modelInfo, outputFile, indent=4, separators=(", ", ": "), sort_keys=True)
print("Dumping model structure to json file successfully")
def sliceDataToBinaryFile(paramValueList, outputDir):
""" 将参数数据分片输出到文件默认分片策略为按4M分片 """
totalParamValuesCount = len(paramValueList)
countPerSlice = int(sliceDataSize * 1024 / 4)
if not os.path.exists(outputDir):
os.makedirs(outputDir)
currentChunkIndex = 0
currentParamDataIndex = 0
while currentParamDataIndex < totalParamValuesCount - 1:
remainCount = totalParamValuesCount - currentParamDataIndex
if remainCount < countPerSlice:
countPerSlice = remainCount
chunkPath = os.path.join(outputDir, 'chunk_%s.dat' % (currentChunkIndex + 1))
file = open(chunkPath, 'wb')
for i in paramValueList[currentParamDataIndex : currentParamDataIndex + countPerSlice]:
byte = struct.pack('f', float(i))
file.write(byte)
file.close()
currentParamDataIndex = currentParamDataIndex + countPerSlice
currentChunkIndex = currentChunkIndex + 1
print("Output No." + str(currentChunkIndex)+ " binary file, remain " + str(totalParamValuesCount - currentParamDataIndex) + " param values.")
print("Slicing data to binary files successfully. (" + str(currentChunkIndex)+ " output files and " + str(currentParamDataIndex) + " param values)")
def reorderParamsValue():
""" 对参数文件中的数值按照variable.name字母序排序返回排序后组合完成的value list """
paramValuesOrderDict = sortDict(paramValuesDict)
paramValues = []
for value in paramValuesOrderDict.values():
paramValues += value
return paramValues
def mapToPaddleJSTypeName(fluidOPName):
""" 处理fluid的OP type与PaddleJS的OP type不对应情况 """
if fluidOPName == "batch_norm":
return "batchnorm"
return fluidOPName
def excludeNegativeShape(shape):
varShape = list(shape)
varShapeExcludeNegativeOne = []
for s in varShape:
# 模型中 ?会自动转为 -1需要单独处理成 1
if s == -1:
s = 1
varShapeExcludeNegativeOne.append(s)
return varShapeExcludeNegativeOne
def organizeModelVariableInfo(result):
""" 组织参数信息 """
print("Organizing model variables info...")
index = 0
# 存放var信息未排序
varInfoDict = {}
# 获取program中所有的var遍历并获取所有未排序的var信息和参数数值
vars = list(program.list_vars())
for v in vars:
# 跳过feed和fetch
if "feed" == v.name:
continue
if "fetch" == v.name:
continue
varShape = excludeNegativeShape(v.shape)
# FIXME:end
# 存放variable信息在dump成json时排序
varInfo = {}
varInfo["shape"] = varShape
# 数据是否是持久化数据如tensor为持久化数据op的output不是持久化数据
# 只输出持久化数据paddlejs中也仅读取持久化数据
varInfo["persistable"] = v.persistable
varInfoDict[v.name] = varInfo
logModel("[Var index:" + str(index) + " name:" + v.name + "]")
jsonDumpsIndentStr = json.dumps(varInfo, indent=2)
logModel(jsonDumpsIndentStr)
logModel("")
index += 1
# persistable数据存入paramValuesDict等待排序
if v.persistable:
tensor = np.array(fluid.global_scope().find_var(v.name).get_tensor())
data = tensor.flatten().tolist()
paramValuesDict[v.name] = data
# shape推断校正
feed_target_names = result[1]
fetch_targets = result[2]
# 获取输入shape
feedData = {}
feeded_vars = [program.global_block().vars[varname] for varname in feed_target_names]
for feedItem in feeded_vars:
curShape = feedItem.shape
feedName = feedItem.name
feedData[feedName] = np.full(excludeNegativeShape(curShape), 1.0, "float32")
for v in program.list_vars():
if not v.persistable:
v.persistable = True
exe.run(program, feed=feedData, fetch_list=fetch_targets, return_numpy=False)
for varKey in varInfoDict:
var = fluid.global_scope().find_var(varKey)
varData = np.array(var.get_tensor())
varShape = list(varData.shape)
varInfoDict[varKey]['shape'] = validateShape(varShape, varKey)
# vars追加
vars = modelInfo['vars']
for appendedVar in appendedVarList:
appendedName = appendedVar['name']
newName = appendedVar['new']
for curVarKey in varInfoDict:
if curVarKey == appendedName:
newVar = copy.deepcopy(varInfoDict[curVarKey])
varInfoDict[newName] = newVar
break
# 对var信息dict按照keyvar名进行字母顺序排序
varInfoOrderDict = sortDict(varInfoDict)
# 将var信息按照顺序添加到model info的vars中
for key, value in varInfoOrderDict.items():
value["name"] = key
modelInfo["vars"][key] = value
print("Organizing model variables info successfully.")
def organizeModelOpInfo():
""" 组织模型OP结构信息 """
print("Organizing model operators info...")
ops = program.current_block().ops
feedOutputName = None
index = 0
for op in ops:
opInfo = {}
# 获取OP type需要映射到PaddleJS的名字
opInfo["type"] = mapToPaddleJSTypeName(op.type)
opInputs = op.input_names
opOutputs = op.output_names
# 获取OP input
inputs = {}
for name in opInputs:
value = op.input(name)
if len(value) <= 0:
continue
if value[0] == feedOutputName:
# FIXME:workaround,PaddleJSfeed 输入必须是image且为单输入这里修改feed后面的OP的input为image建立前后关联
inputs[name] = ["image"]
else:
inputs[name] = value
opInfo["inputs"] = inputs
# 获取OP output
outputs = {}
# 将outputs转换为数组
if op.type == 'density_prior_box' or op.type == 'prior_box' or op.type == 'box_coder':
outputs['Out'] = []
for name in opOutputs:
value = op.output(name)
if len(value) <= 0:
continue
outputs['Out'].append(value[0])
else:
for name in opOutputs:
value = op.output(name)
if len(value) <= 0:
continue
if op.type == "feed":
# FIXME:workaround,PaddleJSfeed 输入必须是image且为单输入这里保存原始的输出名以便映射
feedOutputName = value[0]
outputs[name] = ["image"]
else:
outputs[name] = value
opInfo["outputs"] = outputs
# 收敛outputs[name]
if "Output" in opInfo["outputs"]:
opInfo["outputs"]["Out"] = opInfo["outputs"]["Output"]
del opInfo["outputs"]["Output"]
elif "Y" in opInfo["outputs"]:
opInfo["outputs"]["Out"] = opInfo["outputs"]["Y"]
del opInfo["outputs"]["Y"]
if "Out" not in opInfo["outputs"]:
print("\033[31moutputs[name] not exist Out.\033[0m")
sys.exit(1)
# 有的模型如人脸关键点会出现两个算子合并的情况如lmk_demoelementwise_add后接了relu算子relu的输入输出相等兼容一下
# inputs与outputs只有一个名称相等输入加后缀改上一层算子。
if 'X' in inputs and 'Out' in outputs:
curInputs = inputs['X']
curOutputs = outputs['Out']
if len(curInputs) == 1 and len(curOutputs) == 1 and curInputs[0] == curOutputs[0] and index > 1:
originName = curInputs[0]
changedName = inputs['X'][0] = curInputs[0] = originName + '_changed'
opInfo["inputs"]['X'] = curInputs
# 获取上一层算子
prevOpOutputs = modelInfo["ops"][index - 1]['outputs']
for name in prevOpOutputs:
values = prevOpOutputs[name]
for i, curName in enumerate(values):
if (curName == originName):
modelInfo["ops"][index - 1]['outputs'][name][i] = changedName
appendedVarList.append({'name': originName, 'new': changedName})
# 获取OP attribute
attrs = {}
for name in op.attr_names:
# 过滤不需要的参数
if name in needFilterAttributes:
continue
value = op.attr(name)
attrs[name] = value
opInfo["attrs"] = attrs
if (op.type == 'rnn'):
global rnnList
rnnList.append(index)
# multiclass_nms 单独处理
if (op.type.startswith('multiclass_nms')):
opInfo["type"] = 'multiclass_nms'
postOps.append(opInfo)
else:
# 存入modelInfo
modelInfo["ops"].append(opInfo)
logModel("[OP index:" + str(index) + " type:" + op.type + "]")
jsonDumpsIndentStr = json.dumps(opInfo, indent=2)
logModel(jsonDumpsIndentStr)
logModel("")
index += 1
print("Organizing model operators info successfully.")
def addChunkNumToJson(paramValueList):
totalParamValuesCount = len(paramValueList)
countPerSlice = int(sliceDataSize * 1024 / 4)
count = totalParamValuesCount / countPerSlice
modelInfo["chunkNum"] = math.ceil(count)
print("Model chunkNum set successfully.")
def appendConnectOp(fetch_targets):
targets = []
inputNames = []
totalShape = 0
# 从fetch_targets中提取输出算子信息
for target in fetch_targets:
name = target.name
curVar = fluid.global_scope().find_var(name)
curTensor = np.array(curVar.get_tensor())
shape = list(curTensor.shape)
totalShape += reduce(lambda x, y: x * y, shape)
targets.append({'name': name, 'shape': excludeNegativeShape(shape)})
inputNames.append(name)
# 构造connect算子
op = {
'attrs': {},
'inputs': {'X': inputNames},
'outputs': {'Out': ['connect_result']},
'type': 'connect'
}
# 构造输出var
outputVar = {'name': 'connect_result', 'shape': splitLargeNum(totalShape)}
ops = modelInfo['ops']
vars = modelInfo['vars']
# 收集要删除的算子index
delList = []
for index, item in enumerate(ops):
if item['type'] == 'fetch':
delList.append(index)
# 去除fetch算子
delCount = 0
for delIndex in delList:
del ops[delIndex - delCount]
delCount += 1
fetchOp = {
"attrs": {},
"inputs": {
"X": [
"connect_result"
]
},
"outputs": {
"Out": [
"fetch"
]
},
"type": "fetch"
}
ops.append(op)
ops.append(fetchOp)
vars['connect_result'] = outputVar
modelInfo['multiOutputs'] = targets
return targets
def genModelFeedShape(feed):
if len(feed) != 1:
print("\033[33;1mModel has more than one input feed.\033[0m")
return
originFeedShape = modelInfo['vars'][feed[0]]['shape']
feedShape = {}
if len(originFeedShape) == 3:
feedShape['fc'] = originFeedShape[0]
feedShape['fh'] = originFeedShape[1]
feedShape['fw'] = originFeedShape[2]
elif len(originFeedShape) == 4:
feedShape['fc'] = originFeedShape[1]
feedShape['fh'] = originFeedShape[2]
feedShape['fw'] = originFeedShape[3]
elif len(originFeedShape) == 2:
feedShape['fh'] = originFeedShape[0]
feedShape['fw'] = originFeedShape[1]
else:
print("\033[33;1mFeedShape length is " + str(len(originFeedShape)) + ".\033[0m")
return
modelInfo['feedShape'] = feedShape
print("\033[32mModel FeedShape set successfully.\033[0m")
def convertToPaddleJSModel(modelDir, modelName, paramsName, outputDir, useGPUOpt):
""" 转换fluid modle为paddleJS model """
#In PaddlePaddle 2.x, we turn on dynamic graph mode by default, and 'load_inference_model()' is only supported in static graph mode. So call 'paddle.enable_static()' before this api to enter static graph mode.
paddle.enable_static()
# 初始化fluid运行环境和配置
global exe
exe = fluid.Executor(fluid.CPUPlace())
result = fluid.io.load_inference_model(dirname=modelDir, executor=exe, model_filename=modelName, params_filename=paramsName)
global program
program = result[0]
fetch_targets = result[2]
feed_target_names = result[1]
# 获取program中所有的op按op顺序加入到model info
organizeModelOpInfo()
# 获取program中所有的var按照字母顺序加入到model info同时读取参数数值
organizeModelVariableInfo(result)
# 拆分rnn op
if len(rnnList):
for index in rnnList:
rnn.splice_rnn_op(modelInfo, index)
if useGPUOpt:
# 算子融合
modelInfo['gpuOpt'] = True
opListFuse(modelInfo['ops'])
# 对多输出模型追加connect算子
if len(fetch_targets) > 1:
appendConnectOp(fetch_targets)
if (postOps and len(postOps) > 0):
for op in postOps:
if (op['type'].startswith('multiclass_nms')):
inputNames = []
for input, value in op['inputs'].items():
if len(value) <= 0:
continue
cur = ObjDict()
cur.name = value[0]
inputNames.append(cur)
targets = appendConnectOp(inputNames)
# op['inputs'] = targets
keys = op['inputs'].keys()
for i, val in enumerate(keys):
op['inputs'][val] = targets[i]
modelInfo['postOps'] = postOps
# 对参数数值dict按照key参数名进行字母顺序排序并组合到一起
paramValues = reorderParamsValue()
# model.json 设置分片参数
addChunkNumToJson(paramValues)
# model.json 设置 feedShape 输入 shape 信息
genModelFeedShape(feed_target_names)
# 去掉无意义的 tensor 和对应 op
pruningNoSenseTensor(modelInfo)
# 导出模型文件到json
dumpModelToJsonFile(outputDir)
# 导出分片参数文件
sliceDataToBinaryFile(paramValues, outputDir)
def main():
global sliceDataSize
global enableLogModelInfo
try:
p = argparse.ArgumentParser(description='模型转换参数解析')
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数且模型文件名必须为`__model__`', required=False)
p.add_argument('--modelPath', help='fluid模型文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument('--paramPath', help='fluid参数文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument("--outputDir", help='paddleJS模型输出路径必要参数', required=True)
p.add_argument("--logModelInfo", type=int, default=0, help='是否输出模型结构信息非必要参数0为不输出1为输出默认不输出', required=False)
p.add_argument("--sliceDataSize", type=int, default=4096, help='分片输出参数文件时每片文件的大小单位KB非必要参数默认4096KB', required=False)
p.add_argument('--useGPUOpt', help='转换模型是否执行GPU优化方法', required=False)
args = p.parse_args()
modelDir = args.inputDir
modelPath = args.modelPath
paramPath = args.paramPath
useGPUOpt = args.useGPUOpt
if not modelDir:
modelDir, modelName = os.path.split(modelPath)
paramDir, paramsName = os.path.split(paramPath)
if paramDir != modelDir:
print("\033[31mModel and param file should be put in a same directory!\033[0m")
raise Exception()
outputDir = args.outputDir
sliceDataSize = args.sliceDataSize
if args.logModelInfo == 1:
enableLogModelInfo = True
convertToPaddleJSModel(modelDir, modelName, paramsName, outputDir, useGPUOpt)
except Exception as identifier:
print("\033[31mA fetal error occured. Failed to convert model.\033[0m")
print(traceback.format_exc())
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,141 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import sys
import os
import argparse
import shutil
import stat
import traceback
import copy
def cleanTempModel(optimizedModelTempDir):
""" 清理opt优化完的临时模型文件 """
if os.path.exists(optimizedModelTempDir):
print("Cleaning optimized temporary model...")
shutil.rmtree(optimizedModelTempDir, onerror=grantWritePermission)
def grantWritePermission(func, path, execinfo):
""" 文件授权 """
os.chmod(path, stat.S_IWRITE)
func(path)
def main():
"""
Example:
'python convertToPaddleJSModel.py --modelPath=../infer_model/MobileNetV2/model --paramPath=../infer_model/MobileNetV2/params --outputDir=../jsmodel'
"""
try:
p = argparse.ArgumentParser(description='转化为PaddleJS模型参数解析')
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数且模型文件名必须为`__model__`', required=False)
p.add_argument('--modelPath', help='fluid模型文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument('--paramPath', help='fluid参数文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument("--outputDir", help='paddleJS模型输出路径必要参数', required=True)
p.add_argument("--disableOptimize", type=int, default=0, help='是否关闭模型优化非必要参数1为关闭优化0为开启优化默认开启优化', required=False)
p.add_argument("--logModelInfo", type=int, default=0, help='是否输出模型结构信息非必要参数0为不输出1为输出默认不输出', required=False)
p.add_argument("--sliceDataSize", type=int, default=4096, help='分片输出参数文件时每片文件的大小单位KB非必要参数默认4096KB', required=False)
p.add_argument('--useGPUOpt', help='转换模型是否执行GPU优化方法', required=False)
args = p.parse_args()
# 获取当前用户使用的 python 解释器 bin 位置
pythonCmd = sys.executable
# TODO: 由于PaddleLite和PaddlePaddle存在包冲突因此将整个模型转换工具拆成两个python文件由一个入口python文件通过命令行调用
# 区分本地执行和命令行执行
if os.path.exists("optimizeModel.py"):
optimizeCmd = pythonCmd + " optimizeModel.py"
else:
optimizeCmd = "pdjsOptimizeModel"
if os.path.exists("convertModel.py"):
convertCmd = pythonCmd + " convertModel.py"
else:
convertCmd = "pdjsConvertModel"
inputDir = args.inputDir
modelPath = args.modelPath
paramPath = args.paramPath
outputDir = args.outputDir
disableOptimization = args.disableOptimize
args.disableOptimize = None
enableLogModelInfo = args.logModelInfo
sliceDataSize = args.sliceDataSize
optArgs = copy.deepcopy(args)
enableOptimization = 1 - disableOptimization
optimizedModelTempDir = None
if enableOptimization == 1:
optimizedModelTempDir = os.path.join(outputDir, "optimize")
optArgs.outputDir = optimizedModelTempDir
if inputDir:
args.inputDir = optimizedModelTempDir
else:
args.modelPath = os.path.join(optimizedModelTempDir, "model")
args.paramPath = os.path.join(optimizedModelTempDir, "params")
print("============Convert Model Args=============")
if inputDir:
print("inputDir: " + inputDir)
else:
print("modelPath: " + modelPath)
print("paramPath: " + paramPath)
print("outputDir: " + outputDir)
print("enableOptimizeModel: " + str(enableOptimization))
print("enableLogModelInfo: " + str(enableLogModelInfo))
print("sliceDataSize:" + str(sliceDataSize))
print("Starting...")
if enableOptimization:
print("Optimizing model...")
for param in ["inputDir", "modelPath", "paramPath", "outputDir"]:
if optArgs.__dict__[param]:
# 用""框起命令参数值,解决路径中的空格问题
optimizeCmd += " --" + param + "="+ '"' + str(optArgs.__dict__[param]) + '"'
os.system(optimizeCmd)
try:
os.listdir(optimizedModelTempDir)
except Exception as identifier:
print("\n\033[31mOptimizing model failed.\033[0m")
# restore inputDir or modelPath paramPath from optimize
if inputDir:
args.inputDir = inputDir
else:
args.modelPath = modelPath
args.paramPath = paramPath
else:
print("\n\033[32mOptimizing model successfully.\033[0m")
else:
print("\033[33mYou choosed not to optimize model, consequently, optimizing model is skiped.\033[0m")
print("\nConverting model...")
for param in args.__dict__:
if args.__dict__[param]:
# 用""框起参数,解决路径中的空格问题
convertCmd += " --" + param + "=" + '"' + str(args.__dict__[param]) + '"'
os.system(convertCmd)
try:
file = os.listdir(outputDir)
if len(file) == 0:
raise Exception
except Exception as identifier:
print("\033[31m============ALL DONE============\033[0m")
else:
if enableOptimization:
cleanTempModel(optimizedModelTempDir)
print("Temporary files has been deleted successfully.")
print("\033[32mConverting model successfully.\033[0m")
print("\033[32m============ALL DONE============\033[0m")
except Exception as identifier:
print("\033[31mA fetal error occured. Failed to convert model.\033[0m")
print(traceback.format_exc())
exit(1)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,75 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
def opListFuse(ops):
""" 算子融合 """
fuseOpList = [
'relu',
'relu6',
'leaky_relu',
'scale',
'sigmoid',
'hard_sigmoid',
'pow',
'sqrt',
'tanh',
'hard_swish',
'dropout'
]
# 判断op是否为单节点
def opExistSingleNode(opName):
name = opName
if name:
nodeNum = 0
for i in range(len(ops)):
op = ops[i]
if 'X' not in op['inputs']:
continue
inputName = op['inputs']['X']
for x in inputName:
if x == name:
nodeNum = nodeNum + 1
return True if nodeNum == 1 else False
else:
return False
for index in reversed(range(len(ops))):
if index > 0:
op = ops[index]
# 兼容paddlelite 算子融合字段
if 'act_type' in op['attrs']:
name = op['attrs']['act_type']
op['attrs']['fuse_opt'] = {}
op['attrs']['fuse_opt'][name] = {}
if name == 'hard_swish':
op['attrs']['fuse_opt'][name]['offset'] = op['attrs']['hard_swish_offset']
op['attrs']['fuse_opt'][name]['scale'] = op['attrs']['hard_swish_scale']
op['attrs']['fuse_opt'][name]['threshold'] = op['attrs']['hard_swish_threshold']
if name == 'relu6':
op['attrs']['fuse_opt'][name]['threshold'] = op['attrs']['fuse_brelu_threshold']
for fuse in fuseOpList:
if op['type'] == fuse:
prevOp = ops[index - 1]
if opExistSingleNode(prevOp['outputs']['Out'][0]) and len(prevOp['outputs']['Out']) == 1 :
prevOp['attrs']['fuse_opt'] = {}
if 'fuse_opt' in op['attrs']:
prevOp['attrs']['fuse_opt'] = op['attrs']['fuse_opt']
del op['attrs']['fuse_opt']
prevOp['attrs']['fuse_opt'][fuse] = op['attrs']
prevOp['outputs']['Out'] = op['outputs']['Out']
del ops[index]

View File

@@ -0,0 +1,53 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import collections
import argparse
import traceback
from paddlejslite import lite
import pkg_resources
from packaging import version
lite_version = pkg_resources.get_distribution("paddlelite").version
def optimizeModel(inputDir, modelPath, paramPath, outputDir):
""" 使用opt python接口执行模型优化 """
opt = lite.Opt()
if inputDir:
# 分片参数文件优化
opt.set_model_dir(inputDir)
else:
# 合并参数文件优化
opt.set_model_file(modelPath)
opt.set_param_file(paramPath)
opt.set_valid_places("arm")
opt.set_model_type("protobuf")
opt.set_optimize_out(outputDir)
opt.run()
def main():
try:
p = argparse.ArgumentParser('模型优化参数解析')
p.add_argument('--inputDir', help='fluid模型所在目录。当且仅当使用分片参数文件时使用该参数。将过滤modelPath和paramsPath参数且模型文件名必须为`__model__`', required=False)
p.add_argument('--modelPath', help='fluid模型文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument('--paramPath', help='fluid参数文件所在路径使用合并参数文件时使用该参数', required=False)
p.add_argument("--outputDir", help='优化后fluid模型目录必要参数', required=True)
args = p.parse_args()
inputDir = args.inputDir
modelPath = args.modelPath
paramPath = args.paramPath
outputDir = args.outputDir
optimizeModel(inputDir, modelPath, paramPath, outputDir)
except Exception as identifier:
print("\033[31mA fetal error occured. Failed to optimize model.\033[0m")
print(traceback.format_exc())
pass
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,64 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# pruning op tensor and relatad op with no sense, like ShapeTensor and OutSize
def pruningNoSenseTensor(model):
global ops
ops = model["ops"]
global vars
vars = model["vars"]
for op in ops[:]:
shapeTensor = op["inputs"].get("ShapeTensor")
outSizeTensor = op["inputs"].get("OutSize")
noSenseTensor = shapeTensor or outSizeTensor
if not noSenseTensor:
continue
print(noSenseTensor)
if shapeTensor:
del op["inputs"]["ShapeTensor"]
if outSizeTensor:
del op["inputs"]["OutSize"]
for tensorId in noSenseTensor:
delLeafOpWithoutChildren(tensorId)
# delete leaf op which has no child
def delLeafOpWithoutChildren(tensorId):
# judge if there is an op which used the tensor
for op in ops[:]:
inputsTensor = op["inputs"]
input = inputsTensor.get("Input") or inputsTensor.get("X")
if input and (tensorId in input):
return
op = getOpByOutputTensor(tensorId)
if not op:
return
# del op
ops.remove(op)
# del vars
del vars[tensorId]
# upward recursion
delOpinputsTensor = op["inputs"]
input = delOpinputsTensor.get("Input") or delOpinputsTensor.get("X")
if not input:
return
for inputTensorId in input:
delLeafOpWithoutChildren(inputTensorId)
# find op by output tensor id
def getOpByOutputTensor(tensorId):
for op in ops[:]:
outputTensor = op["outputs"]
out = outputTensor.get("Out") or outputTensor.get("Output") or outputTensor.get("Y")
if out[0] == tensorId:
return op

View File

@@ -0,0 +1,35 @@
#!/bin/bash
set -o errexit
set -o nounset
shopt -s extglob
if [ $# == 0 ];then
ENV="test"
else
ENV=$1
fi
# clear dist
rm -rf dist
# build
python3 setup.py sdist bdist_wheel
# publish
if [ ${ENV} == 'production' ];then
echo "publish to https://pypi.org"
python3 -m twine upload dist/*
else
echo "publish to https://test.pypi.org/"
python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*
fi
# del egg-info and build
rm -rf paddlejsconverter.egg-info
rm -rf build

View File

@@ -0,0 +1,273 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
def splice_rnn_op(model_info, rnn_index):
global input_shape
global weight_0_shape
global weight_1_shape
global rnn_input_name
ops = model_info['ops']
vars = model_info['vars']
op = ops[rnn_index]
rnn_input_name = op['inputs']['Input'][0]
rnn_output_name = op['outputs']['Out'][0]
is_bidirec = 2 if op['attrs']['is_bidirec'] else 1
num_layers = op['attrs']['num_layers']
hidden_size = op['attrs']['hidden_size']
layer_num = num_layers * is_bidirec
# concat input最大值
max_concat_num = 15
def concat_mul(index, list, num):
global rnn_input_name
end = len(list)
if end < max_concat_num:
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(num) + '.tmp_concat'
# 非最后一层遍历将concat作为下一层输入
if index < is_bidirec * num_layers - 1:
rnn_input_name = concat_output_name
# 最后一层遍历将rnn_output_name赋给最后一个concat
else:
concat_output_name = rnn_output_name
concat_op = {
'attrs': {
'axis': 0
},
'inputs': {
'X': []
},
'outputs': {'Out': [concat_output_name]},
'type': 'concat'
}
concat_output_shape = 0
for x in range(0, end):
x_input_name = 'lstm_' + str(index - 1) + '_' + str(list[x]) + '.tmp_concat'
concat_op['inputs']['X'].append(x_input_name)
concat_output_shape += vars[x_input_name]['shape'][0]
concat_var = {
'name': concat_output_name,
'persistable': False,
'shape': [concat_output_shape, 1, weight_1_shape[1] * 2]
}
ops.append(concat_op)
if index < is_bidirec * num_layers - 1:
vars[concat_output_name] = concat_var
return
# concat新列表
new_list = []
for i in range(0, end, max_concat_num):
if i + max_concat_num > end:
for n in range(i, end):
new_list.append(list[n])
break
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(num) + '.tmp_concat'
# concat_list长度为max_concat_num && 最后一层遍历将rnn_output_name赋给最后一个concat
if end == max_concat_num and index == is_bidirec * num_layers - 1:
concat_output_name = rnn_output_name
concat_op = {
'attrs': {
'axis': 0
},
'inputs': {
'X': []
},
'outputs': {'Out': [concat_output_name]},
'type': 'concat'
}
concat_output_shape = 0
for x in range(0, max_concat_num):
x_input_name = 'lstm_' + str(index - 1) + '_' + str(list[i + x]) + '.tmp_concat'
concat_op['inputs']['X'].append(x_input_name)
concat_output_shape += vars[x_input_name]['shape'][0]
concat_var = {
'name': concat_output_name,
'persistable': False,
'shape': [concat_output_shape, 1, weight_1_shape[1] * 2]
}
ops.append(concat_op)
vars[concat_output_name] = concat_var
new_list.append(num)
# 若concat_list长度为max_concat_num在下一次递归时直接修改rnn_input_name结束递归num无需+1
if end != max_concat_num:
num += 1
concat_mul(index, new_list, num)
for index in range(layer_num):
last_hidden = op['inputs']['PreState'][0]
last_cell = op['inputs']['PreState'][1]
weight_list_0 = op['inputs']['WeightList'][index * 2]
weight_list_1 = op['inputs']['WeightList'][index * 2 + 1]
weight_list_2 = op['inputs']['WeightList'][(index + num_layers * is_bidirec) * 2]
weight_list_3 = op['inputs']['WeightList'][(index + num_layers * is_bidirec) * 2 + 1]
output_name = 'rnn_origin_' + str(index)
input_shape = vars[rnn_input_name]['shape']
batch = input_shape[0]
if vars[weight_list_0]:
weight_0_shape = vars[weight_list_0]['shape']
if vars[weight_list_1]:
weight_1_shape = vars[weight_list_1]['shape']
if batch == 0:
continue
origin_op = {
'attrs': {
'state_axis': index
},
'inputs': {
'Input': [rnn_input_name],
'PreState': [last_hidden],
'WeightList': [
weight_list_0,
weight_list_1,
weight_list_2,
weight_list_3
]
},
'outputs': {'Out': [output_name]},
'type': 'rnn_origin'
}
origin_var = {
'name': output_name,
'persistable': False,
'shape': [input_shape[0], input_shape[1], weight_0_shape[0]]
}
ops.append(origin_op)
vars[output_name] = origin_var
for bat in range(batch):
matmul_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_matmul'
cell_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_c'
hidden_output_name = 'lstm_' + str(index) + '_' + str(bat) + '.tmp_h'
matmul_op = {
'attrs': {
'input_axis': bat,
'state_axis': index if bat == 0 else 0,
'batch': batch,
'reverse': False if index % 2 == 0 else True
},
'inputs': {
'Input': [output_name],
'PreState': [last_hidden],
'WeightList': [weight_list_1]
},
'outputs': {'Out': [matmul_output_name]},
'type': 'rnn_matmul'
}
matmul_var = {
'name': matmul_output_name,
'persistable': False,
'shape': [1, 1, weight_0_shape[0]]
}
ops.append(matmul_op)
vars[matmul_output_name] = matmul_var
cell_op = {
'attrs': {
'state_axis': index if bat == 0 else 0,
'hidden_size': hidden_size
},
'inputs': {
'X': [matmul_output_name],
'Y': [last_cell]
},
'outputs': {'Out': [cell_output_name]},
'type': 'rnn_cell'
}
cell_var = {
'name': cell_output_name,
'persistable': False,
'shape': [1, 1, weight_1_shape[1]]
}
ops.append(cell_op)
vars[cell_output_name] = cell_var
hidden_op = {
'attrs': {
'state_axis': index if bat == 0 else 0,
'hidden_size': hidden_size
},
'inputs': {
'X': [matmul_output_name],
'Y': [last_cell]
},
'outputs': {'Out': [hidden_output_name]},
'type': 'rnn_hidden'
}
hidden_var = {
'name': hidden_output_name,
'persistable': False,
'shape': [1, 1, weight_1_shape[1]]
}
ops.append(hidden_op)
vars[hidden_output_name] = hidden_var
last_hidden = hidden_output_name
last_cell = cell_output_name
# concat
if index % 2 == 1:
concat_list = []
concat_num = 0
# concat forword and backword
for bat in range(batch):
x_input_name_0 = 'lstm_' + str(index - 1) + '_' + str(bat) + '.tmp_h'
x_input_name_1 = 'lstm_' + str(index) + '_' + str(batch - bat - 1) + '.tmp_h'
concat_output_name = 'lstm_' + str(index - 1) + '_' + str(bat) + '.tmp_concat'
concat_op = {
'attrs': {
'axis': 2
},
'inputs': {
'X': [x_input_name_0, x_input_name_1]
},
'outputs': {'Out': [concat_output_name]},
'type': 'concat'
}
concat_var = {
'name': concat_output_name,
'persistable': False,
'shape': [1, 1, weight_1_shape[1] * 2]
}
ops.append(concat_op)
vars[concat_output_name] = concat_var
concat_list.append(bat)
concat_num += 1
concat_mul(index, concat_list, concat_num)
# 删除rnn op
del ops[rnn_index]

View File

@@ -0,0 +1,37 @@
import setuptools
PY_MODILES = ["convertToPaddleJSModel", "convertModel", "optimizeModel", "pruningModel", "rnn", "fuseOps"]
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="paddlejsconverter",
version="1.0.7",
author="paddlejs",
author_email="382248373@qq.com",
description="Paddlejs model converter",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/PaddlePaddle/Paddle.js",
py_modules=PY_MODILES,
classifiers=[
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
install_requires=[
"paddlepaddle >= 2.0.0",
"paddlejslite >= 0.0.2",
"numpy"
],
entry_points={
"console_scripts": [
"paddlejsconverter = convertToPaddleJSModel:main",
"pdjsConvertModel = convertModel:main",
"pdjsOptimizeModel = optimizeModel:main"
]
}
)

View File

@@ -0,0 +1,3 @@
module.exports = {
extends: ['@commitlint/config-conventional'], // 使用预设的配置 https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/config-conventional/index.js
}

View File

@@ -0,0 +1,15 @@
/* eslint-env node */
require("@rushstack/eslint-patch/modern-module-resolution");
module.exports = {
root: true,
extends: [
"plugin:vue/vue3-essential",
"eslint:recommended",
"@vue/eslint-config-typescript",
"@vue/eslint-config-prettier",
],
parserOptions: {
ecmaVersion: "latest",
},
};

View File

@@ -0,0 +1,31 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
.DS_Store
dist
dist-ssr
coverage
*.local
/cypress/videos/
/cypress/screenshots/
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
.yalc
yalc.lock

View File

@@ -0,0 +1 @@
{}

View File

@@ -0,0 +1,81 @@
# Paddle.js-demo
该分支是 Paddle.js 应用的Demo集合可以查看各个 Demo 的实现代码。
## Demo 目录
| 分类 | 名称 | 目录 |
|:----:| :--------------- | -------------------------------------------------------- |
| CV | 人像扣图 | /src/views/cv/segmentation/PortraitMatting |
| CV | 人像分割背景替换 | /src/views/cv/segmentation/PortraitBackgroundReplacement |
| CV | 手势识别AI猜丁壳 | /src/views/cv/recognition/GestureRecognition |
| CV | 1000种物品识别 | /src/views/cv/recognition/ItemIdentification |
| CV | 酒瓶识别 | /src/views/cv/recognition/WineBottleIdentification |
| CV | 文本检测 | /src/views/cv/ocr/TextDetection |
| CV | 文本识别 | /src/views/cv/ocr/TextRecognition |
## 开发简介
### 安装依赖
```sh
npm install
```
### 开发
```sh
npm run dev
```
### 构建
```sh
npm run build
```
### [ESLint](https://eslint.org/) 格式化
```sh
npm run lint
```
### 工程风格
1. 项目使用TypeScript
2. 推荐使用 Vue 的组合式 API可以根据 'src/views/ExampleFile.vue' 模板创建新的组件
3. CSS 使用 Less
4. eslint 使用的是 Vue 推荐的,一般情况请尽量符合对应的要求
5. store 使用的是 [Pinia](https://pinia.web3doc.top/)
6. router 使用的是 [vue-router](https://router.vuejs.org/zh/)
### src 目录简介
```text
├─assets 资源文件
├─components 全局组件
├─router 路由
├─stores 存储库
└─views 视图
└─cv cv相关demo
├─ocr ocr相关demo
│ ├─TextDetection
│ └─TextRecognition
├─...
├─recognition 识别相关demo
│ ├─GestureRecognition
│ ├─ItemIdentification
│ ├─...
│ └─WineBottleIdentification
└─segmentation 分割相关demo
├─PortraitBackgroundReplacement
├─...
└─PortraitMatting
```
新增组件在对应类别下新增即可,可以参考模板 'src/views/ExampleFile.vue'
### 提交相关
1. 提交前会自动进行全部文件的 ESLint 格式化
2. commit 格式需要遵守 `<type>(<scope>): <subject>`,会自动进行检查

View File

@@ -0,0 +1,5 @@
// Generated by 'unplugin-auto-import'
export {}
declare global {
}

View File

@@ -0,0 +1,35 @@
// generated by unplugin-vue-components
// We suggest you to commit this file into source control
// Read more: https://github.com/vuejs/core/pull/3399
import '@vue/runtime-core'
export {}
declare module '@vue/runtime-core' {
export interface GlobalComponents {
ElAside: typeof import('element-plus/es')['ElAside']
ElButton: typeof import('element-plus/es')['ElButton']
ElCard: typeof import('element-plus/es')['ElCard']
ElCol: typeof import('element-plus/es')['ElCol']
ElContainer: typeof import('element-plus/es')['ElContainer']
ElDialog: typeof import('element-plus/es')['ElDialog']
ElFooter: typeof import('element-plus/es')['ElFooter']
ElHeader: typeof import('element-plus/es')['ElHeader']
ElInput: typeof import('element-plus/es')['ElInput']
ElMain: typeof import('element-plus/es')['ElMain']
ElMenu: typeof import('element-plus/es')['ElMenu']
ElMenuItem: typeof import('element-plus/es')['ElMenuItem']
ElRow: typeof import('element-plus/es')['ElRow']
ElSubMenu: typeof import('element-plus/es')['ElSubMenu']
HelloWorld: typeof import('./src/components/HelloWorld.vue')['default']
IconCommunity: typeof import('./src/components/icons/IconCommunity.vue')['default']
IconDocumentation: typeof import('./src/components/icons/IconDocumentation.vue')['default']
IconEcosystem: typeof import('./src/components/icons/IconEcosystem.vue')['default']
IconSupport: typeof import('./src/components/icons/IconSupport.vue')['default']
IconTooling: typeof import('./src/components/icons/IconTooling.vue')['default']
RouterLink: typeof import('vue-router')['RouterLink']
RouterView: typeof import('vue-router')['RouterView']
TheWelcome: typeof import('./src/components/TheWelcome.vue')['default']
WelcomeItem: typeof import('./src/components/WelcomeItem.vue')['default']
}
}

View File

@@ -0,0 +1 @@
/// <reference types="vite/client" />

View File

@@ -0,0 +1,52 @@
{
"name": "paddle-js-demo",
"version": "1.0.0",
"scripts": {
"dev": "vite",
"build": "run-p type-check build-only",
"preview": "vite preview --port 4173",
"build-only": "vite build",
"type-check": "vue-tsc --noEmit",
"lint": "eslint . --ext .vue,.js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --fix --ignore-path .gitignore"
},
"dependencies": {
"@antv/x6": "^1.34.2",
"@paddle-js-models/detect": "^3.0.0",
"@paddle-js-models/facedetect": "^3.0.0",
"@paddle-js-models/gesture": "^3.0.0",
"@paddle-js-models/humanseg": "^3.0.0",
"@paddle-js-models/humanseg_gpu": "^3.0.0",
"@paddle-js-models/mobilenet": "^3.0.0",
"@paddle-js-models/ocr": "3.1.0",
"@paddle-js-models/ocrdet": "3.1.0",
"@paddlejs/paddlejs-backend-webgl": "^1.2.9",
"@paddlejs/paddlejs-core": "^2.2.0",
"element-plus": "^2.2.16",
"less": "^4.1.3",
"less-loader": "^11.0.0",
"pinia": "^2.0.21",
"vue": "^3.2.38",
"vue-router": "^4.1.5"
},
"devDependencies": {
"@commitlint/cli": "^17.1.2",
"@commitlint/config-conventional": "^17.1.0",
"@rushstack/eslint-patch": "^1.1.4",
"@types/node": "^16.11.56",
"@vitejs/plugin-vue": "^3.0.3",
"@vue/eslint-config-prettier": "^7.0.0",
"@vue/eslint-config-typescript": "^11.0.0",
"@vue/tsconfig": "^0.1.3",
"commitlint": "^17.1.2",
"eslint": "8.5.0",
"eslint-plugin-vue": "^9.3.0",
"husky": "^8.0.1",
"npm-run-all": "^4.1.5",
"prettier": "^2.7.1",
"typescript": "~4.7.4",
"unplugin-auto-import": "^0.11.2",
"unplugin-vue-components": "^0.22.4",
"vite": "^3.0.9",
"vue-tsc": "^0.40.7"
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

View File

@@ -0,0 +1,74 @@
/* color palette from <https://github.com/vuejs/theme> */
:root {
--vt-c-white: #ffffff;
--vt-c-white-soft: #f8f8f8;
--vt-c-white-mute: #f2f2f2;
--vt-c-black: #181818;
--vt-c-black-soft: #222222;
--vt-c-black-mute: #282828;
--vt-c-indigo: #2c3e50;
--vt-c-divider-light-1: rgba(60, 60, 60, 0.29);
--vt-c-divider-light-2: rgba(60, 60, 60, 0.12);
--vt-c-divider-dark-1: rgba(84, 84, 84, 0.65);
--vt-c-divider-dark-2: rgba(84, 84, 84, 0.48);
--vt-c-text-light-1: var(--vt-c-indigo);
--vt-c-text-light-2: rgba(60, 60, 60, 0.66);
--vt-c-text-dark-1: var(--vt-c-white);
--vt-c-text-dark-2: rgba(235, 235, 235, 0.64);
}
/* semantic color variables for this project */
:root {
--color-background: var(--vt-c-white);
--color-background-soft: var(--vt-c-white-soft);
--color-background-mute: var(--vt-c-white-mute);
--color-border: var(--vt-c-divider-light-2);
--color-border-hover: var(--vt-c-divider-light-1);
--color-heading: var(--vt-c-text-light-1);
--color-text: var(--vt-c-text-light-1);
--section-gap: 160px;
}
@media (prefers-color-scheme: dark) {
:root {
--color-background: var(--vt-c-black);
--color-background-soft: var(--vt-c-black-soft);
--color-background-mute: var(--vt-c-black-mute);
--color-border: var(--vt-c-divider-dark-2);
--color-border-hover: var(--vt-c-divider-dark-1);
--color-heading: var(--vt-c-text-dark-1);
--color-text: var(--vt-c-text-dark-2);
}
}
*,
*::before,
*::after {
box-sizing: border-box;
margin: 0;
position: relative;
font-weight: normal;
}
body {
min-height: 100vh;
color: var(--color-text);
background: var(--color-background);
transition: color 0.5s, background-color 0.5s;
line-height: 1.6;
font-family: Inter, -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu,
Cantarell, 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
font-size: 15px;
text-rendering: optimizeLegibility;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 261.76 226.69" xmlns:v="https://vecta.io/nano"><path d="M161.096.001l-30.225 52.351L100.647.001H-.005l130.877 226.688L261.749.001z" fill="#41b883"/><path d="M161.096.001l-30.225 52.351L100.647.001H52.346l78.526 136.01L209.398.001z" fill="#34495e"/></svg>

After

Width:  |  Height:  |  Size: 308 B

View File

@@ -0,0 +1,8 @@
@import "./base.css";
#app {
height: 100vh;
width: 100%;
margin: 0 auto;
padding: 2rem;
}

View File

@@ -0,0 +1,41 @@
<script setup lang="ts">
defineProps<{
msg: string;
}>();
</script>
<template>
<div class="greetings">
<h1 class="green">{{ msg }}</h1>
<h3>
Youve successfully created a project with
<a href="https://vitejs.dev/" target="_blank" rel="noopener">Vite</a> +
<a href="https://vuejs.org/" target="_blank" rel="noopener">Vue 3</a>.
What's next?
</h3>
</div>
</template>
<style scoped>
h1 {
font-weight: 500;
font-size: 2.6rem;
top: -10px;
}
h3 {
font-size: 1.2rem;
}
.greetings h1,
.greetings h3 {
text-align: center;
}
@media (min-width: 1024px) {
.greetings h1,
.greetings h3 {
text-align: left;
}
}
</style>

View File

@@ -0,0 +1,121 @@
<script setup lang="ts">
import WelcomeItem from "./WelcomeItem.vue";
import DocumentationIcon from "./icons/IconDocumentation.vue";
import ToolingIcon from "./icons/IconTooling.vue";
import EcosystemIcon from "./icons/IconEcosystem.vue";
import CommunityIcon from "./icons/IconCommunity.vue";
import SupportIcon from "./icons/IconSupport.vue";
</script>
<template>
<WelcomeItem>
<template #icon>
<DocumentationIcon />
</template>
<template #heading>Documentation</template>
Vues
<a href="https://vuejs.org/" target="_blank" rel="noopener"
>official documentation</a
>
provides you with all information you need to get started.
</WelcomeItem>
<WelcomeItem>
<template #icon>
<ToolingIcon />
</template>
<template #heading>Tooling</template>
This project is served and bundled with
<a
href="https://vitejs.dev/guide/features.html"
target="_blank"
rel="noopener"
>Vite</a
>. The recommended IDE setup is
<a href="https://code.visualstudio.com/" target="_blank" rel="noopener"
>VSCode</a
>
+
<a
href="https://github.com/johnsoncodehk/volar"
target="_blank"
rel="noopener"
>Volar</a
>. If you need to test your components and web pages, check out
<a href="https://www.cypress.io/" target="_blank" rel="noopener">Cypress</a>
and
<a href="https://on.cypress.io/component" target="_blank"
>Cypress Component Testing</a
>.
<br />
More instructions are available in <code>README.md</code>.
</WelcomeItem>
<WelcomeItem>
<template #icon>
<EcosystemIcon />
</template>
<template #heading>Ecosystem</template>
Get official tools and libraries for your project:
<a href="https://pinia.vuejs.org/" target="_blank" rel="noopener">Pinia</a>,
<a href="https://router.vuejs.org/" target="_blank" rel="noopener"
>Vue Router</a
>,
<a href="https://test-utils.vuejs.org/" target="_blank" rel="noopener"
>Vue Test Utils</a
>, and
<a href="https://github.com/vuejs/devtools" target="_blank" rel="noopener"
>Vue Dev Tools</a
>. If you need more resources, we suggest paying
<a
href="https://github.com/vuejs/awesome-vue"
target="_blank"
rel="noopener"
>Awesome Vue</a
>
a visit.
</WelcomeItem>
<WelcomeItem>
<template #icon>
<CommunityIcon />
</template>
<template #heading>Community</template>
Got stuck? Ask your question on
<a href="https://chat.vuejs.org" target="_blank" rel="noopener">Vue Land</a
>, our official Discord server, or
<a
href="https://stackoverflow.com/questions/tagged/vue.js"
target="_blank"
rel="noopener"
>StackOverflow</a
>. You should also subscribe to
<a href="https://news.vuejs.org" target="_blank" rel="noopener"
>our mailing list</a
>
and follow the official
<a href="https://twitter.com/vuejs" target="_blank" rel="noopener"
>@vuejs</a
>
twitter account for latest news in the Vue world.
</WelcomeItem>
<WelcomeItem>
<template #icon>
<SupportIcon />
</template>
<template #heading>Support Vue</template>
As an independent project, Vue relies on community backing for its
sustainability. You can help us by
<a href="https://vuejs.org/sponsor/" target="_blank" rel="noopener"
>becoming a sponsor</a
>.
</WelcomeItem>
</template>

View File

@@ -0,0 +1,86 @@
<template>
<div class="item">
<i>
<slot name="icon"></slot>
</i>
<div class="details">
<h3>
<slot name="heading"></slot>
</h3>
<slot></slot>
</div>
</div>
</template>
<style scoped>
.item {
margin-top: 2rem;
display: flex;
}
.details {
flex: 1;
margin-left: 1rem;
}
i {
display: flex;
place-items: center;
place-content: center;
width: 32px;
height: 32px;
color: var(--color-text);
}
h3 {
font-size: 1.2rem;
font-weight: 500;
margin-bottom: 0.4rem;
color: var(--color-heading);
}
@media (min-width: 1024px) {
.item {
margin-top: 0;
padding: 0.4rem 0 1rem calc(var(--section-gap) / 2);
}
i {
top: calc(50% - 25px);
left: -26px;
position: absolute;
border: 1px solid var(--color-border);
background: var(--color-background);
border-radius: 8px;
width: 50px;
height: 50px;
}
.item:before {
content: " ";
border-left: 1px solid var(--color-border);
position: absolute;
left: 0;
bottom: calc(50% + 25px);
height: calc(50% - 25px);
}
.item:after {
content: " ";
border-left: 1px solid var(--color-border);
position: absolute;
left: 0;
top: calc(50% + 25px);
height: calc(50% - 25px);
}
.item:first-of-type:before {
display: none;
}
.item:last-of-type:after {
display: none;
}
}
</style>

View File

@@ -0,0 +1,12 @@
<template>
<svg
xmlns="http://www.w3.org/2000/svg"
width="20"
height="20"
fill="currentColor"
>
<path
d="M15 4a1 1 0 1 0 0 2V4zm0 11v-1a1 1 0 0 0-1 1h1zm0 4l-.707.707A1 1 0 0 0 16 19h-1zm-4-4l.707-.707A1 1 0 0 0 11 14v1zm-4.707-1.293a1 1 0 0 0-1.414 1.414l1.414-1.414zm-.707.707l-.707-.707.707.707zM9 11v-1a1 1 0 0 0-.707.293L9 11zm-4 0h1a1 1 0 0 0-1-1v1zm0 4H4a1 1 0 0 0 1.707.707L5 15zm10-9h2V4h-2v2zm2 0a1 1 0 0 1 1 1h2a3 3 0 0 0-3-3v2zm1 1v6h2V7h-2zm0 6a1 1 0 0 1-1 1v2a3 3 0 0 0 3-3h-2zm-1 1h-2v2h2v-2zm-3 1v4h2v-4h-2zm1.707 3.293l-4-4-1.414 1.414 4 4 1.414-1.414zM11 14H7v2h4v-2zm-4 0c-.276 0-.525-.111-.707-.293l-1.414 1.414C5.42 15.663 6.172 16 7 16v-2zm-.707 1.121l3.414-3.414-1.414-1.414-3.414 3.414 1.414 1.414zM9 12h4v-2H9v2zm4 0a3 3 0 0 0 3-3h-2a1 1 0 0 1-1 1v2zm3-3V3h-2v6h2zm0-6a3 3 0 0 0-3-3v2a1 1 0 0 1 1 1h2zm-3-3H3v2h10V0zM3 0a3 3 0 0 0-3 3h2a1 1 0 0 1 1-1V0zM0 3v6h2V3H0zm0 6a3 3 0 0 0 3 3v-2a1 1 0 0 1-1-1H0zm3 3h2v-2H3v2zm1-1v4h2v-4H4zm1.707 4.707l.586-.586-1.414-1.414-.586.586 1.414 1.414z"
/>
</svg>
</template>

View File

@@ -0,0 +1,12 @@
<template>
<svg
xmlns="http://www.w3.org/2000/svg"
width="20"
height="17"
fill="currentColor"
>
<path
d="M11 2.253a1 1 0 1 0-2 0h2zm-2 13a1 1 0 1 0 2 0H9zm.447-12.167a1 1 0 1 0 1.107-1.666L9.447 3.086zM1 2.253L.447 1.42A1 1 0 0 0 0 2.253h1zm0 13H0a1 1 0 0 0 1.553.833L1 15.253zm8.447.833a1 1 0 1 0 1.107-1.666l-1.107 1.666zm0-14.666a1 1 0 1 0 1.107 1.666L9.447 1.42zM19 2.253h1a1 1 0 0 0-.447-.833L19 2.253zm0 13l-.553.833A1 1 0 0 0 20 15.253h-1zm-9.553-.833a1 1 0 1 0 1.107 1.666L9.447 14.42zM9 2.253v13h2v-13H9zm1.553-.833C9.203.523 7.42 0 5.5 0v2c1.572 0 2.961.431 3.947 1.086l1.107-1.666zM5.5 0C3.58 0 1.797.523.447 1.42l1.107 1.666C2.539 2.431 3.928 2 5.5 2V0zM0 2.253v13h2v-13H0zm1.553 13.833C2.539 15.431 3.928 15 5.5 15v-2c-1.92 0-3.703.523-5.053 1.42l1.107 1.666zM5.5 15c1.572 0 2.961.431 3.947 1.086l1.107-1.666C9.203 13.523 7.42 13 5.5 13v2zm5.053-11.914C11.539 2.431 12.928 2 14.5 2V0c-1.92 0-3.703.523-5.053 1.42l1.107 1.666zM14.5 2c1.573 0 2.961.431 3.947 1.086l1.107-1.666C18.203.523 16.421 0 14.5 0v2zm3.5.253v13h2v-13h-2zm1.553 12.167C18.203 13.523 16.421 13 14.5 13v2c1.573 0 2.961.431 3.947 1.086l1.107-1.666zM14.5 13c-1.92 0-3.703.523-5.053 1.42l1.107 1.666C11.539 15.431 12.928 15 14.5 15v-2z"
/>
</svg>
</template>

View File

@@ -0,0 +1,12 @@
<template>
<svg
xmlns="http://www.w3.org/2000/svg"
width="18"
height="20"
fill="currentColor"
>
<path
d="M11.447 8.894a1 1 0 1 0-.894-1.789l.894 1.789zm-2.894-.789a1 1 0 1 0 .894 1.789l-.894-1.789zm0 1.789a1 1 0 1 0 .894-1.789l-.894 1.789zM7.447 7.106a1 1 0 1 0-.894 1.789l.894-1.789zM10 9a1 1 0 1 0-2 0h2zm-2 2.5a1 1 0 1 0 2 0H8zm9.447-5.606a1 1 0 1 0-.894-1.789l.894 1.789zm-2.894-.789a1 1 0 1 0 .894 1.789l-.894-1.789zm2 .789a1 1 0 1 0 .894-1.789l-.894 1.789zm-1.106-2.789a1 1 0 1 0-.894 1.789l.894-1.789zM18 5a1 1 0 1 0-2 0h2zm-2 2.5a1 1 0 1 0 2 0h-2zm-5.447-4.606a1 1 0 1 0 .894-1.789l-.894 1.789zM9 1l.447-.894a1 1 0 0 0-.894 0L9 1zm-2.447.106a1 1 0 1 0 .894 1.789l-.894-1.789zm-6 3a1 1 0 1 0 .894 1.789L.553 4.106zm2.894.789a1 1 0 1 0-.894-1.789l.894 1.789zm-2-.789a1 1 0 1 0-.894 1.789l.894-1.789zm1.106 2.789a1 1 0 1 0 .894-1.789l-.894 1.789zM2 5a1 1 0 1 0-2 0h2zM0 7.5a1 1 0 1 0 2 0H0zm8.553 12.394a1 1 0 1 0 .894-1.789l-.894 1.789zm-1.106-2.789a1 1 0 1 0-.894 1.789l.894-1.789zm1.106 1a1 1 0 1 0 .894 1.789l-.894-1.789zm2.894.789a1 1 0 1 0-.894-1.789l.894 1.789zM8 19a1 1 0 1 0 2 0H8zm2-2.5a1 1 0 1 0-2 0h2zm-7.447.394a1 1 0 1 0 .894-1.789l-.894 1.789zM1 15H0a1 1 0 0 0 .553.894L1 15zm1-2.5a1 1 0 1 0-2 0h2zm12.553 2.606a1 1 0 1 0 .894 1.789l-.894-1.789zM17 15l.447.894A1 1 0 0 0 18 15h-1zm1-2.5a1 1 0 1 0-2 0h2zm-7.447-5.394l-2 1 .894 1.789 2-1-.894-1.789zm-1.106 1l-2-1-.894 1.789 2 1 .894-1.789zM8 9v2.5h2V9H8zm8.553-4.894l-2 1 .894 1.789 2-1-.894-1.789zm.894 0l-2-1-.894 1.789 2 1 .894-1.789zM16 5v2.5h2V5h-2zm-4.553-3.894l-2-1-.894 1.789 2 1 .894-1.789zm-2.894-1l-2 1 .894 1.789 2-1L8.553.106zM1.447 5.894l2-1-.894-1.789-2 1 .894 1.789zm-.894 0l2 1 .894-1.789-2-1-.894 1.789zM0 5v2.5h2V5H0zm9.447 13.106l-2-1-.894 1.789 2 1 .894-1.789zm0 1.789l2-1-.894-1.789-2 1 .894 1.789zM10 19v-2.5H8V19h2zm-6.553-3.894l-2-1-.894 1.789 2 1 .894-1.789zM2 15v-2.5H0V15h2zm13.447 1.894l2-1-.894-1.789-2 1 .894 1.789zM18 15v-2.5h-2V15h2z"
/>
</svg>
</template>

View File

@@ -0,0 +1,12 @@
<template>
<svg
xmlns="http://www.w3.org/2000/svg"
width="20"
height="20"
fill="currentColor"
>
<path
d="M10 3.22l-.61-.6a5.5 5.5 0 0 0-7.666.105 5.5 5.5 0 0 0-.114 7.665L10 18.78l8.39-8.4a5.5 5.5 0 0 0-.114-7.665 5.5 5.5 0 0 0-7.666-.105l-.61.61z"
/>
</svg>
</template>

View File

@@ -0,0 +1,19 @@
<!-- This icon is from <https://github.com/Templarian/MaterialDesign>, distributed under Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0) license-->
<template>
<svg
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
aria-hidden="true"
role="img"
class="iconify iconify--mdi"
width="24"
height="24"
preserveAspectRatio="xMidYMid meet"
viewBox="0 0 24 24"
>
<path
d="M20 18v-4h-3v1h-2v-1H9v1H7v-1H4v4h16M6.33 8l-1.74 4H7v-1h2v1h6v-1h2v1h2.41l-1.74-4H6.33M9 5v1h6V5H9m12.84 7.61c.1.22.16.48.16.8V18c0 .53-.21 1-.6 1.41c-.4.4-.85.59-1.4.59H4c-.55 0-1-.19-1.4-.59C2.21 19 2 18.53 2 18v-4.59c0-.32.06-.58.16-.8L4.5 7.22C4.84 6.41 5.45 6 6.33 6H7V5c0-.55.18-1 .57-1.41C7.96 3.2 8.44 3 9 3h6c.56 0 1.04.2 1.43.59c.39.41.57.86.57 1.41v1h.67c.88 0 1.49.41 1.83 1.22l2.34 5.39z"
fill="currentColor"
></path>
</svg>
</template>

View File

@@ -0,0 +1,15 @@
<template>
<div class="about">
<h1>This is an about page</h1>
</div>
</template>
<style>
@media (min-width: 1024px) {
.about {
min-height: 100vh;
display: flex;
align-items: center;
}
}
</style>

View File

@@ -0,0 +1,7 @@
<template>
<div></div>
</template>
<script setup lang="ts"></script>
<style scoped lang="less"></style>

View File

@@ -0,0 +1,9 @@
<script setup lang="ts">
import TheWelcome from "../components/TheWelcome.vue";
</script>
<template>
<main>
<TheWelcome />
</main>
</template>

View File

@@ -0,0 +1,5 @@
// Generated by 'unplugin-auto-import'
export {}
declare global {
}

View File

@@ -0,0 +1,27 @@
// generated by unplugin-vue-components
// We suggest you to commit this file into source control
// Read more: https://github.com/vuejs/core/pull/3399
import '@vue/runtime-core'
export {}
declare module '@vue/runtime-core' {
export interface GlobalComponents {
ElAside: typeof import('element-plus/es')['ElAside']
ElButton: typeof import('element-plus/es')['ElButton']
ElCard: typeof import('element-plus/es')['ElCard']
ElCol: typeof import('element-plus/es')['ElCol']
ElContainer: typeof import('element-plus/es')['ElContainer']
ElDialog: typeof import('element-plus/es')['ElDialog']
ElFooter: typeof import('element-plus/es')['ElFooter']
ElHeader: typeof import('element-plus/es')['ElHeader']
ElInput: typeof import('element-plus/es')['ElInput']
ElMain: typeof import('element-plus/es')['ElMain']
ElMenu: typeof import('element-plus/es')['ElMenu']
ElMenuItem: typeof import('element-plus/es')['ElMenuItem']
ElRow: typeof import('element-plus/es')['ElRow']
ElSubMenu: typeof import('element-plus/es')['ElSubMenu']
RouterLink: typeof import('vue-router')['RouterLink']
RouterView: typeof import('vue-router')['RouterView']
}
}

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import FaceDetection from "@/pages/cv/detection/FaceDetection/FaceDetection.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<FaceDetection></FaceDetection>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,142 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="12">
<el-row class="small-title">
<h2>上传人脸图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="12">
<el-row class="small-title">
<h2>人脸区域检测</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始检测</el-button>
</el-row>
<el-row>
<canvas id="canvas" class="show-area"></canvas>
</el-row>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import { FaceDetector } from "@paddle-js-models/facedetect";
import { onMounted, ref } from "vue";
interface TransformData {
left: number;
width: number;
top: number;
height: number;
confidence: number;
}
const faceDet = new FaceDetector();
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const isLoadingModel = ref(true);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await faceDet.init();
isLoadingModel.value = false;
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await faceDet.detect(img);
console.log(res);
drawBox(res);
};
const drawBox = (data: TransformData[]) => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const imgHeight = img.height;
const imgWidth = img.width;
canvas.value.width = imgWidth;
canvas.value.height = imgHeight;
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
data.forEach((item: TransformData) => {
// 开始一个新的绘制路径
ctx.beginPath();
// 设置线条颜色为蓝色
ctx.strokeStyle = "red";
// 获取检测框选坐标
const x = item.left * imgWidth;
const y = item.top * imgHeight;
const w = item.width * imgWidth;
const h = item.height * imgHeight;
ctx.beginPath();
// 绘制检测框选矩形
ctx.rect(x, y, w, h);
// 绘制label
ctx.fillText(item.confidence.toFixed(6), x, y);
ctx.stroke();
});
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import ScrewDetection from "@/pages/cv/detection/ScrewDetection/ScrewDetection.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<ScrewDetection></ScrewDetection>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,138 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="12">
<el-row class="small-title">
<h2>上传图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="12">
<el-row class="small-title">
<h2>螺丝螺母区域检测</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始检测</el-button>
</el-row>
<el-row>
<canvas id="canvas" class="show-area"></canvas>
</el-row>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as det from "@paddle-js-models/detect";
import { onMounted, ref } from "vue";
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const isLoadingModel = ref(true);
const label = new Map([
[0, "default"],
[1, "screw"],
[2, "nut"],
]);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await det.init();
isLoadingModel.value = false;
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await det.detect(img);
console.log(res);
drawBox(res);
};
const drawBox = (points: number[][]) => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const imgHeight = img.height;
const imgWidth = img.width;
canvas.value.width = imgWidth;
canvas.value.height = imgHeight;
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
points.forEach((point: number[]) => {
// 开始一个新的绘制路径
ctx.beginPath();
// 设置线条颜色为蓝色
ctx.strokeStyle = "red";
// 获取检测框选坐标
const left = Math.floor(point[2] * imgWidth);
const top = Math.floor(point[3] * imgHeight);
const right = Math.floor(point[4] * imgWidth);
const bottom = Math.floor(point[5] * imgHeight);
ctx.beginPath();
// 绘制检测框选矩形
ctx.rect(left, top, right - left, bottom - top);
// 绘制label
ctx.fillText(label.get(point[0])!, left + 10, top + 10);
ctx.stroke();
});
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import TextDetection from "@/pages/cv/ocr/TextDetection/TextDetection.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<TextDetection></TextDetection>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,126 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="12">
<el-row class="small-title">
<h2>上传文本图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="12">
<el-row class="small-title">
<h2>文字区域检测</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始检测</el-button>
</el-row>
<el-row>
<canvas id="canvas" class="show-area"></canvas>
</el-row>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as ocrDet from "@paddle-js-models/ocrdet";
import { onMounted, ref } from "vue";
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const isLoadingModel = ref(true);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await ocrDet.load("https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json");
isLoadingModel.value = false;
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await ocrDet.detect(img);
console.log(res);
drawBox(res);
};
const drawBox = (points: number[][][]) => {
const img = document.getElementById("raw-img") as HTMLImageElement;
canvas.value.width = img.naturalWidth;
canvas.value.height = img.naturalHeight;
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
points.forEach((point) => {
// 开始一个新的绘制路径
ctx.beginPath();
// 设置线条颜色为蓝色
ctx.strokeStyle = "blue";
// 设置路径起点坐标
ctx.moveTo(point[0][0], point[0][1]);
ctx.lineTo(point[1][0], point[1][1]);
ctx.lineTo(point[2][0], point[2][1]);
ctx.lineTo(point[3][0], point[3][1]);
ctx.closePath();
ctx.stroke();
});
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import TextRecognition from "@/pages/cv/ocr/TextRecognition/TextRecognition.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<TextRecognition></TextRecognition>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,121 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="8">
<el-row class="small-title">
<h2>上传文本图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="8">
<el-row class="small-title">
<h2>文字区域检测</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始识别</el-button>
</el-row>
<el-row>
<canvas id="canvas" class="show-area"></canvas>
</el-row>
</el-col>
<el-col :span="8">
<el-row class="small-title">
<h2>识别结果展示</h2>
</el-row>
<span v-html="result"></span>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as ocr from "@paddle-js-models/ocr";
import { onMounted, onUnmounted, ref } from "vue";
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const result = ref("");
const isLoadingModel = ref(true);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await ocr.init("https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_det_infer_js_960/model.json", "https://js-models.bj.bcebos.com/PaddleOCR/PP-OCRv3/ch_PP-OCRv3_rec_infer_js/model.json");
isLoadingModel.value = false;
});
onUnmounted(() => {
console.log("delete rec");
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await ocr.recognize(img, { canvas: canvas.value });
console.log(res);
if (res.text?.length) {
// 页面展示识别内容
result.value = res.text.reduce((total, cur) => total + `<p>${cur}</p>`);
}
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import GestureRecognition from "@/pages/cv/recognition/GestureRecognition/GestureRecognition.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<GestureRecognition></GestureRecognition>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,168 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="8">
<el-row class="small-title">
<h2>上传手势图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="8">
<el-row class="small-title">
<h2>手势检测</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始识别</el-button>
</el-row>
<el-row>
<canvas id="canvas" class="show-area"></canvas>
</el-row>
</el-col>
<el-col :span="8">
<el-row class="small-title">
<h2>手势识别结果</h2>
</el-row>
<span v-html="result"></span>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as gestureRec from "@paddle-js-models/gesture";
import { onMounted, onUnmounted, ref } from "vue";
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const result = ref("");
const isLoadingModel = ref(true);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await gestureRec.load();
isLoadingModel.value = false;
});
onUnmounted(() => {
console.log("delete rec");
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await gestureRec.classify(img);
console.log(res);
if (res.box && res.box.length) {
result.value = res.type;
drawBox(res.box as number[][]);
} else {
result.value = "未识别到手";
}
};
const drawBox = (box: number[][]) => {
const img = document.getElementById("raw-img") as HTMLImageElement;
canvas.value.width = img.naturalWidth;
canvas.value.height = img.naturalHeight;
const ctx = canvas.value.getContext("2d") as CanvasRenderingContext2D;
ctx.drawImage(img, 0, 0, canvas.value.width, canvas.value.height);
/**
* 计算缩放比率,得到实际绘制框的坐标
*/
let offsetX = 0;
let offsetY = 0;
if (img.width < img.height) {
offsetX = img.height - img.width;
}
if (img.width > img.height) {
offsetY = img.width - img.height;
}
const widthRatio = (img.width + offsetX) / 256;
const heightRatio = (img.height + offsetY) / 256;
const points: number[][] = [];
box.forEach((item) => {
const tmpPonit = [];
tmpPonit[0] = item[0] * widthRatio - offsetX / 2;
tmpPonit[1] = item[1] * heightRatio - offsetY / 2;
points.push(tmpPonit);
});
// 开始一个新的绘制路径
ctx.beginPath();
// 设置线条颜色为蓝色
ctx.strokeStyle = "blue";
// 设置路径起点坐标
ctx.moveTo(points[0][0], points[0][1]);
ctx.lineTo(points[1][0], points[1][1]);
ctx.lineTo(points[2][0], points[2][1]);
ctx.lineTo(points[3][0], points[3][1]);
ctx.closePath();
ctx.stroke();
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import ItemIdentification from "@/pages/cv/recognition/ItemIdentification/ItemIdentification.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<ItemIdentification></ItemIdentification>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,115 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="16">
<el-row class="small-title">
<h2>上传图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="8">
<el-row class="small-title">
<h2>预测结果</h2>
</el-row>
<el-row>
<el-button type="primary" @click="predict">开始检测</el-button>
</el-row>
<el-row>
{{ itemName }}
</el-row>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as mobilenet from "@paddle-js-models/mobilenet";
import { onMounted, ref } from "vue";
import map from "./map.json";
const fileName = ref(null);
const canvas = ref(null as unknown as HTMLCanvasElement);
const isLoadingModel = ref(true);
const itemName = ref(null);
onMounted(async () => {
canvas.value = document.getElementById("canvas") as HTMLCanvasElement;
await mobilenet.load(
{
path: "https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/mobileNetV2_fuse_activation/model.json",
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225],
},
map
);
isLoadingModel.value = false;
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const predict = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await mobilenet.classify(img);
console.log(res);
itemName.value = res;
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

View File

@@ -0,0 +1,9 @@
import { createApp } from "vue";
import App from "./App.vue";
import "../../../../assets/main.css";
const app = createApp(App);
app.mount("#app");

View File

@@ -0,0 +1,7 @@
<template>
<div></div>
</template>
<script setup lang="ts"></script>
<style scoped lang="less"></style>

View File

@@ -0,0 +1,20 @@
<script setup lang="ts">
import HumanSeg from "@/pages/cv/segmentation/HumanSeg/HumanSeg.vue";
</script>
<template>
<el-container>
<el-header>Header</el-header>
<el-main>
<HumanSeg></HumanSeg>
</el-main>
<el-footer>Footer</el-footer>
</el-container>
</template>
<style scoped lang="less">
.el-container {
height: 100%;
width: 100%;
}
</style>

View File

@@ -0,0 +1,229 @@
<template>
<el-dialog
v-model="isLoadingModel"
title="提示"
width="30%"
center
:lock-scroll="true"
:show-close="false"
:close-on-click-modal="false"
:close-on-press-escape="false"
>
<span>正在加载模型请稍等</span>
</el-dialog>
<el-row :gutter="20">
<el-col :span="8">
<el-row class="small-title">
<h2>上传图片</h2>
</el-row>
<el-row>
<el-input type="file" v-model="fileName" @change="uploadImg"></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="show-img" class="show-area" />
<!-- 用于存放真实图片进行文字识别 -->
<img id="raw-img" style="display: none" />
</el-row>
</el-col>
<el-col :span="4">
<el-row class="small-title">
<h2>上传替换背景图片</h2>
</el-row>
<el-row>
<el-input
type="file"
v-model="newBackgroundImgFileName"
@change="uploadNewBackgroundImg"
></el-input>
</el-row>
<el-row>
<!-- 用于展示图片 -->
<img id="new-background-img" class="show-area" />
</el-row>
<el-row>
<canvas id="background" class="show-area"></canvas>
</el-row>
</el-col>
<el-col :span="4">
<el-row class="small-title">
<h2>背景替换</h2>
</el-row>
<el-row>
<el-button type="primary" @click="backgroundReplace">
替换背景
</el-button>
</el-row>
<el-row>
<canvas id="replace-background" class="show-area"></canvas>
</el-row>
</el-col>
<el-col :span="4">
<el-row class="small-title">
<h2>背景虚化</h2>
</el-row>
<el-row>
<el-button type="primary" @click="blurBackground">背景虚化</el-button>
</el-row>
<el-row>
<canvas id="blur" class="show-area"></canvas>
</el-row>
</el-col>
<el-col :span="4">
<el-row class="small-title">
<h2>人形遮罩</h2>
</el-row>
<el-row>
<el-button type="primary" @click="drawHumanoidMask">
绘制人形遮罩
</el-button>
</el-row>
<el-row>
<canvas id="mask" class="show-area"></canvas>
</el-row>
</el-col>
</el-row>
</template>
<script setup lang="ts">
import * as humanSeg from "@paddle-js-models/humanseg";
import { onMounted, ref } from "vue";
const fileName = ref(null);
const newBackgroundImgFileName = ref(null);
const backgroundCanvas = ref(null as unknown as HTMLCanvasElement);
const replaceBackgroundCanvas = ref(null as unknown as HTMLCanvasElement);
const blurCanvas = ref(null as unknown as HTMLCanvasElement);
const maskCanvas = ref(null as unknown as HTMLCanvasElement);
const isLoadingModel = ref(true);
const modelPredictDone = ref(false);
/**
* 存储模型分割后的像素 alpha 值
*/
let garyData: number[];
onMounted(async () => {
backgroundCanvas.value = document.getElementById(
"background"
) as HTMLCanvasElement;
replaceBackgroundCanvas.value = document.getElementById(
"replace-background"
) as HTMLCanvasElement;
blurCanvas.value = document.getElementById("blur") as HTMLCanvasElement;
maskCanvas.value = document.getElementById("mask") as HTMLCanvasElement;
await humanSeg.load();
isLoadingModel.value = false;
});
const uploadImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById("show-img") as HTMLImageElement;
// 用于识别
const rawImg = document.getElementById("raw-img") as HTMLImageElement;
const inputElement = document
.getElementsByClassName("el-input")[0]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
rawImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const uploadNewBackgroundImg = () => {
/**
* 这里由于操作是绑定在 el-input 上;因此需要在内部重新获取 input 再拿到 file
*/
const reader = new FileReader();
// 用于展示
const showImg = document.getElementById(
"new-background-img"
) as HTMLImageElement;
// 获取背景图片的 input
const inputElement = document
.getElementsByClassName("el-input")[1]
.getElementsByTagName("input")[0];
try {
const file = inputElement.files![0];
reader.onload = () => {
showImg.src = URL.createObjectURL(file);
};
reader.readAsDataURL(file);
} catch (err) {
console.error(err);
}
};
const backgroundReplace = async () => {
if (!modelPredictDone.value) {
await seg();
modelPredictDone.value = true;
}
const showImg = document.getElementById(
"new-background-img"
) as HTMLImageElement;
backgroundCanvas.value.width = showImg.naturalWidth;
backgroundCanvas.value.height = showImg.naturalHeight;
backgroundCanvas.value
.getContext("2d")!
.drawImage(showImg, 0, 0, showImg.naturalWidth, showImg.naturalHeight);
humanSeg.drawHumanSeg(
garyData,
replaceBackgroundCanvas.value,
backgroundCanvas.value
);
};
const blurBackground = async () => {
if (!modelPredictDone.value) {
await seg();
modelPredictDone.value = true;
}
humanSeg.blurBackground(garyData, blurCanvas.value);
};
const drawHumanoidMask = async () => {
if (!modelPredictDone.value) {
await seg();
modelPredictDone.value = true;
}
humanSeg.drawMask(garyData, maskCanvas.value, backgroundCanvas.value);
};
const seg = async () => {
const img = document.getElementById("raw-img") as HTMLImageElement;
const res = await humanSeg.getGrayValue(img);
console.log(res);
garyData = res.data;
};
</script>
<style scoped lang="less">
.small-title {
justify-content: space-between;
align-items: center;
}
.show-area {
width: 100%;
}
.el-row {
margin-bottom: 20px;
}
</style>

View File

@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script>var Module;</script>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="main.ts"></script>
</body>
</html>

Some files were not shown because too many files have changed in this diff Show More