MiDaS Depth Estimation Example

Overview

This example uses the MiDaS v3.1 depth estimation for computing depth in a single image.

Usage

Make sure you have downloaded the data files first for the examples. You only need to do this once for all examples.

cd example/
git clone --depth=1 https://github.com/swdee/go-rknnlite-data.git data

Run the MiDaS example on the above living room scene on rk3588 or replace with your Platform model.

cd example/midas
go run midas.go -p rk3588

This will result in the output of:

Driver Version: 0.9.6, API Version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
Model Input Number: 1, Output Number: 1
Input tensors:
  index=0, name=input, n_dims=4, dims=[1, 256, 256, 3], n_elems=196608, size=196608, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=0, scale=0.007843
Output tensors:
  index=0, name=depth, n_dims=4, dims=[1, 1, 256, 256], n_elems=65536, size=65536, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=19.864582
Model first run speed: inference=577.442314ms, post processing=1.180646ms, rendering=3.137693ms, total time=581.760653ms
Saved depth map result to ../data/bedroom-out.jpg
Benchmark time=12.167970519s, count=20, average total time=608.398525ms
done

The saved JPG image with depth estimation map.

See the help for command line parameters.

$ go run midas.go -h

Usage of /tmp/go-build2937772053/b001/exe/midas:
  -i string
        Image file to run depth estimation on (default "../data/bedroom.jpg")
  -m string
        RKNN compiled depth model file (default "../data/models/rk3588/dpt_swin2_tiny_256-rk3588.rknn")
  -o string
        Output JPG file (depth visualization) (default "../data/bedroom-out.jpg")
  -p string
        Rockchip platform [rk3562|rk3566|rk3568|rk3576|rk3582|rk3588] (default "rk3588")

Docker

To run the MiDaS example using the prebuilt docker image, make sure the data files have been downloaded first, then run.

# from project root directory

docker run --rm \
  --device /dev/dri:/dev/dri \
  -v "$(pwd):/go/src/app" \
  -v "$(pwd)/example/data:/go/src/data" \
  -v "/usr/include/rknn_api.h:/usr/include/rknn_api.h" \
  -v "/usr/lib/librknnrt.so:/usr/lib/librknnrt.so" \
  -w /go/src/app \
  swdee/go-rknnlite:latest \
  go run ./example/midas/midas.go -p rk3588

Benchmarks

The following table shows a comparison of the benchmark results across the three distinct platforms.

Platform	Execution Time	Average Inference Time Per Image
rk3588	12.16s	608.39ms
rk3576	16.85s	842.97ms
rk3566	37.49s	1.87s

2.9 KiB Raw Blame History

MiDaS Depth Estimation Example

Overview

Usage

Docker

Benchmarks

2.9 KiB

Raw Blame History