mirror of
https://github.com/Kalit31/AR-Video-Streaming-over-WebRTC.git
synced 2025-09-26 19:31:20 +08:00
125 lines
4.6 KiB
Plaintext
125 lines
4.6 KiB
Plaintext
https://github.com/njpietrow/Filter.io
|
|
https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation
|
|
https://github.com/pion/awesome-pion
|
|
|
|
|
|
|
|
real-time quality - network
|
|
|
|
ffmpeg - encoding/decoding
|
|
|
|
|
|
base frame + delta diff
|
|
h264 encoding
|
|
|
|
|
|
|
|
|
|
client1 client2
|
|
os.exec(ffmpeg) os.exec(ffmpeg)
|
|
|
|
|
|
https://github.com/FFmpeg/FFmpeg
|
|
https://github.com/AlexEidt/Vidio
|
|
https://github.com/njpietrow/Filter.io
|
|
https://github.com/AgustinSRG/webrtc-video-filter
|
|
https://github.com/hybridgroup/cvscope/
|
|
https://github.com/ashellunts/ffmpeg-to-webrtc
|
|
https://github.com/pion/webrtc
|
|
https://github.com/pion/example-webrtc-applications
|
|
https://github.com/pion/example-webrtc-applications/tree/master/ffmpeg-send
|
|
https://github.com/asticode/go-astiav
|
|
https://jsfiddle.net/z17q28cd/
|
|
|
|
ffmpeg -f v4l2 -i /dev/video0 -f mpegts udp://224.0.0.251:5353
|
|
|
|
netstat -anu|sort -nk4
|
|
|
|
metric - time, energy consumption
|
|
|
|
|
|
Oct 16th -
|
|
look up energy consumption models for mobile phones/edge devices
|
|
|
|
drain based on the processor/cpu/RAM - battery drain emulators/simulators - estimate of energy consumption for the program.
|
|
|
|
monitoring tools for battery drains
|
|
|
|
TPUs = Tensor Processing Units
|
|
|
|
Tasks:
|
|
1. Look for other people working on same problem, google scholar
|
|
2. explore AR Workload
|
|
3. Volumetric videos - format, can handle with regular RTP?
|
|
4. battery savings?
|
|
|
|
ffmpeg -i input.webm -f mpegts udp://224.0.0.251:5353
|
|
|
|
/home/kalit/Desktop/GeorgiaTech/Fall_2024/CS_8903/WebRTC_research/ar-filters/filter_imgs/eye.jpg
|
|
/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/eye.jpg
|
|
/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/smile.png
|
|
|
|
|
|
AR workloads:
|
|
https://cuhksz-inml.github.io/full_scene_volumetric_video_dataset/factsfigures.html
|
|
|
|
|
|
Forward streams to localhost:5005 to jetson machine's localhost:5005 -
|
|
ssh -L 5005:localhost:5005 -J fastvideo -i ~/.ssh/picluster epl@10.100.1.165
|
|
|
|
|
|
Jetson: 4 CPU cores ARMv8 Processor rev 1
|
|
Device 0: "NVIDIA Tegra X1"
|
|
CUDA Driver Version / Runtime Version 10.0 / 10.0
|
|
CUDA Capability Major/Minor version number: 5.3
|
|
Total amount of global memory: 3962 MBytes (4154626048 bytes)
|
|
( 2) Multiprocessors, (128) CUDA Cores/MP: 256 CUDA Cores
|
|
GPU Max Clock rate: 998 MHz (1.00 GHz)
|
|
Memory Clock rate: 1600 Mhz
|
|
Memory Bus Width: 64-bit
|
|
L2 Cache Size: 262144 bytes
|
|
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
|
|
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
|
|
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
|
|
Total amount of constant memory: 65536 bytes
|
|
Total amount of shared memory per block: 49152 bytes
|
|
Total number of registers available per block: 32768
|
|
Warp size: 32
|
|
Maximum number of threads per multiprocessor: 2048
|
|
Maximum number of threads per block: 1024
|
|
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
|
|
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
|
|
Maximum memory pitch: 2147483647 bytes
|
|
Texture alignment: 512 bytes
|
|
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
|
|
Run time limit on kernels: Yes
|
|
Integrated GPU sharing Host Memory: Yes
|
|
Support host page-locked memory mapping: Yes
|
|
Alignment requirement for Surfaces: Yes
|
|
Device has ECC support: Disabled
|
|
Device supports Unified Addressing (UVA): Yes
|
|
Device supports Compute Preemption: No
|
|
Supports Cooperative Kernel Launch: No
|
|
Supports MultiDevice Co-op Kernel Launch: No
|
|
Device PCI Domain ID / Bus ID / location ID: 0 / 0 / 0
|
|
Compute Mode:
|
|
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
|
|
|
|
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 1
|
|
Result = PASS
|
|
|
|
|
|
Mediapipe issues (GPU + Jetson :( )
|
|
1. https://github.com/google-ai-edge/mediapipe/issues/4017
|
|
2. https://github.com/google-ai-edge/mediapipe/issues/5344#issuecomment-2076742967
|
|
3. https://github.com/google-ai-edge/mediapipe/issues/3353
|
|
4. https://github.com/google-ai-edge/mediapipe/issues/1651#issuecomment-790176010
|
|
5. https://github.com/google-ai-edge/mediapipe/issues/1344
|
|
6. https://github.com/google-ai-edge/mediapipe/issues/5736
|
|
|
|
|
|
jetson benchmark suites - work on images? object detection filters?
|
|
ar-benchmarks suite
|
|
|
|
|