https://github.com/njpietrow/Filter.io https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation https://github.com/pion/awesome-pion real-time quality - network ffmpeg - encoding/decoding base frame + delta diff h264 encoding client1 client2 os.exec(ffmpeg) os.exec(ffmpeg) https://github.com/FFmpeg/FFmpeg https://github.com/AlexEidt/Vidio https://github.com/njpietrow/Filter.io https://github.com/AgustinSRG/webrtc-video-filter https://github.com/hybridgroup/cvscope/ https://github.com/ashellunts/ffmpeg-to-webrtc https://github.com/pion/webrtc https://github.com/pion/example-webrtc-applications https://github.com/pion/example-webrtc-applications/tree/master/ffmpeg-send https://github.com/asticode/go-astiav https://jsfiddle.net/z17q28cd/ ffmpeg -f v4l2 -i /dev/video0 -f mpegts udp://224.0.0.251:5353 netstat -anu|sort -nk4 metric - time, energy consumption Oct 16th - look up energy consumption models for mobile phones/edge devices drain based on the processor/cpu/RAM - battery drain emulators/simulators - estimate of energy consumption for the program. monitoring tools for battery drains TPUs = Tensor Processing Units Tasks: 1. Look for other people working on same problem, google scholar 2. explore AR Workload 3. Volumetric videos - format, can handle with regular RTP? 4. battery savings? ffmpeg -i input.webm -f mpegts udp://224.0.0.251:5353 /home/kalit/Desktop/GeorgiaTech/Fall_2024/CS_8903/WebRTC_research/ar-filters/filter_imgs/eye.jpg /home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/eye.jpg /home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/smile.png AR workloads: https://cuhksz-inml.github.io/full_scene_volumetric_video_dataset/factsfigures.html Forward streams to localhost:5005 to jetson machine's localhost:5005 - ssh -L 5005:localhost:5005 -J fastvideo -i ~/.ssh/picluster epl@10.100.1.165 Jetson: 4 CPU cores ARMv8 Processor rev 1 Device 0: "NVIDIA Tegra X1" CUDA Driver Version / Runtime Version 10.0 / 10.0 CUDA Capability Major/Minor version number: 5.3 Total amount of global memory: 3962 MBytes (4154626048 bytes) ( 2) Multiprocessors, (128) CUDA Cores/MP: 256 CUDA Cores GPU Max Clock rate: 998 MHz (1.00 GHz) Memory Clock rate: 1600 Mhz Memory Bus Width: 64-bit L2 Cache Size: 262144 bytes Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096) Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers Total amount of constant memory: 65536 bytes Total amount of shared memory per block: 49152 bytes Total number of registers available per block: 32768 Warp size: 32 Maximum number of threads per multiprocessor: 2048 Maximum number of threads per block: 1024 Max dimension size of a thread block (x,y,z): (1024, 1024, 64) Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535) Maximum memory pitch: 2147483647 bytes Texture alignment: 512 bytes Concurrent copy and kernel execution: Yes with 1 copy engine(s) Run time limit on kernels: Yes Integrated GPU sharing Host Memory: Yes Support host page-locked memory mapping: Yes Alignment requirement for Surfaces: Yes Device has ECC support: Disabled Device supports Unified Addressing (UVA): Yes Device supports Compute Preemption: No Supports Cooperative Kernel Launch: No Supports MultiDevice Co-op Kernel Launch: No Device PCI Domain ID / Bus ID / location ID: 0 / 0 / 0 Compute Mode: < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) > deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 1 Result = PASS Mediapipe issues (GPU + Jetson :( ) 1. https://github.com/google-ai-edge/mediapipe/issues/4017 2. https://github.com/google-ai-edge/mediapipe/issues/5344#issuecomment-2076742967 3. https://github.com/google-ai-edge/mediapipe/issues/3353 4. https://github.com/google-ai-edge/mediapipe/issues/1651#issuecomment-790176010 5. https://github.com/google-ai-edge/mediapipe/issues/1344 6. https://github.com/google-ai-edge/mediapipe/issues/5736 jetson benchmark suites - work on images? object detection filters? ar-benchmarks suite