zishuo/AR-Video-Streaming-over-WebRTC
1
0
Fork 0
mirror of https://github.com/Kalit31/AR-Video-Streaming-over-WebRTC.git synced 2025-09-27 03:35:55 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Stuck with mediapipe issues on Jetson

Browse Source
This commit is contained in:
Kalit Inani
2024-11-19 15:09:40 -05:00
parent b257e37207
commit 952ee7b9a2
3 changed files with 57 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
ar-filters/app.py
View File

@@ -10,8 +10,10 @@ import struct
mp_face_mesh = mp.solutions.face_mesh mp_face_mesh = mp.solutions.face_mesh
face_mesh_videos = mp_face_mesh.FaceMesh(static_image_mode=False, max_num_faces=1, min_detection_confidence=0.5, min_tracking_confidence=0.3) face_mesh_videos = mp_face_mesh.FaceMesh(static_image_mode=False, max_num_faces=1, min_detection_confidence=0.5, min_tracking_confidence=0.3)
eye = cv2.imread('/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/eye.jpg') # eye = cv2.imread('/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/eye.jpg')
mouth = cv2.imread('/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/smile.png') # mouth = cv2.imread('/home/epl/Desktop/WebRTC_research/ar-filters/filter_imgs/smile.png')
eye = cv2.imread('/home/kalit/Desktop/GeorgiaTech/Fall_2024/CS_8903/WebRTC_research/ar-filters/filter_imgs/eye.jpg')
mouth = cv2.imread('/home/kalit/Desktop/GeorgiaTech/Fall_2024/CS_8903/WebRTC_research/ar-filters/filter_imgs/smile.png')
def detectFacialLandmarks(image, face_mesh): def detectFacialLandmarks(image, face_mesh):
return face_mesh.process(image[:,:,::-1]) return face_mesh.process(image[:,:,::-1])
@@ -121,7 +123,7 @@ def main():
# output_file_path = 'output.jpg' # output_file_path = 'output.jpg'
# with open(output_file_path, 'wb') as f: # with open(output_file_path, 'wb') as f:
# f.write(buffer) # f.write(buffer)
print("Sending back processed image") # print("Sending back processed image")
conn.sendall(struct.pack('!I', len(buffer)) + buffer.tobytes()) conn.sendall(struct.pack('!I', len(buffer)) + buffer.tobytes())
conn.close() conn.close()

2
client/overlay_ar_filters.go
View File

@@ -114,7 +114,7 @@ func OverlayARFilter(conn net.Conn, frame *astiav.Frame) (*astiav.Frame, error)
return frame, err return frame, err
} }
dumpImageToFile("processed.jpg", processed_image) // dumpImageToFile("processed.jpg", processed_image)
return processed_frame, nil return processed_frame, nil
} }

52
relevant_repos.txt
View File

@@ -65,4 +65,54 @@ https://cuhksz-inml.github.io/full_scene_volumetric_video_dataset/factsfigures.h
Forward streams to localhost:5005 to jetson machine's localhost:5005 - Forward streams to localhost:5005 to jetson machine's localhost:5005 -
ssh -L 5005:localhost:5005 -J fastvideo -i ~/.ssh/picluster epl@10.100.1.165 ssh -L 5005:localhost:5005 -J fastvideo -i ~/.ssh/picluster epl@10.100.1.165
Jetson: 4 CPU cores ARMv8 Processor rev 1
Device 0: "NVIDIA Tegra X1"
CUDA Driver Version / Runtime Version 10.0 / 10.0
CUDA Capability Major/Minor version number: 5.3
Total amount of global memory: 3962 MBytes (4154626048 bytes)
( 2) Multiprocessors, (128) CUDA Cores/MP: 256 CUDA Cores
GPU Max Clock rate: 998 MHz (1.00 GHz)
Memory Clock rate: 1600 Mhz
Memory Bus Width: 64-bit
L2 Cache Size: 262144 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 32768
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: Yes
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Compute Preemption: No
Supports Cooperative Kernel Launch: No
Supports MultiDevice Co-op Kernel Launch: No
Device PCI Domain ID / Bus ID / location ID: 0 / 0 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 1
Result = PASS
Mediapipe issues (GPU + Jetson :( )
1. https://github.com/google-ai-edge/mediapipe/issues/4017
2. https://github.com/google-ai-edge/mediapipe/issues/5344#issuecomment-2076742967
3. https://github.com/google-ai-edge/mediapipe/issues/3353
4. https://github.com/google-ai-edge/mediapipe/issues/1651#issuecomment-790176010
5. https://github.com/google-ai-edge/mediapipe/issues/1344
6. https://github.com/google-ai-edge/mediapipe/issues/5736