yolotriton/README.md

yolotriton

Go (Golang) gRPC client for YOLOv8 inference using the Triton Inference Server.

Installation

Use go get to install this package:

go get github.com/dev6699/yolotriton

Get YOLOv8 TensorRT model

pip install ultralytics
yolo export model=yolov8m.pt format=onnx
trtexec --onnx=yolov8m.onnx --saveEngine=model_repository/yolov8_tensorrt/1/model.plan

References:

1. https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html
2. https://docs.ultralytics.com/modes/export/
3. https://github.com/NVIDIA/TensorRT/tree/master/samples/trtexec

Start trinton server

docker compose up tritonserver

References:

1. https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_repository.html

Sample usage

Check cmd/main.go for more details.

Available args:

  -i string
        Inference Image. Default: images/1.jpg (default "images/1.jpg")
  -m string
        Name of model being served. (Required) (default "yolov8_tensorrt")
  -u string
        Inference Server URL. Default: tritonserver:8001 (default "tritonserver:8001")
  -x string
        Version of model. Default: Latest Version.

go run cmd/main.go

Results

Input	Ouput