Fix H265 support for different sources

README.md

@@ -5,11 +5,10 @@ Ultimate camera streaming application with support RTSP, WebRTC, HomeKit, FFmpeg
 ![](assets/go2rtc.png)
 
 - zero-dependency and zero-config [small app](#go2rtc-binary) for all OS (Windows, macOS, Linux, ARM)
-- zero-delay for all supported protocols (lowest possible streaming latency)
-- streaming from [RTSP](#source-rtsp), [RTMP](#source-rtmp), [MJPEG](#source-ffmpeg), [HLS](#source-ffmpeg), [USB Cameras](#source-ffmpeg-device), [files](#source-ffmpeg) and [other sources](#module-streams)
-- streaming to [RTSP](#module-rtsp), [WebRTC](#module-webrtc) or [MSE](#module-api)
+- zero-delay for many supported protocols (lowest possible streaming latency)
+- streaming from [RTSP](#source-rtsp), [RTMP](#source-rtmp), [MJPEG](#source-ffmpeg), [HLS/HTTP](#source-ffmpeg), [USB Cameras](#source-ffmpeg-device) and [other sources](#module-streams)
+- streaming to [RTSP](#module-rtsp), [WebRTC](#module-webrtc), [MSE/MP4](#module-mp4) or [MJPEG](#module-mjpeg)
 - first project in the World with support streaming from [HomeKit Cameras](#source-homekit)
-- low CPU load for supported codecs
 - on the fly transcoding for unsupported codecs via [FFmpeg](#source-ffmpeg)
 - multi-source 2-way [codecs negotiation](#codecs-negotiation)
    - mixing tracks from different sources to single stream
@@ -98,13 +97,9 @@ Don't forget to fix the rights `chmod +x go2rtc_xxx_xxx` on Linux and Mac.
     - go2rtc > Install > Start
 2. Setup [Integration](#module-hass)
 
-**Optionally:**
-
-- create `go2rtc.yaml` in your Home Assistant [config](https://www.home-assistant.io/docs/configuration) folder
-
 ### go2rtc: Docker
 
-Container [alexxit/go2rtc](https://hub.docker.com/r/alexxit/go2rtc) with support `amd64`, `386`, `arm64`, `arm`. This container same as [Home Assistant Add-on](#go2rtc-home-assistant-add-on), but can be used separately from the Home Assistant. Container has preinstalled [FFmpeg](#source-ffmpeg) and [Ngrok](#module-ngrok) applications.
+Container [alexxit/go2rtc](https://hub.docker.com/r/alexxit/go2rtc) with support `amd64`, `386`, `arm64`, `arm`. This container same as [Home Assistant Add-on](#go2rtc-home-assistant-add-on), but can be used separately from the Home Assistant. Container has preinstalled [FFmpeg](#source-ffmpeg), [Ngrok](#module-ngrok) and [Python](#source-echo).
 
 ```yaml
 services:
@@ -132,8 +127,9 @@ Available modules:
 - [api](#module-api) - HTTP API (important for WebRTC support)
 - [rtsp](#module-rtsp) - RTSP Server (important for FFmpeg support)
 - [webrtc](#module-webrtc) - WebRTC Server
-- [ngrok](#module-ngrok) - Ngrok integration (external access for private network)
+- [mp4](#module-mp4) - MSE, MP4 stream and MP4 shapshot
 - [ffmpeg](#source-ffmpeg) - FFmpeg integration
+- [ngrok](#module-ngrok) - Ngrok integration (external access for private network)
 - [hass](#module-hass) - Home Assistant integration
 - [log](#module-log) - logs config
 
@@ -148,11 +144,11 @@ Available source types:
 - [ffmpeg](#source-ffmpeg) - FFmpeg integration (`MJPEG`, `HLS`, `files` and source types)
 - [ffmpeg:device](#source-ffmpeg-device) - local USB Camera or Webcam
 - [exec](#source-exec) - advanced FFmpeg and GStreamer integration
+- [echo](#source-echo) - get stream link from bash or python
 - [homekit](#source-homekit) - streaming from HomeKit Camera
+- [ivideon](#source-ivideon) - public cameras from [Ivideon](https://tv.ivideon.com/) service
 - [hass](#source-hass) - Home Assistant integration
 
-**PS.** You can use sources like `MJPEG`, `HLS` and others via FFmpeg integration.
-
 #### Source: RTSP
 
 - Support **RTSP and RTSPS** links with multiple video and audio tracks
@@ -254,6 +250,19 @@ streams:
   stream1: exec:ffmpeg -hide_banner -re -stream_loop -1 -i /media/BigBuckBunny.mp4 -c copy -rtsp_transport tcp -f rtsp {output}
 ```
 
+#### Source: Echo
+
+Some sources may have a dynamic link. And you will need to get it using a bash or python script. Your script should echo a link to the source. RTSP, FFmpeg or any of the [supported sources](#module-streams).
+
+**Docker** and **Hass Add-on** users has preinstalled `python3`, `curl`, `jq`.
+
+Check examples in [wiki](https://github.com/AlexxIT/go2rtc/wiki/Source-Echo-examples).
+
+```yaml
+streams:
+  apple_hls: echo:python3 hls.py https://developer.apple.com/streaming/examples/basic-stream-osx-ios5.html
+```
+
 #### Source: HomeKit
 
 **Important:**
@@ -270,6 +279,15 @@ If you see a device but it does not have a pair button - it is paired to some ec
 
 **This source is in active development!** Tested only with [Aqara Camera Hub G3](https://www.aqara.com/eu/product/camera-hub-g3) (both EU and CN versions).
 
+#### Source: Ivideon
+
+Support public cameras from service [Ivideon](https://tv.ivideon.com/).
+
+```yaml
+streams:
+  quailcam: ivideon:100-tu5dkUPct39cTp9oNEN2B6/0
+```
+
 #### Source: Hass
 
 Support import camera links from [Home Assistant](https://www.home-assistant.io/) config files:
@@ -309,13 +327,9 @@ api:
 
 ### Module: RTSP
 
-You can get any stream as RTSP-stream with codecs filter:
+You can get any stream as RTSP-stream: `rtsp://192.168.1.123:8554/{stream_name}`
 
-```
-rtsp://192.168.1.123/{stream_name}?video={codec}&audio={codec1}&audio={codec2}
-```
-
-- you can omit the codecs, so one first video and one first audio will be selected
+- you can omit the codec filters, so one first video and one first audio will be selected
 - you can set `?video=copy` or just `?video`, so only one first video without audio will be selected
 - you can set multiple video or audio, so all of them will be selected
 
@@ -444,25 +458,54 @@ tunnels:
 
 ### Module: Hass
 
-**go2rtc** compatible with Home Assistant [RTSPtoWebRTC](https://www.home-assistant.io/integrations/rtsp_to_webrtc/) integration.
+If you install **go2rtc** as [Hass Add-on](#go2rtc-home-assistant-add-on) - you need to use localhost IP-address. In other cases you need to use IP-address of server with **go2rtc** application.
 
-If you install **go2rtc** as [Hass Add-on](#go2rtc-home-assistant-add-on) - you need to use localhost IP-address, example:
+#### From go2rtc to Hass
 
-- `http://127.0.0.1:1984/` to web interface
-- `rtsp://127.0.0.1:8554/camera1` to RTSP streams
+Add any supported [stream source](#module-streams) as [Generic Camera](https://www.home-assistant.io/integrations/generic/) and view stream with built-in [Stream](https://www.home-assistant.io/integrations/stream/) integration. Technology `HLS`, supported codecs: `H264`, poor latency.
 
-In other cases you need to use IP-address of server with **go2rtc** application.
+1. Add your stream to [go2rtc config](#configuration)
+2. Hass > Settings > Integrations > Add Integration > [Generic Camera](https://my.home-assistant.io/redirect/config_flow_start/?domain=generic) > `rtsp://127.0.0.1:8554/camera1`
 
-1. Add integration with link to go2rtc HTTP API:
-   - Hass > Settings > Integrations > Add Integration > [RTSPtoWebRTC](https://my.home-assistant.io/redirect/config_flow_start/?domain=rtsp_to_webrtc) > `http://127.0.0.1:1984/`
-2. Add generic camera with RTSP link:
-   - Hass > Settings > Integrations > Add Integration > [Generic Camera](https://my.home-assistant.io/redirect/config_flow_start/?domain=generic) > `rtsp://...` or `rtmp://...`
-   - you can use either direct RTSP links to cameras or take RTSP streams from **go2rtc**
-3. Use Picture Entity or Picture Glance lovelace card
-4. Open full screen card - this is should be WebRTC stream
+#### From Hass to go2rtc
+
+View almost any Hass camera using `WebRTC` technology, supported codecs `H264`/`PCMU`/`PCMA`/`OPUS`, best latency.
+
+When the stream starts - the camera `entity_id` will be added to go2rtc "on the fly". You don't need to add cameras manually to [go2rtc config](#configuration). Some cameras (like [Nest](https://www.home-assistant.io/integrations/nest/)) have a dynamic link to the stream, it will be updated each time a stream is started from the Hass interface. 
+
+1. Hass > Settings > Integrations > Add Integration > [RTSPtoWebRTC](https://my.home-assistant.io/redirect/config_flow_start/?domain=rtsp_to_webrtc) > `http://127.0.0.1:1984/`
+2. Use Picture Entity or Picture Glance lovelace card
+
+You can add camera `entity_id` to [go2rtc config](#configuration) if you need transcoding:
+
+```yaml
+streams:
+  "camera.hall": ffmpeg:{input}#video=copy#audio=opus
+```
 
 PS. Default Home Assistant lovelace cards don't support 2-way audio. You can use 2-way audio from [Add-on Web UI](https://my.home-assistant.io/redirect/supervisor_addon/?addon=a889bffc_go2rtc&repository_url=https%3A%2F%2Fgithub.com%2FAlexxIT%2Fhassio-addons). But you need use HTTPS to access the microphone. This is a browser restriction and cannot be avoided.
 
+### Module: MP4
+
+Provides several features:
+
+1. MSE stream (fMP4 over WebSocket)
+2. Camera snapshots in MP4 format (single frame), can be sent to [Telegram](https://www.telegram.org/)
+3. Progressive MP4 stream - bad format for streaming because of high latency, doesn't work in Safari 
+
+### Module: MJPEG
+
+**Important.** For stream as MJPEG format, your source MUST contain the MJPEG codec. If your camera outputs H264/H265 - you SHOULD use transcoding. With this example, your stream will have both H264 and MJPEG codecs:
+
+```yaml
+streams:
+  camera1:
+    - rtsp://rtsp:12345678@192.168.1.123/av_stream/ch0
+    - ffmpeg:rtsp://rtsp:12345678@192.168.1.123/av_stream/ch0#video=mjpeg
+```
+
+Example link to MJPEG: `http://192.168.1.123:1984/api/stream.mjpeg?src=camera1`
+
 ### Module: Log
 
 You can set different log levels for different modules.

build/hassio/Dockerfile

@@ -1,23 +1,40 @@
 ARG BUILD_FROM
-FROM $BUILD_FROM
 
-RUN apk add --no-cache git go ffmpeg
+FROM $BUILD_FROM as build
 
-ARG BUILD_ARCH
+# 1. Build go2rtc
+RUN apk add --no-cache git go
 
 RUN git clone https://github.com/AlexxIT/go2rtc \
     && cd go2rtc \
-    && CGO_ENABLED=0 go build -ldflags "-s -w" -trimpath -o /usr/local/bin
+    && CGO_ENABLED=0 go build -ldflags "-s -w" -trimpath
+
+# 2. Download ngrok
+ARG BUILD_ARCH
 
 # https://github.com/home-assistant/docker-base/blob/master/alpine/Dockerfile
 RUN if [ "${BUILD_ARCH}" = "aarch64" ]; then BUILD_ARCH="arm64"; \
     elif [ "${BUILD_ARCH}" = "armv7" ]; then BUILD_ARCH="arm"; fi \
     && cd go2rtc \
     && curl $(curl -s "https://raw.githubusercontent.com/ngrok/docker-ngrok/main/releases.json" | jq -r ".${BUILD_ARCH}.url") -o ngrok.zip \
-    && unzip ngrok -d /usr/local/bin
+    && unzip ngrok
 
-RUN rm -r /go2rtc
 
+
+# https://devopscube.com/reduce-docker-image-size/
+FROM $BUILD_FROM
+
+# 3. Copy go2rtc and ngrok to release
+COPY --from=build /go2rtc/go2rtc /usr/local/bin
+COPY --from=build /go2rtc/ngrok /usr/local/bin
+
+# 4. Install ffmpeg
+# apk base  OK: 22  MiB in 40  packages
+# ffmpeg    OK: 113 MiB in 110 packages
+# python3   OK: 161 MiB in 114 packages
+RUN apk add --no-cache ffmpeg python3
+
+# 5. Copy run to release
 COPY run.sh /
 RUN chmod a+x /run.sh
 

build/hassio/run.sh

@@ -2,13 +2,13 @@
 
 set +e
 
-# set cwd for go2rtc (for config file, Hass itegration, etc)
+# set cwd for go2rtc (for config file, Hass integration, etc)
 cd /config
 
 # add the feature to override go2rtc binary from Hass config folder
 export PATH="/config:$PATH"
 
 while true; do
-    go2rtc
-    sleep 5
+  go2rtc
+  sleep 5
 done

cmd/api/api.go

@@ -34,11 +34,10 @@ func Init() {
 	log = app.GetLogger("api")
 
 	initStatic(cfg.Mod.StaticDir)
+	initWS()
 
-	HandleFunc("/api/frame.mp4", frameHandler)
-	HandleFunc("/api/frame.raw", frameHandler)
-	HandleFunc("/api/streams", streamsHandler)
-	HandleFunc("/api/ws", apiWS)
+	HandleFunc("api/streams", streamsHandler)
+	HandleFunc("api/ws", apiWS)
 
 	// ensure we can listen without errors
 	listener, err := net.Listen("tcp", cfg.Mod.Listen)
@@ -51,14 +50,29 @@ func Init() {
 
 	go func() {
 		s := http.Server{}
+
+		if log.Trace().Enabled() {
+			s.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+				log.Trace().Stringer("url", r.URL).Msgf("[api] %s", r.Method)
+				http.DefaultServeMux.ServeHTTP(w, r)
+			})
+		}
+
 		if err = s.Serve(listener); err != nil {
 			log.Fatal().Err(err).Msg("[api] serve")
 		}
 	}()
 }
 
+// HandleFunc handle pattern with relative path:
+// - "api/streams" => "{basepath}/api/streams"
+// - "/streams"    => "/streams"
 func HandleFunc(pattern string, handler http.HandlerFunc) {
-	http.HandleFunc(basePath+pattern, handler)
+	if len(pattern) == 0 || pattern[0] != '/' {
+		pattern = basePath + "/" + pattern
+	}
+	log.Trace().Str("path", pattern).Msg("[api] register path")
+	http.HandleFunc(pattern, handler)
 }
 
 func HandleWS(msgType string, handler WSHandler) {
@@ -74,7 +88,7 @@ func streamsHandler(w http.ResponseWriter, r *http.Request) {
 
 	switch r.Method {
 	case "PUT":
-		streams.Get(src)
+		streams.New(src, src)
 		return
 	case "DELETE":
 		streams.Delete(src)
@@ -87,13 +101,10 @@ func streamsHandler(w http.ResponseWriter, r *http.Request) {
 	} else {
 		v = streams.All()
 	}
-	data, err := json.Marshal(v)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.streams] marshal")
-	}
-	if _, err = w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.streams] write")
-	}
+
+	e := json.NewEncoder(w)
+	e.SetIndent("", "  ")
+	_ = e.Encode(v)
 }
 
 func apiWS(w http.ResponseWriter, r *http.Request) {

cmd/api/keyframe.go

@@ -1,40 +0,0 @@
-package api
-
-import (
-	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/pkg/keyframe"
-	"net/http"
-	"strings"
-)
-
-func frameHandler(w http.ResponseWriter, r *http.Request) {
-	src := r.URL.Query().Get("src")
-	stream := streams.Get(src)
-	if stream == nil {
-		return
-	}
-
-	var ch = make(chan []byte)
-
-	cons := new(keyframe.Consumer)
-	cons.IsMP4 = strings.HasSuffix(r.URL.Path, ".mp4")
-	cons.Listen(func(msg interface{}) {
-		switch msg.(type) {
-		case []byte:
-			ch <- msg.([]byte)
-		}
-	})
-
-	if err := stream.AddConsumer(cons); err != nil {
-		log.Warn().Err(err).Msg("[api.frame] add consumer")
-		return
-	}
-
-	data := <-ch
-
-	stream.RemoveConsumer(cons)
-
-	if _, err := w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.frame] write")
-	}
-}

cmd/api/static.go

@@ -14,13 +14,13 @@ func initStatic(staticDir string) {
 		root = http.FS(www.Static)
 	}
 
+	base := len(basePath)
 	fileServer := http.FileServer(root)
 
-	HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
-		if basePath != "" {
-			r.URL.Path = r.URL.Path[len(basePath):]
+	HandleFunc("", func(w http.ResponseWriter, r *http.Request) {
+		if base > 0 {
+			r.URL.Path = r.URL.Path[base:]
 		}
-
 		fileServer.ServeHTTP(w, r)
 	})
 }

cmd/api/ws.go

@@ -4,16 +4,42 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/gorilla/websocket"
 	"net/http"
+	"net/url"
+	"strings"
 	"sync"
 )
 
-type WSHandler func(ctx *Context, msg *streamer.Message)
-
-var apiWsUp = websocket.Upgrader{
-	ReadBufferSize:  1024,
-	WriteBufferSize: 512000,
+func initWS() {
+	wsUp = &websocket.Upgrader{
+		ReadBufferSize:  1024,
+		WriteBufferSize: 512000,
+	}
+	wsUp.CheckOrigin = func(r *http.Request) bool {
+		origin := r.Header["Origin"]
+		if len(origin) == 0 {
+			return true
+		}
+		o, err := url.Parse(origin[0])
+		if err != nil {
+			return false
+		}
+		if o.Host == r.Host {
+			return true
+		}
+		log.Trace().Msgf("[api.ws] origin: %s, host: %s", o.Host, r.Host)
+		// some users change Nginx external port using Docker port
+		// so origin will be with a port and host without
+		if i := strings.IndexByte(o.Host, ':'); i > 0 {
+			return o.Host[:i] == r.Host
+		}
+		return false
+	}
 }
 
+var wsUp *websocket.Upgrader
+
+type WSHandler func(ctx *Context, msg *streamer.Message)
+
 type Context struct {
 	Conn     *websocket.Conn
 	Request  *http.Request
@@ -24,7 +50,7 @@ type Context struct {
 }
 
 func (ctx *Context) Upgrade(w http.ResponseWriter, r *http.Request) (err error) {
-	ctx.Conn, err = apiWsUp.Upgrade(w, r, nil)
+	ctx.Conn, err = wsUp.Upgrade(w, r, nil)
 	ctx.Request = r
 	return
 }

cmd/debug/debug.go

@@ -10,8 +10,8 @@ import (
 )
 
 func Init() {
-	api.HandleFunc("/api/stack", stackHandler)
-	api.HandleFunc("/api/exit", exitHandler)
+	api.HandleFunc("api/stack", stackHandler)
+	api.HandleFunc("api/exit", exitHandler)
 
 	streams.HandleFunc("null", nullHandler)
 }

cmd/echo/echo.go

@@ -0,0 +1,29 @@
+package echo
+
+import (
+	"bytes"
+	"github.com/AlexxIT/go2rtc/cmd/app"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/shell"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"os/exec"
+)
+
+func Init() {
+	log := app.GetLogger("echo")
+
+	streams.HandleFunc("echo", func(url string) (streamer.Producer, error) {
+		args := shell.QuoteSplit(url[5:])
+
+		b, err := exec.Command(args[0], args[1:]...).Output()
+		if err != nil {
+			return nil, err
+		}
+
+		b = bytes.TrimSpace(b)
+
+		log.Debug().Str("url", url).Msgf("[echo] %s", b)
+
+		return streams.GetProducer(string(b))
+	})
+}

cmd/exec/exec.go

@@ -8,6 +8,7 @@ import (
 	"github.com/AlexxIT/go2rtc/cmd/rtsp"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
 	pkg "github.com/AlexxIT/go2rtc/pkg/rtsp"
+	"github.com/AlexxIT/go2rtc/pkg/shell"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/rs/zerolog"
 	"os"
@@ -49,7 +50,7 @@ func Handle(url string) (streamer.Producer, error) {
 	)
 
 	// remove `exec:`
-	args := QuoteSplit(url[5:])
+	args := shell.QuoteSplit(url[5:])
 	cmd := exec.Command(args[0], args[1:]...)
 
 	if log.Trace().Enabled() {
@@ -64,6 +65,8 @@ func Handle(url string) (streamer.Producer, error) {
 
 	log.Debug().Str("url", url).Msg("[exec] run")
 
+	ts := time.Now()
+
 	if err := cmd.Start(); err != nil {
 		log.Error().Err(err).Str("url", url).Msg("[exec]")
 		return nil, err
@@ -75,6 +78,7 @@ func Handle(url string) (streamer.Producer, error) {
 		log.Error().Str("url", url).Msg("[exec] timeout")
 		return nil, errors.New("timeout")
 	case prod := <-ch:
+		log.Debug().Stringer("launch", time.Since(ts)).Msg("[exec] run")
 		return prod, nil
 	}
 }
@@ -83,39 +87,3 @@ func Handle(url string) (streamer.Producer, error) {
 
 var log zerolog.Logger
 var waiters map[string]chan streamer.Producer
-
-func QuoteSplit(s string) []string {
-	var a []string
-
-	for len(s) > 0 {
-		is := strings.IndexByte(s, ' ')
-		if is >= 0 {
-			// skip prefix and double spaces
-			if is == 0 {
-				// goto next symbol
-				s = s[1:]
-				continue
-			}
-
-			// check if quote in word
-			if i := strings.IndexByte(s[:is], '"'); i >= 0 {
-				// search quote end
-				if is = strings.Index(s, `" `); is > 0 {
-					is += 1
-				} else {
-					is = -1
-				}
-			}
-		}
-
-		if is >= 0 {
-			a = append(a, strings.ReplaceAll(s[:is], `"`, ""))
-			s = s[is+1:]
-		} else {
-			//add last word
-			a = append(a, s)
-			break
-		}
-	}
-	return a
-}

cmd/ffmpeg/README.md

@@ -39,3 +39,5 @@
 - https://html5test.com/
 - https://trac.ffmpeg.org/wiki/Capture/Webcam
 - https://trac.ffmpeg.org/wiki/DirectShow
+- https://stackoverflow.com/questions/53207692/libav-mjpeg-encoding-and-huffman-table
+- https://github.com/tuupola/esp_video/blob/master/README.md

cmd/ffmpeg/device/devices.go

@@ -15,7 +15,7 @@ import (
 func Init() {
 	log = app.GetLogger("exec")
 
-	api.HandleFunc("/api/devices", handle)
+	api.HandleFunc("api/devices", handle)
 }
 
 func GetInput(src string) (string, error) {

cmd/ffmpeg/ffmpeg.go

@@ -26,7 +26,7 @@ func Init() {
 		"rtsp": "-fflags nobuffer -flags low_delay -rtsp_transport tcp -i {input}",
 
 		// output
-		"out": "-rtsp_transport tcp -f rtsp {output}",
+		"output": "-rtsp_transport tcp -f rtsp {output}",
 
 		// `-g 30` - group of picture, GOP, keyframe interval
 		// `-preset superfast` - we can't use ultrafast because it doesn't support `-profile main -level 4.1`
@@ -37,6 +37,7 @@ func Init() {
 		"h264/ultra": "-codec:v libx264 -g 30 -preset ultrafast -tune zerolatency",
 		"h264/high":  "-codec:v libx264 -g 30 -preset superfast -tune zerolatency",
 		"h265":       "-codec:v libx265 -g 30 -preset ultrafast -tune zerolatency",
+		"mjpeg":      "-codec:v mjpeg -force_duplicated_matrix 1 -huffman 0 -pix_fmt yuvj420p",
 		"opus":       "-codec:a libopus -ar 48000 -ac 2",
 		"pcmu":       "-codec:a pcm_mulaw -ar 8000 -ac 1",
 		"pcmu/16000": "-codec:a pcm_mulaw -ar 16000 -ac 1",
@@ -55,18 +56,36 @@ func Init() {
 		s = s[7:] // remove `ffmpeg:`
 
 		var query url.Values
+		var queryVideo, queryAudio bool
 		if i := strings.IndexByte(s, '#'); i > 0 {
 			query = parseQuery(s[i+1:])
+			queryVideo = query["video"] != nil
+			queryAudio = query["audio"] != nil
 			s = s[:i]
+		} else {
+			// by default query both video and audio
+			queryVideo = true
+			queryAudio = true
 		}
 
 		var input string
 		if i := strings.IndexByte(s, ':'); i > 0 {
 			switch s[:i] {
-			case "http", "https":
+			case "http", "https", "rtmp":
 				input = strings.Replace(tpl["http"], "{input}", s, 1)
 			case "rtsp", "rtsps":
-				input = strings.Replace(tpl["rtsp"], "{input}", s, 1)
+				// https://ffmpeg.org/ffmpeg-protocols.html#rtsp
+				// skip unnecessary input tracks
+				switch {
+				case queryVideo && queryAudio:
+					input = "-allowed_media_types video+audio "
+				case queryVideo:
+					input = "-allowed_media_types video "
+				case queryAudio:
+					input = "-allowed_media_types audio "
+				}
+
+				input += strings.Replace(tpl["rtsp"], "{input}", s, 1)
 			}
 		}
 
@@ -108,17 +127,17 @@ func Init() {
 				}
 			}
 
-			if query["video"] == nil {
-				s += " -vn"
-			}
-			if query["audio"] == nil {
+			switch {
+			case queryVideo && !queryAudio:
 				s += " -an"
+			case queryAudio && !queryVideo:
+				s += " -vn"
 			}
 		} else {
 			s += " -c copy"
 		}
 
-		s += " " + tpl["out"]
+		s += " " + tpl["output"]
 
 		return exec.Handle(s)
 	})

cmd/hass/api.go

@@ -0,0 +1,149 @@
+package hass
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/cmd/webrtc"
+	"net/http"
+	"net/url"
+	"strings"
+)
+
+func initAPI() {
+	ok := func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		_, _ = w.Write([]byte(`{"status":1,"payload":{}}`))
+	}
+
+	// support https://www.home-assistant.io/integrations/rtsp_to_webrtc/
+	api.HandleFunc("/static", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	api.HandleFunc("/streams", ok)
+
+	api.HandleFunc("/stream/", func(w http.ResponseWriter, r *http.Request) {
+		switch {
+		// /stream/{id}/add
+		case strings.HasSuffix(r.RequestURI, "/add"):
+			var v addJSON
+			if err := json.NewDecoder(r.Body).Decode(&v); err != nil {
+				return
+			}
+
+			// we can get three types of links:
+			// 1. link to go2rtc stream: rtsp://...:8554/{stream_name}
+			// 2. static link to Hass camera
+			// 3. dynamic link to Hass camera
+			stream := streams.Get(v.Name)
+			if stream == nil {
+				// check if it is rtsp link to go2rtc
+				stream = rtspStream(v.Channels.First.Url)
+				if stream != nil {
+					streams.New(v.Name, stream)
+				} else {
+					stream = streams.New(v.Name, "{input}")
+				}
+			}
+
+			stream.SetSource(v.Channels.First.Url)
+
+			ok(w, r)
+
+		// /stream/{id}/channel/0/webrtc
+		default:
+			i := strings.IndexByte(r.RequestURI[8:], '/')
+			name := r.RequestURI[8 : 8+i]
+
+			stream := streams.Get(name)
+			if stream == nil {
+				w.WriteHeader(http.StatusNotFound)
+				return
+			}
+
+			if err := r.ParseForm(); err != nil {
+				log.Error().Err(err).Msg("[api.hass] parse form")
+				return
+			}
+
+			s := r.FormValue("data")
+			offer, err := base64.StdEncoding.DecodeString(s)
+			if err != nil {
+				log.Error().Err(err).Msg("[api.hass] sdp64 decode")
+				return
+			}
+
+			s, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
+			if err != nil {
+				log.Error().Err(err).Msg("[api.hass] exchange SDP")
+				return
+			}
+
+			s = base64.StdEncoding.EncodeToString([]byte(s))
+			_, _ = w.Write([]byte(s))
+		}
+	})
+
+	// api from RTSPtoWebRTC
+	api.HandleFunc("/stream", func(w http.ResponseWriter, r *http.Request) {
+		if err := r.ParseForm(); err != nil {
+			return
+		}
+
+		str := r.FormValue("sdp64")
+		offer, err := base64.StdEncoding.DecodeString(str)
+		if err != nil {
+			return
+		}
+
+		src := r.FormValue("url")
+		src, err = url.QueryUnescape(src)
+		if err != nil {
+			return
+		}
+
+		stream := streams.Get(src)
+		if stream == nil {
+			if stream = rtspStream(src); stream != nil {
+				streams.New(src, stream)
+			} else {
+				stream = streams.New(src, src)
+			}
+		}
+
+		str, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
+		if err != nil {
+			return
+		}
+
+		v := struct {
+			Answer string `json:"sdp64"`
+		}{
+			Answer: base64.StdEncoding.EncodeToString([]byte(str)),
+		}
+
+		w.Header().Set("Content-Type", "application/json")
+		_ = json.NewEncoder(w).Encode(v)
+	})
+}
+
+func rtspStream(url string) *streams.Stream {
+	if strings.HasPrefix(url, "rtsp://") {
+		if i := strings.IndexByte(url[7:], '/'); i > 0 {
+			return streams.Get(url[8+i:])
+		}
+	}
+	return nil
+}
+
+type addJSON struct {
+	Name     string `json:"name"`
+	Channels struct {
+		First struct {
+			//Name string `json:"name"`
+			Url string `json:"url"`
+		} `json:"0"`
+	} `json:"channels"`
+}

cmd/hass/hass.go

@@ -1,20 +1,14 @@
 package hass
 
 import (
-	"encoding/base64"
 	"encoding/json"
 	"fmt"
-	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/app"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/cmd/webrtc"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/rs/zerolog"
-	"net/http"
-	"net/url"
 	"os"
 	"path"
-	"strings"
 )
 
 func Init() {
@@ -28,11 +22,7 @@ func Init() {
 
 	log = app.GetLogger("hass")
 
-	// support https://www.home-assistant.io/integrations/rtsp_to_webrtc/
-	api.HandleFunc("/static", func(w http.ResponseWriter, r *http.Request) {
-		w.WriteHeader(http.StatusOK)
-	})
-	api.HandleFunc("/stream", handler)
+	initAPI()
 
 	// support load cameras from Hass config file
 	filename := path.Join(conf.Mod.Config, ".storage/core.config_entries")
@@ -78,73 +68,13 @@ func Init() {
 			continue
 		}
 
-		log.Info().Str("url", "hass:" + entrie.Title).Msg("[hass] load stream")
+		log.Info().Str("url", "hass:"+entrie.Title).Msg("[hass] load stream")
 		//streams.Get("hass:" + entrie.Title)
 	}
 }
 
 var log zerolog.Logger
 
-func handler(w http.ResponseWriter, r *http.Request) {
-	if err := r.ParseForm(); err != nil {
-		log.Error().Err(err).Msg("[api.hass] parse form")
-		return
-	}
-
-	src := r.FormValue("url")
-	src, err := url.QueryUnescape(src)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] query unescape")
-		return
-	}
-
-	str := r.FormValue("sdp64")
-
-	offer, err := base64.StdEncoding.DecodeString(str)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] sdp64 decode")
-		return
-	}
-
-	// check if stream links to our rtsp server
-	if strings.HasPrefix(src, "rtsp://") {
-		i := strings.IndexByte(src[7:], '/')
-		if i > 0 && streams.Has(src[8+i:]) {
-			src = src[8+i:]
-		}
-	}
-
-	stream := streams.Get(src)
-	if stream == nil {
-		log.Error().Str("url", src).Msg("[api.hass] unsupported source")
-		return
-	}
-
-	str, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] exchange SDP")
-		return
-	}
-
-	resp := struct {
-		Answer string `json:"sdp64"`
-	}{
-		Answer: base64.StdEncoding.EncodeToString([]byte(str)),
-	}
-
-	data, err := json.Marshal(resp)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] marshal JSON")
-		return
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	if _, err = w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.hass] write")
-		return
-	}
-}
-
 type entries struct {
 	Data struct {
 		Entries []struct {

cmd/homekit/api.go

@@ -54,15 +54,7 @@ func apiHandler(w http.ResponseWriter, r *http.Request) {
 			items = append(items, device)
 		}
 
-		data, err := json.Marshal(items)
-		if err != nil {
-			log.Error().Err(err).Msg("[api.homekit]")
-			return
-		}
-
-		if _, err = w.Write(data); err != nil {
-			log.Error().Err(err).Msg("[api.homekit]")
-		}
+		_= json.NewEncoder(w).Encode(items)
 
 	case "POST":
 		// TODO: post params...

cmd/homekit/homekit.go

@@ -14,7 +14,7 @@ func Init() {
 
 	streams.HandleFunc("homekit", streamHandler)
 
-	api.HandleFunc("/api/homekit", apiHandler)
+	api.HandleFunc("api/homekit", apiHandler)
 }
 
 var log zerolog.Logger

cmd/ivideon/ivideon.go

@@ -0,0 +1,19 @@
+package ivideon
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/ivideon"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"strings"
+)
+
+func Init() {
+	streams.HandleFunc("ivideon", func(url string) (streamer.Producer, error) {
+		id := strings.Replace(url[8:], "/", ":", 1)
+		prod := ivideon.NewClient(id)
+		if err := prod.Dial(); err != nil {
+			return nil, err
+		}
+		return prod, nil
+	})
+}

cmd/mjpeg/mjpeg.go

@@ -0,0 +1,54 @@
+package mjpeg
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/mjpeg"
+	"github.com/rs/zerolog/log"
+	"net/http"
+	"strconv"
+)
+
+func Init() {
+	api.HandleFunc("api/stream.mjpeg", handler)
+}
+
+const header = "--frame\r\nContent-Type: image/jpeg\r\nContent-Length: "
+
+func handler(w http.ResponseWriter, r *http.Request) {
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan struct{})
+
+	cons := &mjpeg.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			data := []byte(header + strconv.Itoa(len(msg)))
+			data = append(data, 0x0D, 0x0A, 0x0D, 0x0A)
+			data = append(data, msg...)
+			data = append(data, 0x0D, 0x0A)
+
+			if _, err := w.Write(data); err != nil {
+				exit <- struct{}{}
+			}
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.mjpeg] add consumer")
+		return
+	}
+
+	w.Header().Set("Content-Type", `multipart/x-mixed-replace; boundary=frame`)
+
+	<-exit
+
+	stream.RemoveConsumer(cons)
+
+	//log.Trace().Msg("[api.mjpeg] close")
+}

cmd/mp4/mp4.go

@@ -0,0 +1,138 @@
+package mp4
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/app"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/mp4"
+	"github.com/rs/zerolog"
+	"net/http"
+	"strconv"
+	"strings"
+)
+
+func Init() {
+	log = app.GetLogger("mp4")
+
+	api.HandleWS(MsgTypeMSE, handlerWS)
+
+	api.HandleFunc("api/frame.mp4", handlerKeyframe)
+	api.HandleFunc("api/stream.mp4", handlerMP4)
+}
+
+var log zerolog.Logger
+
+func handlerKeyframe(w http.ResponseWriter, r *http.Request) {
+	if isChromeFirst(w, r) {
+		return
+	}
+
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan []byte)
+
+	cons := &mp4.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			exit <- msg
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] add consumer")
+		return
+	}
+
+	defer stream.RemoveConsumer(cons)
+
+	w.Header().Set("Content-Type", cons.MimeType())
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] init")
+		return
+	}
+	data = append(data, <-exit...)
+
+	// Apple Safari won't show frame without length
+	w.Header().Set("Content-Length", strconv.Itoa(len(data)))
+
+	if _, err := w.Write(data); err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] add consumer")
+	}
+}
+
+func handlerMP4(w http.ResponseWriter, r *http.Request) {
+	if isChromeFirst(w, r) || isSafari(w, r) {
+		return
+	}
+
+	log.Trace().Msgf("[api.mp4] %+v", r)
+
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan struct{})
+
+	cons := &mp4.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			if _, err := w.Write(msg); err != nil {
+				exit <- struct{}{}
+			}
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.mp4] add consumer")
+		return
+	}
+
+	defer stream.RemoveConsumer(cons)
+
+	w.Header().Set("Content-Type", cons.MimeType())
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Error().Err(err).Msg("[api.mp4] init")
+		return
+	}
+
+	if _, err = w.Write(data); err != nil {
+		log.Error().Err(err).Msg("[api.mp4] write")
+		return
+	}
+
+	<-exit
+
+	log.Trace().Msg("[api.mp4] close")
+}
+
+func isChromeFirst(w http.ResponseWriter, r *http.Request) bool {
+	// Chrome 105 does two requests: without Range and with `Range: bytes=0-`
+	if strings.Contains(r.UserAgent(), " Chrome/") {
+		if r.Header.Values("Range") == nil {
+			w.Header().Set("Content-Type", "video/mp4")
+			w.WriteHeader(http.StatusOK)
+			return true
+		}
+	}
+	return false
+}
+
+func isSafari(w http.ResponseWriter, r *http.Request) bool {
+	if r.Header.Get("Range") == "bytes=0-1" {
+		handlerKeyframe(w, r)
+		return true
+	}
+	return false
+}

cmd/mp4/mse.go

@@ -1,35 +1,34 @@
-package mse
+package mp4
 
 import (
 	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/pkg/mse"
+	"github.com/AlexxIT/go2rtc/pkg/mp4"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/rs/zerolog/log"
 )
 
-func Init() {
-	api.HandleWS("mse", handler)
-}
+const MsgTypeMSE = "mse" // fMP4
 
-func handler(ctx *api.Context, msg *streamer.Message) {
+func handlerWS(ctx *api.Context, msg *streamer.Message) {
 	src := ctx.Request.URL.Query().Get("src")
-	stream := streams.Get(src)
+	stream := streams.GetOrNew(src)
 	if stream == nil {
 		return
 	}
 
-	cons := new(mse.Consumer)
+	cons := &mp4.Consumer{}
 	cons.UserAgent = ctx.Request.UserAgent()
 	cons.RemoteAddr = ctx.Request.RemoteAddr
+
 	cons.Listen(func(msg interface{}) {
 		switch msg.(type) {
 		case *streamer.Message, []byte:
 			ctx.Write(msg)
 		}
 	})
+
 	if err := stream.AddConsumer(cons); err != nil {
-		log.Warn().Err(err).Msg("[api.mse] Add consumer")
+		log.Warn().Err(err).Msg("[api.mse] add consumer")
 		ctx.Error(err)
 		return
 	}
@@ -38,5 +37,16 @@ func handler(ctx *api.Context, msg *streamer.Message) {
 		stream.RemoveConsumer(cons)
 	})
 
-	cons.Init()
+	ctx.Write(&streamer.Message{
+		Type: MsgTypeMSE, Value: cons.MimeType(),
+	})
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Warn().Err(err).Msg("[api.mse] init")
+		ctx.Error(err)
+		return
+	}
+
+	ctx.Write(data)
 }

cmd/rtsp/rtsp.go

@@ -8,6 +8,7 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/tcp"
 	"github.com/rs/zerolog"
 	"net"
+	"strings"
 )
 
 func Init() {
@@ -47,6 +48,15 @@ var OnProducer func(conn streamer.Producer) bool // TODO: maybe rewrite...
 var log zerolog.Logger
 
 func rtspHandler(url string) (streamer.Producer, error) {
+	backchannel := true
+
+	if i := strings.IndexByte(url, '#'); i > 0 {
+		if url[i+1:] == "backchannel=0" {
+			backchannel = false
+		}
+		url = url[:i]
+	}
+
 	conn, err := rtsp.NewClient(url)
 	if err != nil {
 		return nil, err
@@ -67,8 +77,12 @@ func rtspHandler(url string) (streamer.Producer, error) {
 		return nil, err
 	}
 
-	conn.Backchannel = true
+	conn.Backchannel = backchannel
 	if err = conn.Describe(); err != nil {
+		if !backchannel {
+			return nil, err
+		}
+
 		// second try without backchannel, we need to reconnect
 		if err = conn.Dial(); err != nil {
 			return nil, err
@@ -143,15 +157,15 @@ func worker(address string) {
 
 					log.Debug().Str("stream", name).Msg("[rtsp] new producer")
 
-					str := streams.Get(conn.URL.Path[1:])
-					if str == nil {
+					stream := streams.Get(name)
+					if stream == nil {
 						return
 					}
 
-					str.AddProducer(conn)
+					stream.AddProducer(conn)
 
 					onDisconnect = func() {
-						str.RemoveProducer(conn)
+						stream.RemoveProducer(conn)
 					}
 
 				case streamer.StatePlaying:

cmd/streams/producer.go

@@ -2,6 +2,7 @@ package streams
 
 import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"strings"
 	"sync"
 )
 
@@ -17,7 +18,9 @@ const (
 type Producer struct {
 	streamer.Element
 
-	url     string
+	url      string
+	template string
+
 	element streamer.Producer
 	tracks  []*streamer.Track
 
@@ -25,6 +28,13 @@ type Producer struct {
 	mx    sync.Mutex
 }
 
+func (p *Producer) SetSource(s string) {
+	if p.template == "" {
+		p.template = p.url
+	}
+	p.url = strings.Replace(p.template, "{input}", s, 1)
+}
+
 func (p *Producer) GetMedias() []*streamer.Media {
 	p.mx.Lock()
 	defer p.mx.Unlock()
@@ -79,7 +89,11 @@ func (p *Producer) start() {
 	log.Debug().Str("url", p.url).Msg("[streams] start producer")
 
 	p.state = stateStart
-	go p.element.Start()
+	go func() {
+		if err := p.element.Start(); err != nil {
+			log.Warn().Err(err).Str("url", p.url).Msg("[streams] start")
+		}
+	}()
 }
 
 func (p *Producer) stop() {

cmd/streams/stream.go

@@ -17,25 +17,34 @@ type Stream struct {
 }
 
 func NewStream(source interface{}) *Stream {
-	s := new(Stream)
-
 	switch source := source.(type) {
 	case string:
+		s := new(Stream)
 		prod := &Producer{url: source}
 		s.producers = append(s.producers, prod)
+		return s
 	case []interface{}:
+		s := new(Stream)
 		for _, source := range source {
 			prod := &Producer{url: source.(string)}
 			s.producers = append(s.producers, prod)
 		}
+		return s
+	case *Stream:
+		return source
 	case map[string]interface{}:
 		return NewStream(source["url"])
 	case nil:
+		return new(Stream)
 	default:
 		panic("wrong source type")
 	}
+}
 
-	return s
+func (s *Stream) SetSource(source string) {
+	for _, prod := range s.producers {
+		prod.SetSource(source)
+	}
 }
 
 func (s *Stream) AddConsumer(cons streamer.Consumer) (err error) {
@@ -94,6 +103,11 @@ func (s *Stream) AddConsumer(cons streamer.Consumer) (err error) {
 
 func (s *Stream) RemoveConsumer(cons streamer.Consumer) {
 	for i, consumer := range s.consumers {
+		if consumer == nil {
+			log.Warn().Msgf("empty consumer: %+v\n", s)
+			continue
+		}
+
 		if consumer.element == cons {
 			// remove consumer pads from all producers
 			for _, track := range consumer.tracks {
@@ -106,6 +120,11 @@ func (s *Stream) RemoveConsumer(cons streamer.Consumer) {
 	}
 
 	for _, producer := range s.producers {
+		if producer == nil {
+			log.Warn().Msgf("empty producer: %+v\n", s)
+			continue
+		}
+
 		var sink bool
 		for _, track := range producer.tracks {
 			if len(track.Sink) > 0 {

cmd/streams/streams.go

@@ -24,10 +24,19 @@ func Init() {
 	}
 }
 
-func Get(src string) *Stream {
+func Get(name string) *Stream {
+	return streams[name]
+}
+
+func New(name string, source interface{}) *Stream {
+	stream := NewStream(source)
+	streams[name] = stream
+	return stream
+}
+
+func GetOrNew(src string) *Stream {
 	if stream, ok := streams[src]; ok {
 		return stream
-
 	}
 
 	if !HasProducer(src) {
@@ -35,17 +44,8 @@ func Get(src string) *Stream {
 	}
 
 	log.Info().Str("url", src).Msg("[streams] create new stream")
-	stream := NewStream(src)
-	streams[src] = stream
-	return stream
-}
 
-func Has(src string) bool {
-	return streams[src] != nil
-}
-
-func New(name string, source interface{}) {
-	streams[name] = NewStream(source)
+	return New(src, src)
 }
 
 func Delete(name string) {

cmd/webrtc/candidates.go

@@ -5,7 +5,6 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/AlexxIT/go2rtc/pkg/webrtc"
 	"github.com/pion/sdp/v3"
-	"strings"
 )
 
 var candidates []string
@@ -32,15 +31,11 @@ func addCanditates(answer string) (string, error) {
 	}
 
 	for _, address := range candidates {
-		if strings.HasPrefix(address, "stun:") {
-			ip, err := webrtc.GetPublicIP()
-			if err != nil {
-				log.Warn().Err(err).Msg("[webrtc] public IP")
-				continue
-			}
-			address = ip.String() + address[4:]
-
-			log.Debug().Str("addr", address).Msg("[webrtc] stun public address")
+		var err error
+		address, err = webrtc.LookupIP(address)
+		if err != nil {
+			log.Warn().Err(err).Msg("[webrtc] candidate")
+			continue
 		}
 
 		cand, err := webrtc.NewCandidate(address)

go.mod

@@ -54,8 +54,4 @@ replace (
 	github.com/brutella/dnssd v1.2.2 => github.com/rblenkinsopp/dnssd v1.2.3-0.20220516082132-0923f3c787a1
 	// RTP tlv8 fix
 	github.com/brutella/hap v0.0.17 => github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657
-	// MSE update
-	github.com/deepch/vdk v0.0.19 => github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e
-	// AES_256_CM_HMAC_SHA1_80 support
-	github.com/pion/srtp/v2 v2.0.10 => github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10
 )

go.sum

@@ -1,9 +1,5 @@
 github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657 h1:FUzXAJfm6sRLJ8T6vfzvy/Hm3aioX8+fbxgx2VZoI78=
 github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657/go.mod h1:c2vEL5pzjRWEx07sa32kTVjzI9bBVlstrwBwKe3DlJ0=
-github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10 h1:4aKRthhmkYcStKuk1hcyvkeNJ/BDx5BTIvYmDO9ZJvg=
-github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10/go.mod h1:5TtM9yw6lsH0ppNCehB/EjEUli7VkUgKSPJqWVqbhQ4=
-github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e h1:NAgHHZB+JUN3/J4/yq1q1EAc8xwJ8bb/Qp0AcjkfzAA=
-github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e/go.mod h1:KqQ/KU3hOc4a62l/jPRH5Hiz5fhTq5cGCl8IqeCxWQI=
 github.com/brutella/dnssd v1.2.3 h1:4fBLjZjPH7SbcHhEcIJhZcC9nOhIDZ0m3rn9bjl1/i0=
 github.com/brutella/dnssd v1.2.3/go.mod h1:JoW2sJUrmVIef25G6lrLj7HS6Xdwh6q8WUIvMkkBYXs=
 github.com/cheekybits/genny v1.0.0/go.mod h1:+tQajlRqAUrPI7DOSpB0XAqZYtQakVtB7wXkRAgjxjQ=
@@ -11,6 +7,8 @@ github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534/go.mod h1
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/deepch/vdk v0.0.19 h1:r6xYyBTtXEIEh+csO0XHT00sI7xLF+hQFkJE9/go5II=
+github.com/deepch/vdk v0.0.19/go.mod h1:7ydHfSkflMZxBXfWR79dMjrT54xzvLxnPaByOa9Jpzg=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
 github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
 github.com/go-chi/chi v1.5.4 h1:QHdzF2szwjqVV4wmByUnTcsbIg7UGaQ0tPF2t5GcAIs=
@@ -103,6 +101,8 @@ github.com/pion/sdp/v3 v3.0.5/go.mod h1:iiFWFpQO8Fy3S5ldclBkpXqmWy02ns78NOKoLLL0
 github.com/pion/srtp v1.5.1 h1:9Q3jAfslYZBt+C69SI/ZcONJh9049JUHZWYRRf5KEKw=
 github.com/pion/srtp v1.5.1/go.mod h1:B+QgX5xPeQTNc1CJStJPHzOlHK66ViMDWTT0HZTCkcA=
 github.com/pion/srtp/v2 v2.0.9/go.mod h1:5TtM9yw6lsH0ppNCehB/EjEUli7VkUgKSPJqWVqbhQ4=
+github.com/pion/srtp/v2 v2.0.10 h1:b8ZvEuI+mrL8hbr/f1YiJFB34UMrOac3R3N1yq2UN0w=
+github.com/pion/srtp/v2 v2.0.10/go.mod h1:XEeSWaK9PfuMs7zxXyiN252AHPbH12NX5q/CFDWtUuA=
 github.com/pion/stun v0.3.5 h1:uLUCBCkQby4S1cf6CGuR9QrVOKcvUwFeemaC865QHDg=
 github.com/pion/stun v0.3.5/go.mod h1:gDMim+47EeEtfWogA37n6qXZS88L5V6LqFcf+DZA2UA=
 github.com/pion/transport v0.6.0/go.mod h1:iWZ07doqOosSLMhZ+FXUTq+TamDoXSllxpbGcfkCmbE=
@@ -121,7 +121,7 @@ github.com/pion/udp v0.1.0/go.mod h1:BPELIjbwE9PRbd/zxI/KYBnbo7B6+oA6YuEaNE8lths
 github.com/pion/udp v0.1.1 h1:8UAPvyqmsxK8oOjloDk4wUt63TzFe9WEJkg5lChlj7o=
 github.com/pion/udp v0.1.1/go.mod h1:6AFo+CMdKQm7UiA0eUPA8/eVCTx8jBIITLZHc9DWX5M=
 github.com/pion/webrtc/v2 v2.2.26/go.mod h1:XMZbZRNHyPDe1gzTIHFcQu02283YO45CbiwFgKvXnmc=
-github.com/pion/webrtc/v3 v3.1.41/go.mod h1:sUcW9SFPEWerDqGOBmdYEMfRvbdd7rgwo4bNzfsXww4=
+github.com/pion/webrtc/v3 v3.1.42/go.mod h1:ffD9DulDrPxyWvDPUIPAOSAWx9GUlOExiJPf7cCcMLA=
 github.com/pion/webrtc/v3 v3.1.43 h1:YT3ZTO94UT4kSBvZnRAH82+0jJPUruiKr9CEstdlQzk=
 github.com/pion/webrtc/v3 v3.1.43/go.mod h1:G/J8k0+grVsjC/rjCZ24AKoCCxcFFODgh7zThNZGs0M=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=

main.go

@@ -4,11 +4,14 @@ import (
 	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/app"
 	"github.com/AlexxIT/go2rtc/cmd/debug"
+	"github.com/AlexxIT/go2rtc/cmd/echo"
 	"github.com/AlexxIT/go2rtc/cmd/exec"
 	"github.com/AlexxIT/go2rtc/cmd/ffmpeg"
 	"github.com/AlexxIT/go2rtc/cmd/hass"
 	"github.com/AlexxIT/go2rtc/cmd/homekit"
-	"github.com/AlexxIT/go2rtc/cmd/mse"
+	"github.com/AlexxIT/go2rtc/cmd/ivideon"
+	"github.com/AlexxIT/go2rtc/cmd/mjpeg"
+	"github.com/AlexxIT/go2rtc/cmd/mp4"
 	"github.com/AlexxIT/go2rtc/cmd/ngrok"
 	"github.com/AlexxIT/go2rtc/cmd/rtmp"
 	"github.com/AlexxIT/go2rtc/cmd/rtsp"
@@ -24,20 +27,25 @@ func main() {
 	app.Init()     // init config and logs
 	streams.Init() // load streams list
 
+	api.Init() // init HTTP API server
+
+	echo.Init()
+
 	rtsp.Init()   // add support RTSP client and RTSP server
 	rtmp.Init()   // add support RTMP client
 	exec.Init()   // add support exec scheme (depends on RTSP server)
 	ffmpeg.Init() // add support ffmpeg scheme (depends on exec scheme)
 	hass.Init()   // add support hass scheme
 
-	api.Init() // init HTTP API server
-
 	webrtc.Init()
-	mse.Init()
+	mp4.Init()
+	mjpeg.Init()
 
 	srtp.Init()
 	homekit.Init()
 
+	ivideon.Init()
+
 	ngrok.Init()
 	debug.Init()
 

pkg/README.md

@@ -0,0 +1,5 @@
+## Useful links
+
+- https://www.wowza.com/blog/streaming-protocols
+- https://vimeo.com/blog/post/rtmp-stream/
+- https://sanjeev-pandey.medium.com/understanding-the-mpeg-4-moov-atom-pseudo-streaming-in-mp4-93935e1b9e9a

pkg/h264/README.md

@@ -1,3 +1,14 @@
+## RTP H264
+
+Camera | NALu
+-------|-----
+EZVIZ C3S          | 7f, 8f, 28:28:28 -> 5t, 28:28:28 -> 1t, 1t, 1t, 1t
+Sonoff GK-200MP2-B | 28:28:28 -> 5t, 1t, 1t, 1t
+Dahua IPC-K42      | 7f, 8f, 28:28:28 -> 5t, 28:28:28 -> 1t, 28:28:28 -> 1t
+FFmpeg copy        | 5t, 1t, 1t, 28:28:28 -> 1t, 28:28:28 -> 1t
+FFmpeg h264        | 24 -> 6:5:5:5:5t, 24 -> 1:1:1:1t, 28:28:28 -> 5f, 28:28:28 -> 5f, 28:28:28 -> 5t
+FFmpeg resize      | 6f, 28:28:28 -> 5f, 28... -> 5t, 24 -> 1:1f, 24 -> 1:1t
+
 ## WebRTC
 
 Video codec	    | Media string | Device
@@ -25,3 +36,4 @@ H.264/high      | avc1.6400xx  | FFmpeg superfast
 - [AVC levels](https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels)
 - [AVC profiles table](https://developer.mozilla.org/ru/docs/Web/Media/Formats/codecs_parameter)
 - [Supported Media for Google Cast](https://developers.google.com/cast/docs/media)
+- [Two stream formats, Annex-B, AVCC (H.264) and HVCC (H.265)](https://www.programmersought.com/article/3901815022/)

pkg/h264/avc.go

@@ -58,3 +58,21 @@ func RepairAVC(track *streamer.Track) streamer.WrapperFunc {
 		}
 	}
 }
+
+func SplitAVC(data []byte) [][]byte {
+	var nals [][]byte
+	for {
+		// get AVC length
+		size := int(binary.BigEndian.Uint32(data))
+
+		// check if multiple items in one packet
+		if size+4 < len(data) {
+			nals = append(nals, data[:size+4])
+			data = data[size+4:]
+		} else {
+			nals = append(nals, data)
+			break
+		}
+	}
+	return nals
+}

pkg/h264/helper.go

@@ -9,6 +9,7 @@ import (
 const (
 	NALUTypePFrame = 1
 	NALUTypeIFrame = 5
+	NALUTypeSEI    = 6
 	NALUTypeSPS    = 7
 	NALUTypePPS    = 8
 )
@@ -17,6 +18,17 @@ func NALUType(b []byte) byte {
 	return b[4] & 0x1F
 }
 
+func IsKeyframe(b []byte) bool {
+	return NALUType(b) == NALUTypeIFrame
+}
+
+func GetProfileLevelID(fmtp string) string {
+	if fmtp == "" {
+		return ""
+	}
+	return streamer.Between(fmtp, "profile-level-id=", ";")
+}
+
 func GetParameterSet(fmtp string) (sps, pps []byte) {
 	if fmtp == "" {
 		return

pkg/h264/rtp.go

@@ -22,49 +22,62 @@ func RTPDepay(track *streamer.Track) streamer.WrapperFunc {
 		return func(packet *rtp.Packet) error {
 			//nalUnitType := packet.Payload[0] & 0x1F
 			//fmt.Printf(
-			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d\n",
+			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d\n",
 			//	track.Codec.Name, nalUnitType, len(packet.Payload), packet.Timestamp,
-			//	packet.PayloadType, packet.SSRC,
+			//	packet.PayloadType, packet.SSRC, packet.SequenceNumber,
 			//)
 
-			// NALu packets can be split in different ways:
-			// - single type 7 and type 8 packets
-			// - join type 7 and type 8 packet (type 24)
-			// - split type 5 on multiple 28 packets
-			// - split type 5 on multiple separate 28 packets
-			units, err := depack.Unmarshal(packet.Payload)
-			if len(units) == 0 || err != nil {
+			data, err := depack.Unmarshal(packet.Payload)
+			if len(data) == 0 || err != nil {
 				return nil
 			}
 
-			for len(units) > 0 {
-				i := int(binary.BigEndian.Uint32(units)) + 4
-				unit := units[:i] // NAL Unit with AVC header
-				units = units[i:]
+			for {
+				unitType := NALUType(data)
+				//fmt.Printf("[H264] nalu: %2d, size: %6d\n", unitType, len(data))
+
+				// multiple 5 and 1 in one payload is OK
+				if unitType != NALUTypeIFrame && unitType != NALUTypePFrame {
+					i := int(binary.BigEndian.Uint32(data)) + 4
+					if i < len(data) {
+						data0 := data[:i] // NAL Unit with AVC header
+						data = data[i:]
+						switch unitType {
+						case NALUTypeSPS:
+							sps = data0
+							continue
+						case NALUTypePPS:
+							pps = data0
+							continue
+						case NALUTypeSEI:
+							// some unnecessary text information
+							continue
+						}
+					}
+				}
 
-				unitType := NALUType(unit)
-				//fmt.Printf("[H264] type: %2d, size: %6d\n", unitType, i)
 				switch unitType {
 				case NALUTypeSPS:
-					//println("new SPS")
-					sps = unit
-					continue
+					sps = data
+					return nil
 				case NALUTypePPS:
-					//println("new PPS")
-					pps = unit
-					continue
+					pps = data
+					return nil
+				case NALUTypeSEI:
+					// some unnecessary text information
+					return nil
 				}
 
 				// ffmpeg with `-tune zerolatency` enable option `-x264opts sliced-threads=1`
 				// and every NALU will be sliced to multiple NALUs
 				if !packet.Marker {
-					buffer = append(buffer, unit...)
-					continue
+					buffer = append(buffer, data...)
+					return nil
 				}
 
 				if buffer != nil {
-					buffer = append(buffer, unit...)
-					unit = buffer
+					buffer = append(buffer, data...)
+					data = buffer
 					buffer = nil
 				}
 
@@ -88,13 +101,9 @@ func RTPDepay(track *streamer.Track) streamer.WrapperFunc {
 
 				clone = *packet
 				clone.Version = RTPPacketVersionAVC
-				clone.Payload = unit
-				if err = push(&clone); err != nil {
-					return err
-				}
+				clone.Payload = data
+				return push(&clone)
 			}
-
-			return nil
 		}
 	}
 }

pkg/h265/README.md

@@ -0,0 +1,3 @@
+## Useful links
+
+- https://datatracker.ietf.org/doc/html/rfc7798

pkg/h265/helper.go

@@ -0,0 +1,35 @@
+package h265
+
+import (
+	"encoding/base64"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+)
+
+const (
+	NALUnitTypeIFrame = 19
+)
+
+func NALUnitType(b []byte) byte {
+	return b[4] >> 1
+}
+
+func IsKeyframe(b []byte) bool {
+	return NALUnitType(b) == NALUnitTypeIFrame
+}
+
+func GetParameterSet(fmtp string) (vps, sps, pps []byte) {
+	if fmtp == "" {
+		return
+	}
+
+	s := streamer.Between(fmtp, "sprop-vps=", ";")
+	vps, _ = base64.StdEncoding.DecodeString(s)
+
+	s = streamer.Between(fmtp, "sprop-sps=", ";")
+	sps, _ = base64.StdEncoding.DecodeString(s)
+
+	s = streamer.Between(fmtp, "sprop-pps=", ";")
+	pps, _ = base64.StdEncoding.DecodeString(s)
+
+	return
+}

pkg/h265/rtp.go

@@ -0,0 +1,153 @@
+package h265
+
+import (
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/deepch/vdk/codec/h265parser"
+	"github.com/pion/rtp"
+)
+
+func RTPDepay(track *streamer.Track) streamer.WrapperFunc {
+	vps, sps, pps := GetParameterSet(track.Codec.FmtpLine)
+
+	var buffer []byte
+
+	return func(push streamer.WriterFunc) streamer.WriterFunc {
+		return func(packet *rtp.Packet) error {
+			nut := (packet.Payload[0] >> 1) & 0x3f
+			//fmt.Printf(
+			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d\n",
+			//	track.Codec.Name, nut, len(packet.Payload), packet.Timestamp,
+			//	packet.PayloadType, packet.SSRC, packet.SequenceNumber,
+			//)
+
+			switch nut {
+			case h265parser.NAL_UNIT_UNSPECIFIED_49:
+				data := packet.Payload
+				switch data[2] >> 6 {
+				case 2: // begin
+					buffer = []byte{
+						(data[0] & 0x81) | (data[2] & 0x3f << 1), data[1],
+					}
+					buffer = append(buffer, data[3:]...)
+					return nil
+				case 0: // continue
+					buffer = append(buffer, data[3:]...)
+					return nil
+				case 1: // end
+					packet.Payload = append(buffer, data[3:]...)
+				}
+			case h265parser.NAL_UNIT_VPS:
+				vps = packet.Payload
+				return nil
+			case h265parser.NAL_UNIT_SPS:
+				sps = packet.Payload
+				return nil
+			case h265parser.NAL_UNIT_PPS:
+				pps = packet.Payload
+				return nil
+			default:
+				//panic("not implemented")
+			}
+
+			var clone rtp.Packet
+
+			nut = (packet.Payload[0] >> 1) & 0x3f
+			if nut >= h265parser.NAL_UNIT_CODED_SLICE_BLA_W_LP && nut <= h265parser.NAL_UNIT_CODED_SLICE_CRA {
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(vps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(sps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(pps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+			}
+
+			clone = *packet
+			clone.Version = h264.RTPPacketVersionAVC
+			clone.Payload = h264.EncodeAVC(packet.Payload)
+
+			return push(&clone)
+		}
+	}
+}
+
+// SafariPay - generate Safari friendly payload for H265
+func SafariPay(mtu uint16) streamer.WrapperFunc {
+	sequencer := rtp.NewRandomSequencer()
+	size := int(mtu - 12) // rtp.Header size
+
+	var buffer []byte
+
+	return func(push streamer.WriterFunc) streamer.WriterFunc {
+		return func(packet *rtp.Packet) error {
+			if packet.Version != h264.RTPPacketVersionAVC {
+				return push(packet)
+			}
+
+			data := packet.Payload
+			data[0] = 0
+			data[1] = 0
+			data[2] = 0
+			data[3] = 1
+
+			var start byte
+
+			nut := (data[4] >> 1) & 0b111111
+			//fmt.Printf("[H265] nut: %2d, size: %6d, data: %16x\n", nut, len(data), data[4:20])
+			switch {
+			case nut >= h265parser.NAL_UNIT_VPS && nut <= h265parser.NAL_UNIT_PPS:
+				buffer = append(buffer, data...)
+				return nil
+			case nut >= h265parser.NAL_UNIT_CODED_SLICE_BLA_W_LP && nut <= h265parser.NAL_UNIT_CODED_SLICE_CRA:
+				buffer = append([]byte{3}, buffer...)
+				data = append(buffer, data...)
+				start = 1
+			default:
+				data = append([]byte{2}, data...)
+				start = 0
+			}
+
+			for len(data) > size {
+				clone := rtp.Packet{
+					Header: rtp.Header{
+						Version:        2,
+						Marker:         false,
+						SequenceNumber: sequencer.NextSequenceNumber(),
+						Timestamp:      packet.Timestamp,
+					},
+					Payload: data[:size],
+				}
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				data = append([]byte{start}, data[size:]...)
+			}
+
+			clone := rtp.Packet{
+				Header: rtp.Header{
+					Version:        2,
+					Marker:         true,
+					SequenceNumber: sequencer.NextSequenceNumber(),
+					Timestamp:      packet.Timestamp,
+				},
+				Payload: data,
+			}
+			return push(&clone)
+		}
+	}
+}

pkg/ivideon/client.go

@@ -0,0 +1,286 @@
+package ivideon
+
+import (
+	"bytes"
+	"encoding/base64"
+	"encoding/binary"
+	"encoding/json"
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/deepch/vdk/codec/h264parser"
+	"github.com/deepch/vdk/format/fmp4/fmp4io"
+	"github.com/gorilla/websocket"
+	"github.com/pion/rtp"
+	"io"
+	"net/http"
+	"strings"
+	"time"
+)
+
+type Client struct {
+	streamer.Element
+
+	ID string
+
+	conn   *websocket.Conn
+	medias []*streamer.Media
+	tracks map[byte]*streamer.Track
+
+	closed bool
+
+	msg *message
+	t0  time.Time
+
+	buffer chan []byte
+}
+
+func NewClient(id string) *Client {
+	return &Client{ID: id}
+}
+
+func (c *Client) Dial() (err error) {
+	resp, err := http.Get(
+		"https://openapi-alpha.ivideon.com/cameras/" + c.ID +
+			"/live_stream?op=GET&access_token=public&q=2&" +
+			"video_codecs=h264&format=ws-fmp4",
+	)
+
+	data, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return err
+	}
+
+	var v liveResponse
+	if err = json.Unmarshal(data, &v); err != nil {
+		return err
+	}
+
+	if !v.Success {
+		return fmt.Errorf("wrong response: %s", data)
+	}
+
+	c.conn, _, err = websocket.DefaultDialer.Dial(v.Result.URL, nil)
+	if err != nil {
+		return err
+	}
+
+	if err = c.getTracks(); err != nil {
+		_ = c.conn.Close()
+		return err
+	}
+
+	return nil
+}
+
+func (c *Client) Handle() error {
+	c.buffer = make(chan []byte, 5)
+	// add delay to the stream for smooth playing (not a best solution)
+	c.t0 = time.Now().Add(time.Second)
+
+	// processing stream in separate thread for lower delay between packets
+	go c.worker()
+
+	_, data, err := c.conn.ReadMessage()
+	if err != nil {
+		return err
+	}
+
+	track := c.tracks[c.msg.Track]
+	if track != nil {
+		c.buffer <- data
+	}
+
+	// we have one unprocessed msg after getTracks
+	for {
+		_, data, err = c.conn.ReadMessage()
+		if err != nil {
+			return err
+		}
+
+		var msg message
+		if err = json.Unmarshal(data, &msg); err != nil {
+			return err
+		}
+
+		switch msg.Type {
+		case "stream-init":
+			continue
+
+		case "fragment":
+			_, data, err = c.conn.ReadMessage()
+			if err != nil {
+				return err
+			}
+
+			track = c.tracks[msg.Track]
+			if track != nil {
+				c.buffer <- data
+			}
+
+		default:
+			return fmt.Errorf("wrong message type: %s", data)
+		}
+	}
+}
+
+func (c *Client) Close() error {
+	if c.conn == nil {
+		return nil
+	}
+	close(c.buffer)
+	c.closed = true
+	return c.conn.Close()
+}
+
+func (c *Client) getTracks() error {
+	c.tracks = map[byte]*streamer.Track{}
+
+	for {
+		_, data, err := c.conn.ReadMessage()
+		if err != nil {
+			return err
+		}
+
+		var msg message
+		if err = json.Unmarshal(data, &msg); err != nil {
+			return err
+		}
+
+		switch msg.Type {
+		case "stream-init":
+			s := msg.CodecString
+			i := strings.IndexByte(s, '.')
+			if i > 0 {
+				s = s[:i]
+			}
+
+			switch s {
+			case "avc1": // avc1.4d0029
+				// skip multiple identical init
+				if c.tracks[msg.TrackID] != nil {
+					continue
+				}
+
+				codec := streamer.NewCodec(streamer.CodecH264)
+				codec.FmtpLine = "profile-level-id=" + msg.CodecString[i+1:]
+				codec.PayloadType = h264.PayloadTypeAVC
+
+				i = bytes.Index(msg.Data, []byte("avcC")) - 4
+				if i < 0 {
+					return fmt.Errorf("wrong AVC: %s", msg.Data)
+				}
+
+				avccLen := binary.BigEndian.Uint32(msg.Data[i:])
+				data = msg.Data[i+8 : i+int(avccLen)]
+
+				record := h264parser.AVCDecoderConfRecord{}
+				if _, err = record.Unmarshal(data); err != nil {
+					return err
+				}
+
+				codec.FmtpLine += ";sprop-parameter-sets=" +
+					base64.StdEncoding.EncodeToString(record.SPS[0]) + "," +
+					base64.StdEncoding.EncodeToString(record.PPS[0])
+
+				media := &streamer.Media{
+					Kind:      streamer.KindVideo,
+					Direction: streamer.DirectionSendonly,
+					Codecs:    []*streamer.Codec{codec},
+				}
+				c.medias = append(c.medias, media)
+
+				track := &streamer.Track{
+					Direction: streamer.DirectionSendonly,
+					Codec:     codec,
+				}
+				c.tracks[msg.TrackID] = track
+
+			case "mp4a": // mp4a.40.2
+			}
+
+		case "fragment":
+			c.msg = &msg
+			return nil
+
+		default:
+			return fmt.Errorf("wrong message type: %s", data)
+		}
+	}
+}
+
+func (c *Client) worker() {
+	var track *streamer.Track
+	for _, track = range c.tracks {
+		break
+	}
+
+	for data := range c.buffer {
+		moof := &fmp4io.MovieFrag{}
+		if _, err := moof.Unmarshal(data, 0); err != nil {
+			continue
+		}
+
+		moofLen := binary.BigEndian.Uint32(data)
+		_ = moofLen
+
+		mdat := moof.Unknowns[0]
+		if mdat.Tag() != fmp4io.MDAT {
+			continue
+		}
+		i, _ := mdat.Pos() // offset, size
+		data = data[i+8:]
+
+		traf := moof.Tracks[0]
+		ts := uint32(traf.DecodeTime.Time)
+
+		//println("!!!", (time.Duration(ts) * time.Millisecond).String(), time.Since(c.t0).String())
+
+		for _, entry := range traf.Run.Entries {
+			// synchronize framerate for WebRTC and MSE
+			d := time.Duration(ts)*time.Millisecond - time.Since(c.t0)
+			if d < 0 {
+				d = time.Duration(entry.Duration) * time.Millisecond / 2
+			}
+			time.Sleep(d)
+
+			// can be SPS, PPS and IFrame in one packet
+			for _, payload := range h264.SplitAVC(data[:entry.Size]) {
+				packet := &rtp.Packet{
+					// ivideon clockrate=1000, RTP clockrate=90000
+					Header:  rtp.Header{Timestamp: ts * 90},
+					Payload: payload,
+				}
+				_ = track.WriteRTP(packet)
+			}
+
+			data = data[entry.Size:]
+			ts += entry.Duration
+		}
+
+		if len(data) != 0 {
+			continue
+		}
+	}
+}
+
+type liveResponse struct {
+	Result struct {
+		URL string `json:"url"`
+	} `json:"result"`
+	Success bool `json:"success"`
+}
+
+type message struct {
+	Type string `json:"type"`
+
+	CodecString string `json:"codec_string"`
+	Data        []byte `json:"data"`
+	TrackID     byte   `json:"track_id"`
+
+	Track      byte    `json:"track"`
+	StartTime  float32 `json:"start_time"`
+	Duration   float32 `json:"duration"`
+	IsKey      bool    `json:"is_key"`
+	DataOffset uint32  `json:"data_offset"`
+}

pkg/ivideon/streamer.go

@@ -0,0 +1,31 @@
+package ivideon
+
+import (
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+)
+
+func (c *Client) GetMedias() []*streamer.Media {
+	return c.medias
+}
+
+func (c *Client) GetTrack(media *streamer.Media, codec *streamer.Codec) *streamer.Track {
+	for _, track := range c.tracks {
+		if track.Codec == codec {
+			return track
+		}
+	}
+	panic(fmt.Sprintf("wrong media/codec: %+v %+v", media, codec))
+}
+
+func (c *Client) Start() error {
+	err := c.Handle()
+	if c.closed {
+		return nil
+	}
+	return err
+}
+
+func (c *Client) Stop() error {
+	return c.Close()
+}

pkg/keyframe/consumer.go

@@ -1,72 +0,0 @@
-package keyframe
-
-import (
-	"github.com/AlexxIT/go2rtc/pkg/h264"
-	"github.com/AlexxIT/go2rtc/pkg/mp4"
-	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/pion/rtp"
-)
-
-var annexB = []byte{0, 0, 0, 1}
-
-type Consumer struct {
-	streamer.Element
-	IsMP4 bool
-}
-
-func (k *Consumer) GetMedias() []*streamer.Media {
-	// support keyframe extraction only for one coded...
-	codec := streamer.NewCodec(streamer.CodecH264)
-	return []*streamer.Media{
-		{
-			Kind: streamer.KindVideo, Direction: streamer.DirectionRecvonly,
-			Codecs: []*streamer.Codec{codec},
-		},
-	}
-}
-
-func (k *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
-	// sps and pps without AVC headers
-	sps, pps := h264.GetParameterSet(track.Codec.FmtpLine)
-
-	push := func(packet *rtp.Packet) error {
-		// TODO: remove it, unnecessary
-		if packet.Version != h264.RTPPacketVersionAVC {
-			panic("wrong packet type")
-		}
-
-		switch h264.NALUType(packet.Payload) {
-		case h264.NALUTypeSPS:
-			sps = packet.Payload[4:] // remove AVC header
-		case h264.NALUTypePPS:
-			pps = packet.Payload[4:] // remove AVC header
-		case h264.NALUTypeIFrame:
-			if sps == nil || pps == nil {
-				return nil
-			}
-
-			var data []byte
-
-			if k.IsMP4 {
-				data = mp4.MarshalMP4(sps, pps, packet.Payload)
-			} else {
-				data = append(data, annexB...)
-				data = append(data, sps...)
-				data = append(data, annexB...)
-				data = append(data, pps...)
-				data = append(data, annexB...)
-				data = append(data, packet.Payload[4:]...)
-			}
-
-			k.Fire(data)
-		}
-		return nil
-	}
-
-	if !h264.IsAVC(track.Codec) {
-		wrapper := h264.RTPDepay(track)
-		push = wrapper(push)
-	}
-
-	return track.Bind(push)
-}

pkg/mjpeg/README.md

@@ -0,0 +1,4 @@
+## Useful links
+
+- https://www.rfc-editor.org/rfc/rfc2435
+- https://github.com/GStreamer/gst-plugins-good/blob/master/gst/rtp/gstrtpjpegdepay.c

pkg/mjpeg/consumer.go

@@ -0,0 +1,87 @@
+package mjpeg
+
+import (
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/pion/rtp"
+)
+
+type Consumer struct {
+	streamer.Element
+
+	UserAgent  string
+	RemoteAddr string
+
+	codecs []*streamer.Codec
+	start  bool
+
+	send int
+}
+
+func (c *Consumer) GetMedias() []*streamer.Media {
+	return []*streamer.Media{{
+		Kind:      streamer.KindVideo,
+		Direction: streamer.DirectionRecvonly,
+		Codecs:    []*streamer.Codec{{Name: streamer.CodecJPEG}},
+	}}
+}
+
+func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
+	var header, payload []byte
+
+	push := func(packet *rtp.Packet) error {
+		//fmt.Printf(
+		//	"[RTP] codec: %s, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d, mark: %v\n",
+		//	track.Codec.Name, len(packet.Payload), packet.Timestamp,
+		//	packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker,
+		//)
+
+		// https://www.rfc-editor.org/rfc/rfc2435#section-3.1
+		b := packet.Payload
+
+		// 3.1.  JPEG header
+		t := b[4]
+
+		// 3.1.7.  Restart Marker header
+		if 64 <= t && t <= 127 {
+			b = b[12:] // skip it
+		} else {
+			b = b[8:]
+		}
+
+		if header == nil {
+			var lqt, cqt []byte
+
+			// 3.1.8.  Quantization Table header
+			q := packet.Payload[5]
+			if q >= 128 {
+				lqt = b[4:68]
+				cqt = b[68:132]
+				b = b[132:]
+			} else {
+				lqt, cqt = MakeTables(q)
+			}
+
+			w := uint16(packet.Payload[6]) << 3
+			h := uint16(packet.Payload[7]) << 3
+			//fmt.Printf("t: %d, q: %d, w: %d, h: %d\n", t, q, w, h)
+			header = MakeHeaders(t, w, h, lqt, cqt)
+		}
+
+		// 3.1.9.  JPEG Payload
+		payload = append(payload, b...)
+
+		if packet.Marker {
+			b = append(header, payload...)
+			if end := b[len(b)-2:]; end[0] != 0xFF && end[1] != 0xD9 {
+				b = append(b, 0xFF, 0xD9)
+			}
+			c.Fire(b)
+
+			header = nil
+			payload = nil
+		}
+
+		return nil
+	}
+	return track.Bind(push)
+}

pkg/mjpeg/rfc2435.go

@@ -0,0 +1,182 @@
+package mjpeg
+
+// RFC 2435. Appendix A
+
+var jpeg_luma_quantizer = []byte{
+	16, 11, 10, 16, 24, 40, 51, 61,
+	12, 12, 14, 19, 26, 58, 60, 55,
+	14, 13, 16, 24, 40, 57, 69, 56,
+	14, 17, 22, 29, 51, 87, 80, 62,
+	18, 22, 37, 56, 68, 109, 103, 77,
+	24, 35, 55, 64, 81, 104, 113, 92,
+	49, 64, 78, 87, 103, 121, 120, 101,
+	72, 92, 95, 98, 112, 100, 103, 99,
+}
+var jpeg_chroma_quantizer = []byte{
+	17, 18, 24, 47, 99, 99, 99, 99,
+	18, 21, 26, 66, 99, 99, 99, 99,
+	24, 26, 56, 99, 99, 99, 99, 99,
+	47, 66, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+}
+
+func MakeTables(q byte) (lqt, cqt []byte) {
+	var factor int
+
+	switch {
+	case q < 1:
+		factor = 1
+	case q > 99:
+		factor = 99
+	default:
+		factor = int(q)
+	}
+
+	if q < 50 {
+		factor = 5000 / factor
+	} else if q > 99 {
+		factor = 200 - factor*2
+	}
+
+	lqt = make([]byte, 64)
+	cqt = make([]byte, 64)
+
+	for i := 0; i < 64; i++ {
+		lq := (int(jpeg_luma_quantizer[i])*factor + 50) / 100
+		cq := (int(jpeg_chroma_quantizer[i])*factor + 50) / 100
+
+		/* Limit the quantizers to 1 <= q <= 255 */
+		switch {
+		case lq < 1:
+			lqt[i] = 1
+		case lq > 255:
+			lqt[i] = 255
+		default:
+			lqt[i] = byte(lq)
+		}
+
+		switch {
+		case cq < 1:
+			cqt[i] = 1
+		case cq > 255:
+			cqt[i] = 255
+		default:
+			cqt[i] = byte(cq)
+		}
+	}
+
+	return
+}
+
+// RFC 2435. Appendix B
+
+var lum_dc_codelens = []byte{
+	0, 1, 5, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0,
+}
+var lum_dc_symbols = []byte{
+	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+}
+var lum_ac_codelens = []byte{
+	0, 2, 1, 3, 3, 2, 4, 3, 5, 5, 4, 4, 0, 0, 1, 0x7d,
+}
+var lum_ac_symbols = []byte{
+	0x01, 0x02, 0x03, 0x00, 0x04, 0x11, 0x05, 0x12,
+	0x21, 0x31, 0x41, 0x06, 0x13, 0x51, 0x61, 0x07,
+	0x22, 0x71, 0x14, 0x32, 0x81, 0x91, 0xa1, 0x08,
+	0x23, 0x42, 0xb1, 0xc1, 0x15, 0x52, 0xd1, 0xf0,
+	0x24, 0x33, 0x62, 0x72, 0x82, 0x09, 0x0a, 0x16,
+	0x17, 0x18, 0x19, 0x1a, 0x25, 0x26, 0x27, 0x28,
+	0x29, 0x2a, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39,
+	0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49,
+	0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59,
+	0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69,
+	0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79,
+	0x7a, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89,
+	0x8a, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98,
+	0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
+	0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6,
+	0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3, 0xc4, 0xc5,
+	0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2, 0xd3, 0xd4,
+	0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xe1, 0xe2,
+	0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea,
+	0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,
+	0xf9, 0xfa,
+}
+var chm_dc_codelens = []byte{
+	0, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
+}
+var chm_dc_symbols = []byte{
+	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+}
+var chm_ac_codelens = []byte{
+	0, 2, 1, 2, 4, 4, 3, 4, 7, 5, 4, 4, 0, 1, 2, 0x77,
+}
+var chm_ac_symbols = []byte{
+	0x00, 0x01, 0x02, 0x03, 0x11, 0x04, 0x05, 0x21,
+	0x31, 0x06, 0x12, 0x41, 0x51, 0x07, 0x61, 0x71,
+	0x13, 0x22, 0x32, 0x81, 0x08, 0x14, 0x42, 0x91,
+	0xa1, 0xb1, 0xc1, 0x09, 0x23, 0x33, 0x52, 0xf0,
+	0x15, 0x62, 0x72, 0xd1, 0x0a, 0x16, 0x24, 0x34,
+	0xe1, 0x25, 0xf1, 0x17, 0x18, 0x19, 0x1a, 0x26,
+	0x27, 0x28, 0x29, 0x2a, 0x35, 0x36, 0x37, 0x38,
+	0x39, 0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,
+	0x49, 0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58,
+	0x59, 0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68,
+	0x69, 0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78,
+	0x79, 0x7a, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
+	0x88, 0x89, 0x8a, 0x92, 0x93, 0x94, 0x95, 0x96,
+	0x97, 0x98, 0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5,
+	0xa6, 0xa7, 0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4,
+	0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3,
+	0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2,
+	0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda,
+	0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9,
+	0xea, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,
+	0xf9, 0xfa,
+}
+
+func MakeHeaders(t byte, w, h uint16, lqt, cqt []byte) []byte {
+	// Appendix A from https://www.rfc-editor.org/rfc/rfc2435
+	p := []byte{0xFF, 0xD8}
+
+	p = MakeQuantHeader(p, lqt, 0)
+	p = MakeQuantHeader(p, cqt, 1)
+
+	if t == 0 {
+		t = 0x21
+	} else {
+		t = 0x22
+	}
+
+	p = append(p,
+		0xFF, 0xC0, 0, 17, 8,
+		byte(h>>8), byte(h&0xFF),
+		byte(w>>8), byte(w&0xFF),
+		3, 0, t, 0, 1, 0x11, 1, 2, 0x11, 1,
+	)
+
+	p = MakeHuffmanHeader(p, lum_dc_codelens, lum_dc_symbols, 0, 0)
+	p = MakeHuffmanHeader(p, lum_ac_codelens, lum_ac_symbols, 0, 1)
+	p = MakeHuffmanHeader(p, chm_dc_codelens, chm_dc_symbols, 1, 0)
+	p = MakeHuffmanHeader(p, chm_ac_codelens, chm_ac_symbols, 1, 1)
+
+	return append(p, 0xFF, 0xDA, 0, 12, 3, 0, 0, 1, 0x11, 2, 0x11, 0, 63, 0)
+}
+
+func MakeQuantHeader(p []byte, qt []byte, tableNo byte) []byte {
+	p = append(p, 0xFF, 0xDB, 0, 67, tableNo)
+	return append(p, qt...)
+}
+
+func MakeHuffmanHeader(p, codelens, symbols []byte, tableNo, tableClass byte) []byte {
+	p = append(p,
+		0xFF, 0xC4, 0,
+		byte(3+len(codelens)+len(symbols)),
+		(tableClass<<4)|tableNo,
+	)
+	p = append(p, codelens...)
+	return append(p, symbols...)
+}

pkg/mp4/README.md

@@ -0,0 +1,23 @@
+## HEVC
+
+Browser     | avc1 | hvc1 | hev1
+------------|------|------|---
+Mac Chrome  | +    | -    | +
+Mac Safari  | +    | +    | -
+iOS 15?     | +    | +    | -
+Mac Firefox | +    | -    | -
+iOS 12      | +    | -    | -
+Android 13  | +    | -    | -
+
+```
+ffmpeg -i input-hev1.mp4 -c:v copy -tag:v hvc1 -c:a copy output-hvc1.mp4
+Stream #0:0(eng): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, progressive), 720x404, 164 kb/s, 29.97 fps,
+Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, progressive), 720x404, 164 kb/s, 29.97 fps,
+```
+
+## Useful links
+
+- https://stackoverflow.com/questions/63468587/what-hevc-codec-tag-to-use-with-fmp4-hvc1-or-hev1
+- https://stackoverflow.com/questions/32152090/encode-h265-to-hvc1-codec
+- https://jellyfin.org/docs/general/clients/codec-support.html
+- https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding

pkg/mp4/const.go

@@ -0,0 +1,94 @@
+package mp4
+
+import (
+	"encoding/binary"
+	"github.com/deepch/vdk/format/mp4/mp4io"
+	"time"
+)
+
+var matrix = [9]int32{0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x40000000}
+var time0 = time.Date(1904, time.January, 1, 0, 0, 0, 0, time.UTC)
+
+func FTYP() []byte {
+	b := make([]byte, 0x18)
+	binary.BigEndian.PutUint32(b, 0x18)
+	copy(b[0x04:], "ftyp")
+	copy(b[0x08:], "iso5")
+	copy(b[0x10:], "iso5")
+	copy(b[0x14:], "avc1")
+	return b
+}
+
+func MOOV() *mp4io.Movie {
+	return &mp4io.Movie{
+		Header: &mp4io.MovieHeader{
+			PreferredRate:     1,
+			PreferredVolume:   1,
+			Matrix:            matrix,
+			NextTrackId:       -1,
+			Duration:          0,
+			TimeScale:         1000,
+			CreateTime:        time0,
+			ModifyTime:        time0,
+			PreviewTime:       time0,
+			PreviewDuration:   time0,
+			PosterTime:        time0,
+			SelectionTime:     time0,
+			SelectionDuration: time0,
+			CurrentTime:       time0,
+		},
+		MovieExtend: &mp4io.MovieExtend{
+			Tracks: []*mp4io.TrackExtend{
+				{
+					TrackId:               1,
+					DefaultSampleDescIdx:  1,
+					DefaultSampleDuration: 40,
+				},
+			},
+		},
+	}
+}
+
+func TRAK() *mp4io.Track {
+	return &mp4io.Track{
+		// trak > tkhd
+		Header: &mp4io.TrackHeader{
+			TrackId:    int32(1), // change me
+			Flags:      0x0007,   // 7 ENABLED IN-MOVIE IN-PREVIEW
+			Duration:   0,        // OK
+			Matrix:     matrix,
+			CreateTime: time0,
+			ModifyTime: time0,
+		},
+		// trak > mdia
+		Media: &mp4io.Media{
+			// trak > mdia > mdhd
+			Header: &mp4io.MediaHeader{
+				TimeScale:  1000,
+				Duration:   0,
+				Language:   0x55C4,
+				CreateTime: time0,
+				ModifyTime: time0,
+			},
+			// trak > mdia > minf
+			Info: &mp4io.MediaInfo{
+				// trak > mdia > minf > dinf
+				Data: &mp4io.DataInfo{
+					Refer: &mp4io.DataRefer{
+						Url: &mp4io.DataReferUrl{
+							Flags: 0x000001, // self reference
+						},
+					},
+				},
+				// trak > mdia > minf > stbl
+				Sample: &mp4io.SampleTable{
+					SampleDesc:    &mp4io.SampleDesc{},
+					TimeToSample:  &mp4io.TimeToSample{},
+					SampleToChunk: &mp4io.SampleToChunk{},
+					SampleSize:    &mp4io.SampleSize{},
+					ChunkOffset:   &mp4io.ChunkOffset{},
+				},
+			},
+		},
+	}
+}

pkg/mp4/consumer.go

@@ -1,27 +1,23 @@
-package mse
+package mp4
 
 import (
 	"encoding/json"
+	"fmt"
 	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/deepch/vdk/av"
-	"github.com/deepch/vdk/codec/h264parser"
-	"github.com/deepch/vdk/format/mp4f"
 	"github.com/pion/rtp"
-	"time"
 )
 
-const MsgTypeMSE = "mse"
-
 type Consumer struct {
 	streamer.Element
 
 	UserAgent  string
 	RemoteAddr string
 
-	muxer   *mp4f.Muxer
-	streams []av.CodecData
-	start   bool
+	muxer  *Muxer
+	codecs []*streamer.Codec
+	start  bool
 
 	send int
 }
@@ -33,14 +29,16 @@ func (c *Consumer) GetMedias() []*streamer.Media {
 			Direction: streamer.DirectionRecvonly,
 			Codecs: []*streamer.Codec{
 				{Name: streamer.CodecH264, ClockRate: 90000},
-			},
-		}, {
-			Kind:      streamer.KindAudio,
-			Direction: streamer.DirectionRecvonly,
-			Codecs: []*streamer.Codec{
-				{Name: streamer.CodecAAC, ClockRate: 16000},
+				{Name: streamer.CodecH265, ClockRate: 90000},
 			},
 		},
+		//{
+		//	Kind:      streamer.KindAudio,
+		//	Direction: streamer.DirectionRecvonly,
+		//	Codecs: []*streamer.Codec{
+		//		{Name: streamer.CodecAAC, ClockRate: 16000},
+		//	},
+		//},
 	}
 }
 
@@ -48,18 +46,7 @@ func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *strea
 	codec := track.Codec
 	switch codec.Name {
 	case streamer.CodecH264:
-		idx := int8(len(c.streams))
-
-		sps, pps := h264.GetParameterSet(codec.FmtpLine)
-		stream, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
-		if err != nil {
-			return nil
-		}
-		c.streams = append(c.streams, stream)
-
-		pkt := av.Packet{Idx: idx, CompositionTime: time.Millisecond}
-
-		ts2time := time.Second / time.Duration(codec.ClockRate)
+		c.codecs = append(c.codecs, track.Codec)
 
 		push := func(packet *rtp.Packet) error {
 			if packet.Version != h264.RTPPacketVersionAVC {
@@ -69,7 +56,6 @@ func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *strea
 			switch h264.NALUType(packet.Payload) {
 			case h264.NALUTypeIFrame:
 				c.start = true
-				pkt.IsKeyFrame = true
 			case h264.NALUTypePFrame:
 				if !c.start {
 					return nil
@@ -78,17 +64,9 @@ func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *strea
 				return nil
 			}
 
-			pkt.Data = packet.Payload
-			newTime := time.Duration(packet.Timestamp) * ts2time
-			if pkt.Time > 0 {
-				pkt.Duration = newTime - pkt.Time
-			}
-			pkt.Time = newTime
-
-			for _, buf := range c.muxer.WritePacketV5(pkt) {
-				c.send += len(buf)
-				c.Fire(buf)
-			}
+			buf := c.muxer.Marshal(packet)
+			c.send += len(buf)
+			c.Fire(buf)
 
 			return nil
 		}
@@ -98,30 +76,60 @@ func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *strea
 			push = wrapper(push)
 		}
 
+		return track.Bind(push)
+
+	case streamer.CodecH265:
+		c.codecs = append(c.codecs, track.Codec)
+
+		push := func(packet *rtp.Packet) error {
+			if packet.Version != h264.RTPPacketVersionAVC {
+				return nil
+			}
+
+			if !c.start {
+				if h265.IsKeyframe(packet.Payload) {
+					c.start = true
+				} else {
+					return nil
+				}
+			}
+
+			buf := c.muxer.Marshal(packet)
+			c.send += len(buf)
+			c.Fire(buf)
+
+			return nil
+		}
+
+		if !h264.IsAVC(codec) {
+			wrapper := h265.RTPDepay(track)
+			push = wrapper(push)
+		}
+
 		return track.Bind(push)
 	}
 
-	panic("unsupported codec")
+	fmt.Printf("[rtmp] unsupported codec: %+v\n", track.Codec)
+
+	return nil
 }
 
-func (c *Consumer) Init() {
-	c.muxer = mp4f.NewMuxer(nil)
-	if err := c.muxer.WriteHeader(c.streams); err != nil {
-		return
+func (c *Consumer) MimeType() string {
+	return c.muxer.MimeType(c.codecs)
+}
+
+func (c *Consumer) Init() ([]byte, error) {
+	if c.muxer == nil {
+		c.muxer = &Muxer{}
 	}
-
-	codecs, buf := c.muxer.GetInit(c.streams)
-	c.Fire(&streamer.Message{Type: MsgTypeMSE, Value: codecs})
-
-	c.send += len(buf)
-	c.Fire(buf)
+	return c.muxer.GetInit(c.codecs)
 }
 
 //
 
 func (c *Consumer) MarshalJSON() ([]byte, error) {
 	v := map[string]interface{}{
-		"type":        "MSE server consumer",
+		"type":        "MP4 server consumer",
 		"send":        c.send,
 		"remote_addr": c.RemoteAddr,
 		"user_agent":  c.UserAgent,

pkg/mp4/helpers.go

@@ -1,47 +0,0 @@
-package mp4
-
-import (
-	"errors"
-	"io"
-)
-
-type MemoryWriter struct {
-	buf []byte
-	pos int
-}
-
-func (m *MemoryWriter) Write(p []byte) (n int, err error) {
-	minCap := m.pos + len(p)
-	if minCap > cap(m.buf) { // Make sure buf has enough capacity:
-		buf2 := make([]byte, len(m.buf), minCap+len(p)) // add some extra
-		copy(buf2, m.buf)
-		m.buf = buf2
-	}
-	if minCap > len(m.buf) {
-		m.buf = m.buf[:minCap]
-	}
-	copy(m.buf[m.pos:], p)
-	m.pos += len(p)
-	return len(p), nil
-}
-
-func (m *MemoryWriter) Seek(offset int64, whence int) (int64, error) {
-	newPos, offs := 0, int(offset)
-	switch whence {
-	case io.SeekStart:
-		newPos = offs
-	case io.SeekCurrent:
-		newPos = m.pos + offs
-	case io.SeekEnd:
-		newPos = len(m.buf) + offs
-	}
-	if newPos < 0 {
-		return 0, errors.New("negative result pos")
-	}
-	m.pos = newPos
-	return int64(newPos), nil
-}
-
-func (m *MemoryWriter) Bytes() []byte {
-	return m.buf
-}

pkg/mp4/muxer.go

@@ -1,37 +1,209 @@
 package mp4
 
 import (
-	"github.com/deepch/vdk/av"
+	"encoding/binary"
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/deepch/vdk/codec/h264parser"
-	"github.com/deepch/vdk/format/mp4"
-	"time"
+	"github.com/deepch/vdk/codec/h265parser"
+	"github.com/deepch/vdk/format/fmp4/fmp4io"
+	"github.com/deepch/vdk/format/mp4/mp4io"
+	"github.com/deepch/vdk/format/mp4f/mp4fio"
+	"github.com/pion/rtp"
 )
 
-func MarshalMP4(sps, pps, frame []byte) []byte {
-	writer := &MemoryWriter{}
-	muxer := mp4.NewMuxer(writer)
-
-	stream, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
-	if err != nil {
-		panic(err)
-	}
-
-	if err = muxer.WriteHeader([]av.CodecData{stream}); err != nil {
-		panic(err)
-	}
-
-	pkt := av.Packet{
-		CompositionTime: time.Millisecond,
-		IsKeyFrame:      true,
-		Duration:        time.Second,
-		Data:            frame,
-	}
-	if err = muxer.WritePacket(pkt); err != nil {
-		panic(err)
-	}
-	if err = muxer.WriteTrailer(); err != nil {
-		panic(err)
-	}
-
-	return writer.buf
+type Muxer struct {
+	fragIndex uint32
+	dts       uint64
+	pts       uint32
+	data      []byte
+	total     int
+}
+
+func (m *Muxer) MimeType(codecs []*streamer.Codec) string {
+	s := `video/mp4; codecs="`
+
+	for _, codec := range codecs {
+		switch codec.Name {
+		case streamer.CodecH264:
+			s += "avc1." + h264.GetProfileLevelID(codec.FmtpLine)
+		case streamer.CodecH265:
+			// +Safari +Chrome +Edge -iOS15 -Android13
+			s += "hvc1.1.6.L93.B0" // hev1.1.6.L93.B0
+		}
+	}
+
+	return s + `"`
+}
+
+func (m *Muxer) GetInit(codecs []*streamer.Codec) ([]byte, error) {
+	moov := MOOV()
+
+	for _, codec := range codecs {
+		switch codec.Name {
+		case streamer.CodecH264:
+			sps, pps := h264.GetParameterSet(codec.FmtpLine)
+			if sps == nil {
+				return nil, fmt.Errorf("empty SPS: %#v", codec)
+			}
+
+			// TODO: remove
+			codecData, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
+			if err != nil {
+				return nil, err
+			}
+
+			width := codecData.Width()
+			height := codecData.Height()
+
+			trak := TRAK()
+			trak.Media.Header.TimeScale = int32(codec.ClockRate)
+			trak.Header.TrackWidth = float64(width)
+			trak.Header.TrackHeight = float64(height)
+
+			trak.Media.Info.Video = &mp4io.VideoMediaInfo{
+				Flags: 0x000001,
+			}
+			trak.Media.Info.Sample.SampleDesc.AVC1Desc = &mp4io.AVC1Desc{
+				DataRefIdx:           1,
+				HorizontalResolution: 72,
+				VorizontalResolution: 72,
+				Width:                int16(width),
+				Height:               int16(height),
+				FrameCount:           1,
+				Depth:                24,
+				ColorTableId:         -1,
+				Conf: &mp4io.AVC1Conf{
+					Data: codecData.AVCDecoderConfRecordBytes(),
+				},
+			}
+
+			trak.Media.Handler = &mp4io.HandlerRefer{
+				SubType: [4]byte{'v', 'i', 'd', 'e'},
+				Name:    []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'm', 'a', 'i', 'n', 0},
+			}
+
+			moov.Tracks = append(moov.Tracks, trak)
+
+		case streamer.CodecH265:
+			vps, sps, pps := h265.GetParameterSet(codec.FmtpLine)
+			if sps == nil {
+				return nil, fmt.Errorf("empty SPS: %#v", codec)
+			}
+
+			codecData, err := h265parser.NewCodecDataFromVPSAndSPSAndPPS(vps, sps, pps)
+			if err != nil {
+				return nil, err
+			}
+
+			width := codecData.Width()
+			height := codecData.Height()
+
+			trak := TRAK()
+			trak.Media.Header.TimeScale = int32(codec.ClockRate)
+			trak.Header.TrackWidth = float64(width)
+			trak.Header.TrackHeight = float64(height)
+
+			trak.Media.Info.Video = &mp4io.VideoMediaInfo{
+				Flags: 0x000001,
+			}
+			trak.Media.Info.Sample.SampleDesc.HV1Desc = &mp4io.HV1Desc{
+				DataRefIdx:           1,
+				HorizontalResolution: 72,
+				VorizontalResolution: 72,
+				Width:                int16(width),
+				Height:               int16(height),
+				FrameCount:           1,
+				Depth:                24,
+				ColorTableId:         -1,
+				Conf: &mp4io.HV1Conf{
+					Data: codecData.AVCDecoderConfRecordBytes(),
+				},
+			}
+
+			trak.Media.Handler = &mp4io.HandlerRefer{
+				SubType: [4]byte{'v', 'i', 'd', 'e'},
+				Name:    []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'm', 'a', 'i', 'n', 0},
+			}
+
+			moov.Tracks = append(moov.Tracks, trak)
+		}
+	}
+
+	data := make([]byte, moov.Len())
+	moov.Marshal(data)
+
+	return append(FTYP(), data...), nil
+}
+
+func (m *Muxer) Rewind() {
+	m.dts = 0
+	m.pts = 0
+}
+
+func (m *Muxer) Marshal(packet *rtp.Packet) []byte {
+	trackID := uint8(1)
+
+	run := &mp4fio.TrackFragRun{
+		Flags:            0x000b05,
+		FirstSampleFlags: uint32(fmp4io.SampleNoDependencies),
+		DataOffset:       0,
+		Entries:          []mp4io.TrackFragRunEntry{},
+	}
+
+	moof := &mp4fio.MovieFrag{
+		Header: &mp4fio.MovieFragHeader{
+			Seqnum: m.fragIndex + 1,
+		},
+		Tracks: []*mp4fio.TrackFrag{
+			{
+				Header: &mp4fio.TrackFragHeader{
+					Data: []byte{0x00, 0x02, 0x00, 0x20, 0x00, 0x00, 0x00, trackID, 0x01, 0x01, 0x00, 0x00},
+				},
+				DecodeTime: &mp4fio.TrackFragDecodeTime{
+					Version: 1,
+					Flags:   0,
+					Time:    m.dts,
+				},
+				Run: run,
+			},
+		},
+	}
+
+	entry := mp4io.TrackFragRunEntry{
+		//Duration: 90000,
+		Size: uint32(len(packet.Payload)),
+	}
+
+	newTime := packet.Timestamp
+	if m.pts > 0 {
+		//m.dts += uint64(newTime - m.pts)
+		entry.Duration = newTime - m.pts
+		m.dts += uint64(entry.Duration)
+	}
+	m.pts = newTime
+
+	// important before moof.Len()
+	run.Entries = append(run.Entries, entry)
+
+	moofLen := moof.Len()
+	mdatLen := 8 + len(packet.Payload)
+
+	// important after moof.Len()
+	run.DataOffset = uint32(moofLen + 8)
+
+	buf := make([]byte, moofLen+mdatLen)
+	moof.Marshal(buf)
+
+	binary.BigEndian.PutUint32(buf[moofLen:], uint32(mdatLen))
+	copy(buf[moofLen+4:], "mdat")
+	copy(buf[moofLen+8:], packet.Payload)
+
+	m.fragIndex++
+
+	m.total += moofLen + mdatLen
+
+	return buf
 }

pkg/rtmp/client.go

@@ -2,7 +2,6 @@ package rtmp
 
 import (
 	"encoding/base64"
-	"encoding/binary"
 	"encoding/hex"
 	"fmt"
 	"github.com/AlexxIT/go2rtc/pkg/h264"
@@ -134,7 +133,7 @@ func (c *Client) Handle() (err error) {
 
 		var payloads [][]byte
 		if track.Codec.Name == streamer.CodecH264 {
-			payloads = splitAVC(pkt.Data)
+			payloads = h264.SplitAVC(pkt.Data)
 		} else {
 			payloads = [][]byte{pkt.Data}
 		}
@@ -156,21 +155,3 @@ func (c *Client) Close() error {
 	c.closed = true
 	return c.conn.Close()
 }
-
-func splitAVC(data []byte) [][]byte {
-	var nals [][]byte
-	for {
-		// get AVC length
-		size := int(binary.BigEndian.Uint32(data))
-
-		// check if multiple items in one packet
-		if size+4 < len(data) {
-			nals = append(nals, data[:size+4])
-			data = data[size+4:]
-		} else {
-			nals = append(nals, data)
-			break
-		}
-	}
-	return nals
-}

pkg/rtsp/conn.go

@@ -61,10 +61,10 @@ type Conn struct {
 
 	auth     *tcp.Auth
 	conn     net.Conn
+	mode     Mode
 	reader   *bufio.Reader
 	sequence int
-
-	mode Mode
+	uri      string
 
 	tracks   []*streamer.Track
 	channels map[byte]*streamer.Track
@@ -76,24 +76,10 @@ type Conn struct {
 }
 
 func NewClient(uri string) (*Conn, error) {
-	var err error
-
 	c := new(Conn)
-	c.URL, err = url.Parse(uri)
-	if err != nil {
-		return nil, err
-	}
-
-	if strings.IndexByte(c.URL.Host, ':') < 0 {
-		c.URL.Host += ":554"
-	}
-
-	// remove UserInfo from URL
-	c.auth = tcp.NewAuth(c.URL.User)
 	c.mode = ModeClientProducer
-	c.URL.User = nil
-
-	return c, nil
+	c.uri = uri
+	return c, c.parseURI()
 }
 
 func NewServer(conn net.Conn) *Conn {
@@ -104,12 +90,29 @@ func NewServer(conn net.Conn) *Conn {
 	return c
 }
 
+func (c *Conn) parseURI() (err error) {
+	c.URL, err = url.Parse(c.uri)
+	if err != nil {
+		return err
+	}
+
+	if strings.IndexByte(c.URL.Host, ':') < 0 {
+		c.URL.Host += ":554"
+	}
+
+	// remove UserInfo from URL
+	c.auth = tcp.NewAuth(c.URL.User)
+	c.URL.User = nil
+
+	return nil
+}
+
 func (c *Conn) Dial() (err error) {
 	//if c.state != StateClientInit {
 	//	panic("wrong state")
 	//}
-	if c.conn != nil && c.auth != nil {
-		c.auth.Reset()
+	if c.conn != nil {
+		_ = c.parseURI()
 	}
 
 	c.conn, err = net.DialTimeout(
@@ -359,7 +362,21 @@ func (c *Conn) SetupMedia(
 	var res *tcp.Response
 	res, err = c.Do(req)
 	if err != nil {
-		return nil, err
+		// Dahua VTO2111D fail on this step because of backchannel
+		if c.Backchannel {
+			if err = c.Dial(); err != nil {
+				return nil, err
+			}
+			c.Backchannel = false
+			if err = c.Describe(); err != nil {
+				return nil, err
+			}
+			res, err = c.Do(req)
+		}
+
+		if err != nil {
+			return nil, err
+		}
 	}
 
 	if c.Session == "" {

pkg/shell/shell.go

@@ -0,0 +1,41 @@
+package shell
+
+import (
+	"strings"
+)
+
+func QuoteSplit(s string) []string {
+	var a []string
+
+	for len(s) > 0 {
+		is := strings.IndexByte(s, ' ')
+		if is >= 0 {
+			// skip prefix and double spaces
+			if is == 0 {
+				// goto next symbol
+				s = s[1:]
+				continue
+			}
+
+			// check if quote in word
+			if i := strings.IndexByte(s[:is], '"'); i >= 0 {
+				// search quote end
+				if is = strings.Index(s, `" `); is > 0 {
+					is += 1
+				} else {
+					is = -1
+				}
+			}
+		}
+
+		if is >= 0 {
+			a = append(a, strings.ReplaceAll(s[:is], `"`, ""))
+			s = s[is+1:]
+		} else {
+			//add last word
+			a = append(a, s)
+			break
+		}
+	}
+	return a
+}

pkg/srtp/session.go

@@ -90,8 +90,8 @@ func GuessProfile(masterKey []byte) srtp.ProtectionProfile {
 	switch len(masterKey) {
 	case 16:
 		return srtp.ProtectionProfileAes128CmHmacSha1_80
-	case 32:
-		return srtp.ProtectionProfileAes256CmHmacSha1_80
+	//case 32:
+	//	return srtp.ProtectionProfileAes256CmHmacSha1_80
 	}
 	return 0
 }

pkg/streamer/media.go

@@ -5,6 +5,7 @@ import (
 	"github.com/pion/sdp/v3"
 	"strconv"
 	"strings"
+	"unicode"
 )
 
 const (
@@ -24,6 +25,7 @@ const (
 	CodecVP8  = "VP8"
 	CodecVP9  = "VP9"
 	CodecAV1  = "AV1"
+	CodecJPEG = "JPEG" // payloadType: 26
 
 	CodecPCMU = "PCMU" // payloadType: 0
 	CodecPCMA = "PCMA" // payloadType: 8
@@ -35,7 +37,7 @@ const (
 
 func GetKind(name string) string {
 	switch name {
-	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1:
+	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1, CodecJPEG:
 		return KindVideo
 	case CodecPCMU, CodecPCMA, CodecAAC, CodecOpus, CodecG722, CodecMPA:
 		return KindAudio
@@ -128,12 +130,14 @@ type Codec struct {
 func NewCodec(name string) *Codec {
 	name = strings.ToUpper(name)
 	switch name {
-	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1:
+	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1, CodecJPEG:
 		return &Codec{Name: name, ClockRate: 90000}
 	case CodecPCMU, CodecPCMA:
 		return &Codec{Name: name, ClockRate: 8000}
 	case CodecOpus:
 		return &Codec{Name: name, ClockRate: 48000, Channels: 2}
+	case "MJPEG":
+		return &Codec{Name: CodecJPEG, ClockRate: 90000}
 	}
 
 	panic(fmt.Sprintf("unsupported codec: %s", name))
@@ -154,8 +158,8 @@ func (c *Codec) Clone() *Codec {
 
 func (c *Codec) Match(codec *Codec) bool {
 	return c.Name == codec.Name &&
-		c.ClockRate == codec.ClockRate &&
-		c.Channels == codec.Channels
+		(c.ClockRate == codec.ClockRate || codec.ClockRate == 0) &&
+		(c.Channels == codec.Channels || codec.Channels == 0)
 }
 
 func UnmarshalSDP(rawSDP []byte) ([]*Media, error) {
@@ -242,7 +246,8 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 			ss := strings.Split(attr.Value[i+1:], "/")
 
 			c.Name = strings.ToUpper(ss[0])
-			c.ClockRate = uint32(atoi(ss[1]))
+			// fix tailing space: `a=rtpmap:96 H264/90000 `
+			c.ClockRate = uint32(atoi(strings.TrimRightFunc(ss[1], unicode.IsSpace)))
 
 			if len(ss) == 3 && ss[2] == "2" {
 				c.Channels = 2
@@ -255,6 +260,7 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 	}
 
 	if c.Name == "" {
+		// https://en.wikipedia.org/wiki/RTP_payload_formats
 		switch payloadType {
 		case "0":
 			c.Name = CodecPCMU
@@ -265,6 +271,9 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 		case "14":
 			c.Name = CodecMPA
 			c.ClockRate = 44100
+		case "26":
+			c.Name = CodecJPEG
+			c.ClockRate = 90000
 		default:
 			c.Name = payloadType
 		}

pkg/streamer/track.go

@@ -32,6 +32,8 @@ func (t *Track) WriteRTP(p *rtp.Packet) error {
 }
 
 func (t *Track) Bind(w WriterFunc) *Track {
+	t.mx.Lock()
+
 	if t.Sink == nil {
 		t.Sink = map[*Track]WriterFunc{}
 	}
@@ -39,9 +41,10 @@ func (t *Track) Bind(w WriterFunc) *Track {
 	clone := &Track{
 		Codec: t.Codec, Direction: t.Direction, Sink: t.Sink,
 	}
-	t.mx.Lock()
 	t.Sink[clone] = w
+
 	t.mx.Unlock()
+
 	return clone
 }
 

pkg/tcp/auth.go

@@ -80,12 +80,6 @@ func (a *Auth) Write(req *Request) {
 	}
 }
 
-func (a *Auth) Reset() {
-	if a.Method == AuthDigest {
-		a.Method = AuthUnknown
-	}
-}
-
 func Between(s, sub1, sub2 string) string {
 	i := strings.Index(s, sub1)
 	if i < 0 {

pkg/webrtc/api.go

@@ -86,10 +86,6 @@ func RegisterDefaultCodecs(m *webrtc.MediaEngine) error {
 			PayloadType:        98, //123,
 		},
 		// macOS Safari 15.1
-		{
-			RTPCodecCapability: webrtc.RTPCodecCapability{webrtc.MimeTypeH264, 90000, 0, "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f", videoRTCPFeedback},
-			PayloadType:        99,
-		},
 		{
 			RTPCodecCapability: webrtc.RTPCodecCapability{webrtc.MimeTypeH265, 90000, 0, "", videoRTCPFeedback},
 			PayloadType:        100,

pkg/webrtc/conn.go

@@ -59,7 +59,6 @@ func (c *Conn) Init() {
 		}
 
 		fmt.Printf("TODO: webrtc ontrack %+v\n", remote)
-		fmt.Printf("TODO: webrtc ontrack %#v\n", remote)
 	})
 
 	// OK connection:

pkg/webrtc/consumer.go

@@ -3,6 +3,7 @@ package webrtc
 import (
 	"encoding/json"
 	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/pion/rtp"
 	"github.com/pion/webrtc/v3"
@@ -51,7 +52,8 @@ func (c *Conn) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.
 			return trackLocal.WriteRTP(packet)
 		}
 
-		if codec.Name == streamer.CodecH264 {
+		switch codec.Name {
+		case streamer.CodecH264:
 			wrapper := h264.RTPPay(1200)
 			push = wrapper(push)
 
@@ -61,6 +63,15 @@ func (c *Conn) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.
 				wrapper = h264.RTPDepay(track)
 			}
 			push = wrapper(push)
+
+		case streamer.CodecH265:
+			// SafariPay because it is the only browser in the world
+			// that supports WebRTC + H265
+			wrapper := h265.SafariPay(1200)
+			push = wrapper(push)
+
+			wrapper = h265.RTPDepay(track)
+			push = wrapper(push)
 		}
 
 		track = track.Bind(push)

pkg/webrtc/helper.go

@@ -1,12 +1,14 @@
 package webrtc
 
 import (
+	"fmt"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/pion/ice/v2"
 	"github.com/pion/stun"
 	"github.com/pion/webrtc/v3"
 	"net"
 	"strconv"
+	"strings"
 )
 
 func NewCandidate(address string) (string, error) {
@@ -34,6 +36,31 @@ func NewCandidate(address string) (string, error) {
 	return "candidate:" + cand.Marshal(), nil
 }
 
+func LookupIP(address string) (string, error) {
+	if strings.HasPrefix(address, "stun:") {
+		ip, err := GetPublicIP()
+		if err != nil {
+			return "", err
+		}
+		return ip.String() + address[4:], nil
+	}
+
+	if IsIP(address) {
+		return address, nil
+	}
+
+	i := strings.IndexByte(address, ':')
+	ips, err := net.LookupIP(address[:i])
+	if err != nil {
+		return "", err
+	}
+	if len(ips) == 0 {
+		return "", fmt.Errorf("can't resolve: %s", address)
+	}
+
+	return ips[0].String() + address[i:], nil
+}
+
 // GetPublicIP example from https://github.com/pion/stun
 func GetPublicIP() (net.IP, error) {
 	c, err := stun.Dial("udp", "stun.l.google.com:19302")
@@ -63,6 +90,15 @@ func GetPublicIP() (net.IP, error) {
 	return xorAddr.IP, nil
 }
 
+func IsIP(host string) bool {
+	for _, i := range host {
+		if i >= 'A' {
+			return false
+		}
+	}
+	return true
+}
+
 func MimeType(codec *streamer.Codec) string {
 	switch codec.Name {
 	case streamer.CodecH264:

www/README.md

@@ -51,4 +51,5 @@ pc.ontrack = ev => {
 ## Useful links
 
 - https://www.webrtc-experiment.com/DetectRTC/
-- https://divtable.com/table-styler/
+- https://divtable.com/table-styler/
+- https://www.chromium.org/audio-video/

www/codecs.html

@@ -45,7 +45,9 @@
         'video/mp4; codecs="avc1.640032"',
         'video/mp4; codecs="avc1.640C32"',
         'video/mp4; codecs="avc1.F4001F"',
-        'video/mp4; codecs="hvc1.016000"',
+        'video/mp4; codecs="hvc1.1.6.L93.B0"',
+        'video/mp4; codecs="hev1.1.6.L93.B0"',
+        'video/mp4; codecs="hev1.2.4.L120.B0"',
     ];
 
     const video = document.createElement("video");

www/index.html

@@ -66,7 +66,10 @@
     const links = [
         '<a href="webrtc.html?src={name}">webrtc</a>',
         '<a href="mse.html?src={name}">mse</a>',
-        '<a href="api/frame.mp4?src={name}">frame.mp4</a>',
+        // '<a href="video.html?src={name}">video</a>',
+        '<a href="api/stream.mp4?src={name}">mp4</a>',
+        '<a href="api/frame.mp4?src={name}">frame</a>',
+        `<a href="rtsp://${location.hostname}:8554/{name}">rtsp</a>`,
         '<a href="api/streams?src={name}">info</a>',
     ];
 

www/mse.html

@@ -60,9 +60,7 @@
             console.debug("ws.onmessage", data);
 
             if (data.type === "mse") {
-                sourceBuffer = mediaSource.addSourceBuffer(
-                    `video/mp4; codecs="${data.value}"`
-                );
+                sourceBuffer = mediaSource.addSourceBuffer(data.value);
                 // important: segments supports TrackFragDecodeTime
                 // sequence supports only TrackFragRunEntry Duration
                 sourceBuffer.mode = "segments";

www/video.html

@@ -0,0 +1,53 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <title>go2rtc - WebRTC</title>
+    <style>
+        body {
+            margin: 0;
+            padding: 0;
+        }
+
+        html, body {
+            height: 100%;
+            width: 100%;
+        }
+
+        #video {
+            /* video "container" size */
+            width: 100%;
+            height: 100%;
+            background: black;
+        }
+    </style>
+</head>
+<body>
+<video id="video" autoplay controls playsinline muted></video>
+<!--<video id="video" preload="auto" controls playsinline muted></video>-->
+<script>
+    const baseUrl = location.origin + location.pathname.substr(
+        0, location.pathname.lastIndexOf("/")
+    );
+    const video = document.getElementById('video');
+
+    video.oncanplay = ev => console.log(ev.type, ev);
+    video.onplaying = ev => console.log(ev.type, ev);
+    video.onwaiting = ev => console.log(ev.type, ev);
+    video.onseeking = ev => console.log(ev.type, ev);
+    video.onloadeddata = ev => console.log(ev.type, ev);
+    video.oncanplaythrough = ev => console.log(ev.type, ev);
+    // video.ondurationchange = ev => console.log(ev.type, ev);
+    // video.ontimeupdate = ev => console.log(ev.type, ev);
+    video.onplay = ev => console.log(ev.type, ev);
+    video.onpause = ev => console.log(ev.type, ev);
+    video.onsuspended = ev => console.log(ev.type, ev);
+    video.onemptied = ev => console.log(ev.type, ev);
+    video.onstalled = ev => console.log(ev.type, ev);
+
+    console.log("start");
+
+    video.src = baseUrl + "/api/stream.mp4" + location.search;
+</script>
+</body>
+</html>