Add mjpeg link to stream to main page

Add ability for API to set name of stream

README.md

@@ -5,11 +5,10 @@ Ultimate camera streaming application with support RTSP, WebRTC, HomeKit, FFmpeg
 ![](assets/go2rtc.png)
 
 - zero-dependency and zero-config [small app](#go2rtc-binary) for all OS (Windows, macOS, Linux, ARM)
-- zero-delay for all supported protocols (lowest possible streaming latency)
-- streaming from [RTSP](#source-rtsp), [RTMP](#source-rtmp), [MJPEG](#source-ffmpeg), [HLS](#source-ffmpeg), [USB Cameras](#source-ffmpeg-device), [files](#source-ffmpeg) and [other sources](#module-streams)
-- streaming to [RTSP](#module-rtsp), [WebRTC](#module-webrtc) or [MSE](#module-api)
+- zero-delay for many supported protocols (lowest possible streaming latency)
+- streaming from [RTSP](#source-rtsp), [RTMP](#source-rtmp), [MJPEG](#source-ffmpeg), [HLS/HTTP](#source-ffmpeg), [USB Cameras](#source-ffmpeg-device) and [other sources](#module-streams)
+- streaming to [RTSP](#module-rtsp), [WebRTC](#module-webrtc), [MSE/MP4](#module-mp4) or [MJPEG](#module-mjpeg)
 - first project in the World with support streaming from [HomeKit Cameras](#source-homekit)
-- low CPU load for supported codecs
 - on the fly transcoding for unsupported codecs via [FFmpeg](#source-ffmpeg)
 - multi-source 2-way [codecs negotiation](#codecs-negotiation)
    - mixing tracks from different sources to single stream
@@ -98,13 +97,9 @@ Don't forget to fix the rights `chmod +x go2rtc_xxx_xxx` on Linux and Mac.
     - go2rtc > Install > Start
 2. Setup [Integration](#module-hass)
 
-**Optionally:**
-
-- create `go2rtc.yaml` in your Home Assistant [config](https://www.home-assistant.io/docs/configuration) folder
-
 ### go2rtc: Docker
 
-Container [alexxit/go2rtc](https://hub.docker.com/r/alexxit/go2rtc) with support `amd64`, `386`, `arm64`, `arm`. This container same as [Home Assistant Add-on](#go2rtc-home-assistant-add-on), but can be used separately from the Home Assistant. Container has preinstalled [FFmpeg](#source-ffmpeg) and [Ngrok](#module-ngrok) applications.
+Container [alexxit/go2rtc](https://hub.docker.com/r/alexxit/go2rtc) with support `amd64`, `386`, `arm64`, `arm`. This container same as [Home Assistant Add-on](#go2rtc-home-assistant-add-on), but can be used separately from the Home Assistant. Container has preinstalled [FFmpeg](#source-ffmpeg), [Ngrok](#module-ngrok) and [Python](#source-echo).
 
 ```yaml
 services:
@@ -132,8 +127,9 @@ Available modules:
 - [api](#module-api) - HTTP API (important for WebRTC support)
 - [rtsp](#module-rtsp) - RTSP Server (important for FFmpeg support)
 - [webrtc](#module-webrtc) - WebRTC Server
-- [ngrok](#module-ngrok) - Ngrok integration (external access for private network)
+- [mp4](#module-mp4) - MSE, MP4 stream and MP4 shapshot
 - [ffmpeg](#source-ffmpeg) - FFmpeg integration
+- [ngrok](#module-ngrok) - Ngrok integration (external access for private network)
 - [hass](#module-hass) - Home Assistant integration
 - [log](#module-log) - logs config
 
@@ -148,11 +144,11 @@ Available source types:
 - [ffmpeg](#source-ffmpeg) - FFmpeg integration (`MJPEG`, `HLS`, `files` and source types)
 - [ffmpeg:device](#source-ffmpeg-device) - local USB Camera or Webcam
 - [exec](#source-exec) - advanced FFmpeg and GStreamer integration
+- [echo](#source-echo) - get stream link from bash or python
 - [homekit](#source-homekit) - streaming from HomeKit Camera
+- [ivideon](#source-ivideon) - public cameras from [Ivideon](https://tv.ivideon.com/) service
 - [hass](#source-hass) - Home Assistant integration
 
-**PS.** You can use sources like `MJPEG`, `HLS` and others via FFmpeg integration.
-
 #### Source: RTSP
 
 - Support **RTSP and RTSPS** links with multiple video and audio tracks
@@ -254,6 +250,19 @@ streams:
   stream1: exec:ffmpeg -hide_banner -re -stream_loop -1 -i /media/BigBuckBunny.mp4 -c copy -rtsp_transport tcp -f rtsp {output}
 ```
 
+#### Source: Echo
+
+Some sources may have a dynamic link. And you will need to get it using a bash or python script. Your script should echo a link to the source. RTSP, FFmpeg or any of the [supported sources](#module-streams).
+
+**Docker** and **Hass Add-on** users has preinstalled `python3`, `curl`, `jq`.
+
+Check examples in [wiki](https://github.com/AlexxIT/go2rtc/wiki/Source-Echo-examples).
+
+```yaml
+streams:
+  apple_hls: echo:python3 hls.py https://developer.apple.com/streaming/examples/basic-stream-osx-ios5.html
+```
+
 #### Source: HomeKit
 
 **Important:**
@@ -270,6 +279,15 @@ If you see a device but it does not have a pair button - it is paired to some ec
 
 **This source is in active development!** Tested only with [Aqara Camera Hub G3](https://www.aqara.com/eu/product/camera-hub-g3) (both EU and CN versions).
 
+#### Source: Ivideon
+
+Support public cameras from service [Ivideon](https://tv.ivideon.com/).
+
+```yaml
+streams:
+  quailcam: ivideon:100-tu5dkUPct39cTp9oNEN2B6/0
+```
+
 #### Source: Hass
 
 Support import camera links from [Home Assistant](https://www.home-assistant.io/) config files:
@@ -309,13 +327,9 @@ api:
 
 ### Module: RTSP
 
-You can get any stream as RTSP-stream with codecs filter:
+You can get any stream as RTSP-stream: `rtsp://192.168.1.123:8554/{stream_name}`
 
-```
-rtsp://192.168.1.123/{stream_name}?video={codec}&audio={codec1}&audio={codec2}
-```
-
-- you can omit the codecs, so one first video and one first audio will be selected
+- you can omit the codec filters, so one first video and one first audio will be selected
 - you can set `?video=copy` or just `?video`, so only one first video without audio will be selected
 - you can set multiple video or audio, so all of them will be selected
 
@@ -444,25 +458,54 @@ tunnels:
 
 ### Module: Hass
 
-**go2rtc** compatible with Home Assistant [RTSPtoWebRTC](https://www.home-assistant.io/integrations/rtsp_to_webrtc/) integration.
+If you install **go2rtc** as [Hass Add-on](#go2rtc-home-assistant-add-on) - you need to use localhost IP-address. In other cases you need to use IP-address of server with **go2rtc** application.
 
-If you install **go2rtc** as [Hass Add-on](#go2rtc-home-assistant-add-on) - you need to use localhost IP-address, example:
+#### From go2rtc to Hass
 
-- `http://127.0.0.1:1984/` to web interface
-- `rtsp://127.0.0.1:8554/camera1` to RTSP streams
+Add any supported [stream source](#module-streams) as [Generic Camera](https://www.home-assistant.io/integrations/generic/) and view stream with built-in [Stream](https://www.home-assistant.io/integrations/stream/) integration. Technology `HLS`, supported codecs: `H264`, poor latency.
 
-In other cases you need to use IP-address of server with **go2rtc** application.
+1. Add your stream to [go2rtc config](#configuration)
+2. Hass > Settings > Integrations > Add Integration > [Generic Camera](https://my.home-assistant.io/redirect/config_flow_start/?domain=generic) > `rtsp://127.0.0.1:8554/camera1`
 
-1. Add integration with link to go2rtc HTTP API:
-   - Hass > Settings > Integrations > Add Integration > [RTSPtoWebRTC](https://my.home-assistant.io/redirect/config_flow_start/?domain=rtsp_to_webrtc) > `http://127.0.0.1:1984/`
-2. Add generic camera with RTSP link:
-   - Hass > Settings > Integrations > Add Integration > [Generic Camera](https://my.home-assistant.io/redirect/config_flow_start/?domain=generic) > `rtsp://...` or `rtmp://...`
-   - you can use either direct RTSP links to cameras or take RTSP streams from **go2rtc**
-3. Use Picture Entity or Picture Glance lovelace card
-4. Open full screen card - this is should be WebRTC stream
+#### From Hass to go2rtc
+
+View almost any Hass camera using `WebRTC` technology, supported codecs `H264`/`PCMU`/`PCMA`/`OPUS`, best latency.
+
+When the stream starts - the camera `entity_id` will be added to go2rtc "on the fly". You don't need to add cameras manually to [go2rtc config](#configuration). Some cameras (like [Nest](https://www.home-assistant.io/integrations/nest/)) have a dynamic link to the stream, it will be updated each time a stream is started from the Hass interface. 
+
+1. Hass > Settings > Integrations > Add Integration > [RTSPtoWebRTC](https://my.home-assistant.io/redirect/config_flow_start/?domain=rtsp_to_webrtc) > `http://127.0.0.1:1984/`
+2. Use Picture Entity or Picture Glance lovelace card
+
+You can add camera `entity_id` to [go2rtc config](#configuration) if you need transcoding:
+
+```yaml
+streams:
+  "camera.hall": ffmpeg:{input}#video=copy#audio=opus
+```
 
 PS. Default Home Assistant lovelace cards don't support 2-way audio. You can use 2-way audio from [Add-on Web UI](https://my.home-assistant.io/redirect/supervisor_addon/?addon=a889bffc_go2rtc&repository_url=https%3A%2F%2Fgithub.com%2FAlexxIT%2Fhassio-addons). But you need use HTTPS to access the microphone. This is a browser restriction and cannot be avoided.
 
+### Module: MP4
+
+Provides several features:
+
+1. MSE stream (fMP4 over WebSocket)
+2. Camera snapshots in MP4 format (single frame), can be sent to [Telegram](https://www.telegram.org/)
+3. Progressive MP4 stream - bad format for streaming because of high latency, doesn't work in Safari 
+
+### Module: MJPEG
+
+**Important.** For stream as MJPEG format, your source MUST contain the MJPEG codec. If your camera outputs H264/H265 - you SHOULD use transcoding. With this example, your stream will have both H264 and MJPEG codecs:
+
+```yaml
+streams:
+  camera1:
+    - rtsp://rtsp:12345678@192.168.1.123/av_stream/ch0
+    - ffmpeg:rtsp://rtsp:12345678@192.168.1.123/av_stream/ch0#video=mjpeg
+```
+
+Example link to MJPEG: `http://192.168.1.123:1984/api/stream.mjpeg?src=camera1`
+
 ### Module: Log
 
 You can set different log levels for different modules.
@@ -504,6 +547,30 @@ If you need Web interface protection without Home Assistant Add-on - you need to
 
 PS. Additionally WebRTC opens a lot of random UDP ports for transmit encrypted media. They work without problems on the local network. And sometimes work for external access, even if you haven't opened ports on your router. But for stable external WebRTC access, you need to configure the TCP port.
 
+## Codecs madness
+
+`AVC/H.264` codec can be played almost anywhere. But `HEVC/H.265` has a lot of limitations in supporting with different devices and browsers. It's all about patents and money, you can't do anything about it.
+
+Device | WebRTC | MSE | MP4
+-------|--------|-----|----
+*latency*       | best        | medium      | bad
+Desktop Chrome  | H264        | H264, H265* | H264, H265*
+Desktop Safari  | H264, H265* | H264        | no
+Desktop Edge    | H264        | H264, H265* | H264, H265*
+Desktop Firefox | H264        | H264        | H264
+Desktop Opera   | no          | H264        | H264
+iPhone Safari   | H264, H265* | no          | no
+iPad Safari     | H264, H265* | H264        | no
+Android Chrome  | H264        | H264        | H264
+masOS Hass App  | no          | no          | no
+
+- WebRTC audio codecs: `PCMU/8000`, `PCMA/8000`, `OPUS/48000/2`
+- MSE/MP4 audio codecs: not supported yet (should be: `AAC`)
+- Chrome H265: [read this](https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding)
+- Edge H265: [read this](https://www.reddit.com/r/MicrosoftEdge/comments/v9iw8k/enable_hevc_support_in_edge/)
+- Desktop Safari H265: Menu > Develop > Experimental > WebRTC H265
+- iOS Safari H265: Settings > Safari > Advanced > Experimental > WebRTC H265
+
 ## FAQ
 
 **Q. What's the difference between go2rtc, WebRTC Camera and RTSPtoWebRTC?**

build/hassio/Dockerfile

@@ -1,23 +1,40 @@
 ARG BUILD_FROM
-FROM $BUILD_FROM
 
-RUN apk add --no-cache git go ffmpeg
+FROM $BUILD_FROM as build
 
-ARG BUILD_ARCH
+# 1. Build go2rtc
+RUN apk add --no-cache git go
 
 RUN git clone https://github.com/AlexxIT/go2rtc \
     && cd go2rtc \
-    && CGO_ENABLED=0 go build -ldflags "-s -w" -trimpath -o /usr/local/bin
+    && CGO_ENABLED=0 go build -ldflags "-s -w" -trimpath
+
+# 2. Download ngrok
+ARG BUILD_ARCH
 
 # https://github.com/home-assistant/docker-base/blob/master/alpine/Dockerfile
 RUN if [ "${BUILD_ARCH}" = "aarch64" ]; then BUILD_ARCH="arm64"; \
     elif [ "${BUILD_ARCH}" = "armv7" ]; then BUILD_ARCH="arm"; fi \
     && cd go2rtc \
     && curl $(curl -s "https://raw.githubusercontent.com/ngrok/docker-ngrok/main/releases.json" | jq -r ".${BUILD_ARCH}.url") -o ngrok.zip \
-    && unzip ngrok -d /usr/local/bin
+    && unzip ngrok
 
-RUN rm -r /go2rtc
 
+
+# https://devopscube.com/reduce-docker-image-size/
+FROM $BUILD_FROM
+
+# 3. Copy go2rtc and ngrok to release
+COPY --from=build /go2rtc/go2rtc /usr/local/bin
+COPY --from=build /go2rtc/ngrok /usr/local/bin
+
+# 4. Install ffmpeg
+# apk base  OK: 22  MiB in 40  packages
+# ffmpeg    OK: 113 MiB in 110 packages
+# python3   OK: 161 MiB in 114 packages
+RUN apk add --no-cache ffmpeg python3
+
+# 5. Copy run to release
 COPY run.sh /
 RUN chmod a+x /run.sh
 

build/hassio/run.sh

@@ -2,13 +2,13 @@
 
 set +e
 
-# set cwd for go2rtc (for config file, Hass itegration, etc)
+# set cwd for go2rtc (for config file, Hass integration, etc)
 cd /config
 
 # add the feature to override go2rtc binary from Hass config folder
 export PATH="/config:$PATH"
 
 while true; do
-    go2rtc
-    sleep 5
+  go2rtc
+  sleep 5
 done

cmd/api/api.go

@@ -34,11 +34,10 @@ func Init() {
 	log = app.GetLogger("api")
 
 	initStatic(cfg.Mod.StaticDir)
+	initWS()
 
-	HandleFunc("/api/frame.mp4", frameHandler)
-	HandleFunc("/api/frame.raw", frameHandler)
-	HandleFunc("/api/streams", streamsHandler)
-	HandleFunc("/api/ws", apiWS)
+	HandleFunc("api/streams", streamsHandler)
+	HandleFunc("api/ws", apiWS)
 
 	// ensure we can listen without errors
 	listener, err := net.Listen("tcp", cfg.Mod.Listen)
@@ -51,14 +50,29 @@ func Init() {
 
 	go func() {
 		s := http.Server{}
+
+		if log.Trace().Enabled() {
+			s.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+				log.Trace().Stringer("url", r.URL).Msgf("[api] %s", r.Method)
+				http.DefaultServeMux.ServeHTTP(w, r)
+			})
+		}
+
 		if err = s.Serve(listener); err != nil {
 			log.Fatal().Err(err).Msg("[api] serve")
 		}
 	}()
 }
 
+// HandleFunc handle pattern with relative path:
+// - "api/streams" => "{basepath}/api/streams"
+// - "/streams"    => "/streams"
 func HandleFunc(pattern string, handler http.HandlerFunc) {
-	http.HandleFunc(basePath+pattern, handler)
+	if len(pattern) == 0 || pattern[0] != '/' {
+		pattern = basePath + "/" + pattern
+	}
+	log.Trace().Str("path", pattern).Msg("[api] register path")
+	http.HandleFunc(pattern, handler)
 }
 
 func HandleWS(msgType string, handler WSHandler) {
@@ -71,10 +85,15 @@ var wsHandlers = make(map[string]WSHandler)
 
 func streamsHandler(w http.ResponseWriter, r *http.Request) {
 	src := r.URL.Query().Get("src")
+	name := r.URL.Query().Get("name")
+	
+	if name == "" {
+		name = src
+	}
 
 	switch r.Method {
 	case "PUT":
-		streams.Get(src)
+		streams.New(name, src)
 		return
 	case "DELETE":
 		streams.Delete(src)
@@ -87,13 +106,10 @@ func streamsHandler(w http.ResponseWriter, r *http.Request) {
 	} else {
 		v = streams.All()
 	}
-	data, err := json.Marshal(v)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.streams] marshal")
-	}
-	if _, err = w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.streams] write")
-	}
+
+	e := json.NewEncoder(w)
+	e.SetIndent("", "  ")
+	_ = e.Encode(v)
 }
 
 func apiWS(w http.ResponseWriter, r *http.Request) {

cmd/api/keyframe.go

@@ -1,40 +0,0 @@
-package api
-
-import (
-	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/pkg/keyframe"
-	"net/http"
-	"strings"
-)
-
-func frameHandler(w http.ResponseWriter, r *http.Request) {
-	src := r.URL.Query().Get("src")
-	stream := streams.Get(src)
-	if stream == nil {
-		return
-	}
-
-	var ch = make(chan []byte)
-
-	cons := new(keyframe.Consumer)
-	cons.IsMP4 = strings.HasSuffix(r.URL.Path, ".mp4")
-	cons.Listen(func(msg interface{}) {
-		switch msg.(type) {
-		case []byte:
-			ch <- msg.([]byte)
-		}
-	})
-
-	if err := stream.AddConsumer(cons); err != nil {
-		log.Warn().Err(err).Msg("[api.frame] add consumer")
-		return
-	}
-
-	data := <-ch
-
-	stream.RemoveConsumer(cons)
-
-	if _, err := w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.frame] write")
-	}
-}

cmd/api/static.go

@@ -14,13 +14,13 @@ func initStatic(staticDir string) {
 		root = http.FS(www.Static)
 	}
 
+	base := len(basePath)
 	fileServer := http.FileServer(root)
 
-	HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
-		if basePath != "" {
-			r.URL.Path = r.URL.Path[len(basePath):]
+	HandleFunc("", func(w http.ResponseWriter, r *http.Request) {
+		if base > 0 {
+			r.URL.Path = r.URL.Path[base:]
 		}
-
 		fileServer.ServeHTTP(w, r)
 	})
 }

cmd/api/ws.go

@@ -4,16 +4,42 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/gorilla/websocket"
 	"net/http"
+	"net/url"
+	"strings"
 	"sync"
 )
 
-type WSHandler func(ctx *Context, msg *streamer.Message)
-
-var apiWsUp = websocket.Upgrader{
-	ReadBufferSize:  1024,
-	WriteBufferSize: 512000,
+func initWS() {
+	wsUp = &websocket.Upgrader{
+		ReadBufferSize:  1024,
+		WriteBufferSize: 512000,
+	}
+	wsUp.CheckOrigin = func(r *http.Request) bool {
+		origin := r.Header["Origin"]
+		if len(origin) == 0 {
+			return true
+		}
+		o, err := url.Parse(origin[0])
+		if err != nil {
+			return false
+		}
+		if o.Host == r.Host {
+			return true
+		}
+		log.Trace().Msgf("[api.ws] origin: %s, host: %s", o.Host, r.Host)
+		// some users change Nginx external port using Docker port
+		// so origin will be with a port and host without
+		if i := strings.IndexByte(o.Host, ':'); i > 0 {
+			return o.Host[:i] == r.Host
+		}
+		return false
+	}
 }
 
+var wsUp *websocket.Upgrader
+
+type WSHandler func(ctx *Context, msg *streamer.Message)
+
 type Context struct {
 	Conn     *websocket.Conn
 	Request  *http.Request
@@ -24,7 +50,7 @@ type Context struct {
 }
 
 func (ctx *Context) Upgrade(w http.ResponseWriter, r *http.Request) (err error) {
-	ctx.Conn, err = apiWsUp.Upgrade(w, r, nil)
+	ctx.Conn, err = wsUp.Upgrade(w, r, nil)
 	ctx.Request = r
 	return
 }

cmd/debug/debug.go

@@ -10,8 +10,8 @@ import (
 )
 
 func Init() {
-	api.HandleFunc("/api/stack", stackHandler)
-	api.HandleFunc("/api/exit", exitHandler)
+	api.HandleFunc("api/stack", stackHandler)
+	api.HandleFunc("api/exit", exitHandler)
 
 	streams.HandleFunc("null", nullHandler)
 }

cmd/echo/echo.go

@@ -0,0 +1,29 @@
+package echo
+
+import (
+	"bytes"
+	"github.com/AlexxIT/go2rtc/cmd/app"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/shell"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"os/exec"
+)
+
+func Init() {
+	log := app.GetLogger("echo")
+
+	streams.HandleFunc("echo", func(url string) (streamer.Producer, error) {
+		args := shell.QuoteSplit(url[5:])
+
+		b, err := exec.Command(args[0], args[1:]...).Output()
+		if err != nil {
+			return nil, err
+		}
+
+		b = bytes.TrimSpace(b)
+
+		log.Debug().Str("url", url).Msgf("[echo] %s", b)
+
+		return streams.GetProducer(string(b))
+	})
+}

cmd/exec/exec.go

@@ -8,6 +8,7 @@ import (
 	"github.com/AlexxIT/go2rtc/cmd/rtsp"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
 	pkg "github.com/AlexxIT/go2rtc/pkg/rtsp"
+	"github.com/AlexxIT/go2rtc/pkg/shell"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/rs/zerolog"
 	"os"
@@ -49,7 +50,7 @@ func Handle(url string) (streamer.Producer, error) {
 	)
 
 	// remove `exec:`
-	args := QuoteSplit(url[5:])
+	args := shell.QuoteSplit(url[5:])
 	cmd := exec.Command(args[0], args[1:]...)
 
 	if log.Trace().Enabled() {
@@ -64,6 +65,8 @@ func Handle(url string) (streamer.Producer, error) {
 
 	log.Debug().Str("url", url).Msg("[exec] run")
 
+	ts := time.Now()
+
 	if err := cmd.Start(); err != nil {
 		log.Error().Err(err).Str("url", url).Msg("[exec]")
 		return nil, err
@@ -75,6 +78,7 @@ func Handle(url string) (streamer.Producer, error) {
 		log.Error().Str("url", url).Msg("[exec] timeout")
 		return nil, errors.New("timeout")
 	case prod := <-ch:
+		log.Debug().Stringer("launch", time.Since(ts)).Msg("[exec] run")
 		return prod, nil
 	}
 }
@@ -83,39 +87,3 @@ func Handle(url string) (streamer.Producer, error) {
 
 var log zerolog.Logger
 var waiters map[string]chan streamer.Producer
-
-func QuoteSplit(s string) []string {
-	var a []string
-
-	for len(s) > 0 {
-		is := strings.IndexByte(s, ' ')
-		if is >= 0 {
-			// skip prefix and double spaces
-			if is == 0 {
-				// goto next symbol
-				s = s[1:]
-				continue
-			}
-
-			// check if quote in word
-			if i := strings.IndexByte(s[:is], '"'); i >= 0 {
-				// search quote end
-				if is = strings.Index(s, `" `); is > 0 {
-					is += 1
-				} else {
-					is = -1
-				}
-			}
-		}
-
-		if is >= 0 {
-			a = append(a, strings.ReplaceAll(s[:is], `"`, ""))
-			s = s[is+1:]
-		} else {
-			//add last word
-			a = append(a, s)
-			break
-		}
-	}
-	return a
-}

cmd/ffmpeg/README.md

@@ -39,3 +39,5 @@
 - https://html5test.com/
 - https://trac.ffmpeg.org/wiki/Capture/Webcam
 - https://trac.ffmpeg.org/wiki/DirectShow
+- https://stackoverflow.com/questions/53207692/libav-mjpeg-encoding-and-huffman-table
+- https://github.com/tuupola/esp_video/blob/master/README.md

cmd/ffmpeg/device/devices.go

@@ -15,7 +15,7 @@ import (
 func Init() {
 	log = app.GetLogger("exec")
 
-	api.HandleFunc("/api/devices", handle)
+	api.HandleFunc("api/devices", handle)
 }
 
 func GetInput(src string) (string, error) {

cmd/ffmpeg/ffmpeg.go

@@ -26,7 +26,7 @@ func Init() {
 		"rtsp": "-fflags nobuffer -flags low_delay -rtsp_transport tcp -i {input}",
 
 		// output
-		"out": "-rtsp_transport tcp -f rtsp {output}",
+		"output": "-rtsp_transport tcp -f rtsp {output}",
 
 		// `-g 30` - group of picture, GOP, keyframe interval
 		// `-preset superfast` - we can't use ultrafast because it doesn't support `-profile main -level 4.1`
@@ -37,6 +37,7 @@ func Init() {
 		"h264/ultra": "-codec:v libx264 -g 30 -preset ultrafast -tune zerolatency",
 		"h264/high":  "-codec:v libx264 -g 30 -preset superfast -tune zerolatency",
 		"h265":       "-codec:v libx265 -g 30 -preset ultrafast -tune zerolatency",
+		"mjpeg":      "-codec:v mjpeg -force_duplicated_matrix 1 -huffman 0 -pix_fmt yuvj420p",
 		"opus":       "-codec:a libopus -ar 48000 -ac 2",
 		"pcmu":       "-codec:a pcm_mulaw -ar 8000 -ac 1",
 		"pcmu/16000": "-codec:a pcm_mulaw -ar 16000 -ac 1",
@@ -55,18 +56,36 @@ func Init() {
 		s = s[7:] // remove `ffmpeg:`
 
 		var query url.Values
+		var queryVideo, queryAudio bool
 		if i := strings.IndexByte(s, '#'); i > 0 {
 			query = parseQuery(s[i+1:])
+			queryVideo = query["video"] != nil
+			queryAudio = query["audio"] != nil
 			s = s[:i]
+		} else {
+			// by default query both video and audio
+			queryVideo = true
+			queryAudio = true
 		}
 
 		var input string
 		if i := strings.IndexByte(s, ':'); i > 0 {
 			switch s[:i] {
-			case "http", "https":
+			case "http", "https", "rtmp":
 				input = strings.Replace(tpl["http"], "{input}", s, 1)
 			case "rtsp", "rtsps":
-				input = strings.Replace(tpl["rtsp"], "{input}", s, 1)
+				// https://ffmpeg.org/ffmpeg-protocols.html#rtsp
+				// skip unnecessary input tracks
+				switch {
+				case queryVideo && queryAudio:
+					input = "-allowed_media_types video+audio "
+				case queryVideo:
+					input = "-allowed_media_types video "
+				case queryAudio:
+					input = "-allowed_media_types audio "
+				}
+
+				input += strings.Replace(tpl["rtsp"], "{input}", s, 1)
 			}
 		}
 
@@ -102,23 +121,23 @@ func Init() {
 
 			for _, audio := range query["audio"] {
 				if audio == "copy" {
-					s += " -codec:v copy"
+					s += " -codec:a copy"
 				} else {
 					s += " " + tpl[audio]
 				}
 			}
 
-			if query["video"] == nil {
-				s += " -vn"
-			}
-			if query["audio"] == nil {
+			switch {
+			case queryVideo && !queryAudio:
 				s += " -an"
+			case queryAudio && !queryVideo:
+				s += " -vn"
 			}
 		} else {
 			s += " -c copy"
 		}
 
-		s += " " + tpl["out"]
+		s += " " + tpl["output"]
 
 		return exec.Handle(s)
 	})

cmd/hass/api.go

@@ -0,0 +1,149 @@
+package hass
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/cmd/webrtc"
+	"net/http"
+	"net/url"
+	"strings"
+)
+
+func initAPI() {
+	ok := func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		_, _ = w.Write([]byte(`{"status":1,"payload":{}}`))
+	}
+
+	// support https://www.home-assistant.io/integrations/rtsp_to_webrtc/
+	api.HandleFunc("/static", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	api.HandleFunc("/streams", ok)
+
+	api.HandleFunc("/stream/", func(w http.ResponseWriter, r *http.Request) {
+		switch {
+		// /stream/{id}/add
+		case strings.HasSuffix(r.RequestURI, "/add"):
+			var v addJSON
+			if err := json.NewDecoder(r.Body).Decode(&v); err != nil {
+				return
+			}
+
+			// we can get three types of links:
+			// 1. link to go2rtc stream: rtsp://...:8554/{stream_name}
+			// 2. static link to Hass camera
+			// 3. dynamic link to Hass camera
+			stream := streams.Get(v.Name)
+			if stream == nil {
+				// check if it is rtsp link to go2rtc
+				stream = rtspStream(v.Channels.First.Url)
+				if stream != nil {
+					streams.New(v.Name, stream)
+				} else {
+					stream = streams.New(v.Name, "{input}")
+				}
+			}
+
+			stream.SetSource(v.Channels.First.Url)
+
+			ok(w, r)
+
+		// /stream/{id}/channel/0/webrtc
+		default:
+			i := strings.IndexByte(r.RequestURI[8:], '/')
+			name := r.RequestURI[8 : 8+i]
+
+			stream := streams.Get(name)
+			if stream == nil {
+				w.WriteHeader(http.StatusNotFound)
+				return
+			}
+
+			if err := r.ParseForm(); err != nil {
+				log.Error().Err(err).Msg("[api.hass] parse form")
+				return
+			}
+
+			s := r.FormValue("data")
+			offer, err := base64.StdEncoding.DecodeString(s)
+			if err != nil {
+				log.Error().Err(err).Msg("[api.hass] sdp64 decode")
+				return
+			}
+
+			s, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
+			if err != nil {
+				log.Error().Err(err).Msg("[api.hass] exchange SDP")
+				return
+			}
+
+			s = base64.StdEncoding.EncodeToString([]byte(s))
+			_, _ = w.Write([]byte(s))
+		}
+	})
+
+	// api from RTSPtoWebRTC
+	api.HandleFunc("/stream", func(w http.ResponseWriter, r *http.Request) {
+		if err := r.ParseForm(); err != nil {
+			return
+		}
+
+		str := r.FormValue("sdp64")
+		offer, err := base64.StdEncoding.DecodeString(str)
+		if err != nil {
+			return
+		}
+
+		src := r.FormValue("url")
+		src, err = url.QueryUnescape(src)
+		if err != nil {
+			return
+		}
+
+		stream := streams.Get(src)
+		if stream == nil {
+			if stream = rtspStream(src); stream != nil {
+				streams.New(src, stream)
+			} else {
+				stream = streams.New(src, src)
+			}
+		}
+
+		str, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
+		if err != nil {
+			return
+		}
+
+		v := struct {
+			Answer string `json:"sdp64"`
+		}{
+			Answer: base64.StdEncoding.EncodeToString([]byte(str)),
+		}
+
+		w.Header().Set("Content-Type", "application/json")
+		_ = json.NewEncoder(w).Encode(v)
+	})
+}
+
+func rtspStream(url string) *streams.Stream {
+	if strings.HasPrefix(url, "rtsp://") {
+		if i := strings.IndexByte(url[7:], '/'); i > 0 {
+			return streams.Get(url[8+i:])
+		}
+	}
+	return nil
+}
+
+type addJSON struct {
+	Name     string `json:"name"`
+	Channels struct {
+		First struct {
+			//Name string `json:"name"`
+			Url string `json:"url"`
+		} `json:"0"`
+	} `json:"channels"`
+}

cmd/hass/hass.go

@@ -1,20 +1,14 @@
 package hass
 
 import (
-	"encoding/base64"
 	"encoding/json"
 	"fmt"
-	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/app"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/cmd/webrtc"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/rs/zerolog"
-	"net/http"
-	"net/url"
 	"os"
 	"path"
-	"strings"
 )
 
 func Init() {
@@ -26,13 +20,9 @@ func Init() {
 
 	app.LoadConfig(&conf)
 
-	log = app.GetLogger("api")
+	log = app.GetLogger("hass")
 
-	// support https://www.home-assistant.io/integrations/rtsp_to_webrtc/
-	api.HandleFunc("/static", func(w http.ResponseWriter, r *http.Request) {
-		w.WriteHeader(http.StatusOK)
-	})
-	api.HandleFunc("/stream", handler)
+	initAPI()
 
 	// support load cameras from Hass config file
 	filename := path.Join(conf.Mod.Config, ".storage/core.config_entries")
@@ -78,72 +68,13 @@ func Init() {
 			continue
 		}
 
+		log.Info().Str("url", "hass:"+entrie.Title).Msg("[hass] load stream")
 		//streams.Get("hass:" + entrie.Title)
 	}
 }
 
 var log zerolog.Logger
 
-func handler(w http.ResponseWriter, r *http.Request) {
-	if err := r.ParseForm(); err != nil {
-		log.Error().Err(err).Msg("[api.hass] parse form")
-		return
-	}
-
-	src := r.FormValue("url")
-	src, err := url.QueryUnescape(src)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] query unescape")
-		return
-	}
-
-	str := r.FormValue("sdp64")
-
-	offer, err := base64.StdEncoding.DecodeString(str)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] sdp64 decode")
-		return
-	}
-
-	// check if stream links to our rtsp server
-	if strings.HasPrefix(src, "rtsp://") {
-		i := strings.IndexByte(src[7:], '/')
-		if i > 0 && streams.Has(src[8+i:]) {
-			src = src[8+i:]
-		}
-	}
-
-	stream := streams.Get(src)
-	if stream == nil {
-		log.Error().Str("url", src).Msg("[api.hass] unsupported source")
-		return
-	}
-
-	str, err = webrtc.ExchangeSDP(stream, string(offer), r.UserAgent())
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] exchange SDP")
-		return
-	}
-
-	resp := struct {
-		Answer string `json:"sdp64"`
-	}{
-		Answer: base64.StdEncoding.EncodeToString([]byte(str)),
-	}
-
-	data, err := json.Marshal(resp)
-	if err != nil {
-		log.Error().Err(err).Msg("[api.hass] marshal JSON")
-		return
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	if _, err = w.Write(data); err != nil {
-		log.Error().Err(err).Msg("[api.hass] write")
-		return
-	}
-}
-
 type entries struct {
 	Data struct {
 		Entries []struct {

cmd/homekit/api.go

@@ -54,15 +54,7 @@ func apiHandler(w http.ResponseWriter, r *http.Request) {
 			items = append(items, device)
 		}
 
-		data, err := json.Marshal(items)
-		if err != nil {
-			log.Error().Err(err).Msg("[api.homekit]")
-			return
-		}
-
-		if _, err = w.Write(data); err != nil {
-			log.Error().Err(err).Msg("[api.homekit]")
-		}
+		_= json.NewEncoder(w).Encode(items)
 
 	case "POST":
 		// TODO: post params...

cmd/homekit/homekit.go

@@ -14,7 +14,7 @@ func Init() {
 
 	streams.HandleFunc("homekit", streamHandler)
 
-	api.HandleFunc("/api/homekit", apiHandler)
+	api.HandleFunc("api/homekit", apiHandler)
 }
 
 var log zerolog.Logger

cmd/ivideon/ivideon.go

@@ -0,0 +1,19 @@
+package ivideon
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/ivideon"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"strings"
+)
+
+func Init() {
+	streams.HandleFunc("ivideon", func(url string) (streamer.Producer, error) {
+		id := strings.Replace(url[8:], "/", ":", 1)
+		prod := ivideon.NewClient(id)
+		if err := prod.Dial(); err != nil {
+			return nil, err
+		}
+		return prod, nil
+	})
+}

cmd/mjpeg/mjpeg.go

@@ -0,0 +1,54 @@
+package mjpeg
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/mjpeg"
+	"github.com/rs/zerolog/log"
+	"net/http"
+	"strconv"
+)
+
+func Init() {
+	api.HandleFunc("api/stream.mjpeg", handler)
+}
+
+const header = "--frame\r\nContent-Type: image/jpeg\r\nContent-Length: "
+
+func handler(w http.ResponseWriter, r *http.Request) {
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan struct{})
+
+	cons := &mjpeg.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			data := []byte(header + strconv.Itoa(len(msg)))
+			data = append(data, 0x0D, 0x0A, 0x0D, 0x0A)
+			data = append(data, msg...)
+			data = append(data, 0x0D, 0x0A)
+
+			if _, err := w.Write(data); err != nil {
+				exit <- struct{}{}
+			}
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.mjpeg] add consumer")
+		return
+	}
+
+	w.Header().Set("Content-Type", `multipart/x-mixed-replace; boundary=frame`)
+
+	<-exit
+
+	stream.RemoveConsumer(cons)
+
+	//log.Trace().Msg("[api.mjpeg] close")
+}

cmd/mp4/mp4.go

@@ -0,0 +1,138 @@
+package mp4
+
+import (
+	"github.com/AlexxIT/go2rtc/cmd/api"
+	"github.com/AlexxIT/go2rtc/cmd/app"
+	"github.com/AlexxIT/go2rtc/cmd/streams"
+	"github.com/AlexxIT/go2rtc/pkg/mp4"
+	"github.com/rs/zerolog"
+	"net/http"
+	"strconv"
+	"strings"
+)
+
+func Init() {
+	log = app.GetLogger("mp4")
+
+	api.HandleWS(MsgTypeMSE, handlerWS)
+
+	api.HandleFunc("api/frame.mp4", handlerKeyframe)
+	api.HandleFunc("api/stream.mp4", handlerMP4)
+}
+
+var log zerolog.Logger
+
+func handlerKeyframe(w http.ResponseWriter, r *http.Request) {
+	if isChromeFirst(w, r) {
+		return
+	}
+
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan []byte)
+
+	cons := &mp4.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			exit <- msg
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] add consumer")
+		return
+	}
+
+	defer stream.RemoveConsumer(cons)
+
+	w.Header().Set("Content-Type", cons.MimeType())
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] init")
+		return
+	}
+	data = append(data, <-exit...)
+
+	// Apple Safari won't show frame without length
+	w.Header().Set("Content-Length", strconv.Itoa(len(data)))
+
+	if _, err := w.Write(data); err != nil {
+		log.Error().Err(err).Msg("[api.keyframe] add consumer")
+	}
+}
+
+func handlerMP4(w http.ResponseWriter, r *http.Request) {
+	if isChromeFirst(w, r) || isSafari(w, r) {
+		return
+	}
+
+	log.Trace().Msgf("[api.mp4] %+v", r)
+
+	src := r.URL.Query().Get("src")
+	stream := streams.GetOrNew(src)
+	if stream == nil {
+		return
+	}
+
+	exit := make(chan struct{})
+
+	cons := &mp4.Consumer{}
+	cons.Listen(func(msg interface{}) {
+		switch msg := msg.(type) {
+		case []byte:
+			if _, err := w.Write(msg); err != nil {
+				exit <- struct{}{}
+			}
+		}
+	})
+
+	if err := stream.AddConsumer(cons); err != nil {
+		log.Error().Err(err).Msg("[api.mp4] add consumer")
+		return
+	}
+
+	defer stream.RemoveConsumer(cons)
+
+	w.Header().Set("Content-Type", cons.MimeType())
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Error().Err(err).Msg("[api.mp4] init")
+		return
+	}
+
+	if _, err = w.Write(data); err != nil {
+		log.Error().Err(err).Msg("[api.mp4] write")
+		return
+	}
+
+	<-exit
+
+	log.Trace().Msg("[api.mp4] close")
+}
+
+func isChromeFirst(w http.ResponseWriter, r *http.Request) bool {
+	// Chrome 105 does two requests: without Range and with `Range: bytes=0-`
+	if strings.Contains(r.UserAgent(), " Chrome/") {
+		if r.Header.Values("Range") == nil {
+			w.Header().Set("Content-Type", "video/mp4")
+			w.WriteHeader(http.StatusOK)
+			return true
+		}
+	}
+	return false
+}
+
+func isSafari(w http.ResponseWriter, r *http.Request) bool {
+	if r.Header.Get("Range") == "bytes=0-1" {
+		handlerKeyframe(w, r)
+		return true
+	}
+	return false
+}

cmd/mp4/mse.go

@@ -1,35 +1,34 @@
-package mse
+package mp4
 
 import (
 	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/streams"
-	"github.com/AlexxIT/go2rtc/pkg/mse"
+	"github.com/AlexxIT/go2rtc/pkg/mp4"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/rs/zerolog/log"
 )
 
-func Init() {
-	api.HandleWS("mse", handler)
-}
+const MsgTypeMSE = "mse" // fMP4
 
-func handler(ctx *api.Context, msg *streamer.Message) {
+func handlerWS(ctx *api.Context, msg *streamer.Message) {
 	src := ctx.Request.URL.Query().Get("src")
-	stream := streams.Get(src)
+	stream := streams.GetOrNew(src)
 	if stream == nil {
 		return
 	}
 
-	cons := new(mse.Consumer)
+	cons := &mp4.Consumer{}
 	cons.UserAgent = ctx.Request.UserAgent()
 	cons.RemoteAddr = ctx.Request.RemoteAddr
+
 	cons.Listen(func(msg interface{}) {
 		switch msg.(type) {
 		case *streamer.Message, []byte:
 			ctx.Write(msg)
 		}
 	})
+
 	if err := stream.AddConsumer(cons); err != nil {
-		log.Warn().Err(err).Msg("[api.mse] Add consumer")
+		log.Warn().Err(err).Msg("[api.mse] add consumer")
 		ctx.Error(err)
 		return
 	}
@@ -38,5 +37,16 @@ func handler(ctx *api.Context, msg *streamer.Message) {
 		stream.RemoveConsumer(cons)
 	})
 
-	cons.Init()
+	ctx.Write(&streamer.Message{
+		Type: MsgTypeMSE, Value: cons.MimeType(),
+	})
+
+	data, err := cons.Init()
+	if err != nil {
+		log.Warn().Err(err).Msg("[api.mse] init")
+		ctx.Error(err)
+		return
+	}
+
+	ctx.Write(data)
 }

cmd/rtsp/rtsp.go

@@ -8,6 +8,7 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/tcp"
 	"github.com/rs/zerolog"
 	"net"
+	"strings"
 )
 
 func Init() {
@@ -47,6 +48,15 @@ var OnProducer func(conn streamer.Producer) bool // TODO: maybe rewrite...
 var log zerolog.Logger
 
 func rtspHandler(url string) (streamer.Producer, error) {
+	backchannel := true
+
+	if i := strings.IndexByte(url, '#'); i > 0 {
+		if url[i+1:] == "backchannel=0" {
+			backchannel = false
+		}
+		url = url[:i]
+	}
+
 	conn, err := rtsp.NewClient(url)
 	if err != nil {
 		return nil, err
@@ -67,8 +77,12 @@ func rtspHandler(url string) (streamer.Producer, error) {
 		return nil, err
 	}
 
-	conn.Backchannel = true
+	conn.Backchannel = backchannel
 	if err = conn.Describe(); err != nil {
+		if !backchannel {
+			return nil, err
+		}
+
 		// second try without backchannel, we need to reconnect
 		if err = conn.Dial(); err != nil {
 			return nil, err
@@ -143,15 +157,15 @@ func worker(address string) {
 
 					log.Debug().Str("stream", name).Msg("[rtsp] new producer")
 
-					str := streams.Get(conn.URL.Path[1:])
-					if str == nil {
+					stream := streams.Get(name)
+					if stream == nil {
 						return
 					}
 
-					str.AddProducer(conn)
+					stream.AddProducer(conn)
 
 					onDisconnect = func() {
-						str.RemoveProducer(conn)
+						stream.RemoveProducer(conn)
 					}
 
 				case streamer.StatePlaying:

cmd/streams/producer.go

@@ -2,6 +2,7 @@ package streams
 
 import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"strings"
 	"sync"
 )
 
@@ -17,7 +18,9 @@ const (
 type Producer struct {
 	streamer.Element
 
-	url     string
+	url      string
+	template string
+
 	element streamer.Producer
 	tracks  []*streamer.Track
 
@@ -25,6 +28,13 @@ type Producer struct {
 	mx    sync.Mutex
 }
 
+func (p *Producer) SetSource(s string) {
+	if p.template == "" {
+		p.template = p.url
+	}
+	p.url = strings.Replace(p.template, "{input}", s, 1)
+}
+
 func (p *Producer) GetMedias() []*streamer.Media {
 	p.mx.Lock()
 	defer p.mx.Unlock()
@@ -79,7 +89,11 @@ func (p *Producer) start() {
 	log.Debug().Str("url", p.url).Msg("[streams] start producer")
 
 	p.state = stateStart
-	go p.element.Start()
+	go func() {
+		if err := p.element.Start(); err != nil {
+			log.Warn().Err(err).Str("url", p.url).Msg("[streams] start")
+		}
+	}()
 }
 
 func (p *Producer) stop() {

cmd/streams/stream.go

@@ -17,25 +17,34 @@ type Stream struct {
 }
 
 func NewStream(source interface{}) *Stream {
-	s := new(Stream)
-
 	switch source := source.(type) {
 	case string:
+		s := new(Stream)
 		prod := &Producer{url: source}
 		s.producers = append(s.producers, prod)
+		return s
 	case []interface{}:
+		s := new(Stream)
 		for _, source := range source {
 			prod := &Producer{url: source.(string)}
 			s.producers = append(s.producers, prod)
 		}
+		return s
+	case *Stream:
+		return source
 	case map[string]interface{}:
 		return NewStream(source["url"])
 	case nil:
+		return new(Stream)
 	default:
 		panic("wrong source type")
 	}
+}
 
-	return s
+func (s *Stream) SetSource(source string) {
+	for _, prod := range s.producers {
+		prod.SetSource(source)
+	}
 }
 
 func (s *Stream) AddConsumer(cons streamer.Consumer) (err error) {
@@ -94,6 +103,11 @@ func (s *Stream) AddConsumer(cons streamer.Consumer) (err error) {
 
 func (s *Stream) RemoveConsumer(cons streamer.Consumer) {
 	for i, consumer := range s.consumers {
+		if consumer == nil {
+			log.Warn().Msgf("empty consumer: %+v\n", s)
+			continue
+		}
+
 		if consumer.element == cons {
 			// remove consumer pads from all producers
 			for _, track := range consumer.tracks {
@@ -106,6 +120,11 @@ func (s *Stream) RemoveConsumer(cons streamer.Consumer) {
 	}
 
 	for _, producer := range s.producers {
+		if producer == nil {
+			log.Warn().Msgf("empty producer: %+v\n", s)
+			continue
+		}
+
 		var sink bool
 		for _, track := range producer.tracks {
 			if len(track.Sink) > 0 {

cmd/streams/streams.go

@@ -24,10 +24,19 @@ func Init() {
 	}
 }
 
-func Get(src string) *Stream {
+func Get(name string) *Stream {
+	return streams[name]
+}
+
+func New(name string, source interface{}) *Stream {
+	stream := NewStream(source)
+	streams[name] = stream
+	return stream
+}
+
+func GetOrNew(src string) *Stream {
 	if stream, ok := streams[src]; ok {
 		return stream
-
 	}
 
 	if !HasProducer(src) {
@@ -35,17 +44,8 @@ func Get(src string) *Stream {
 	}
 
 	log.Info().Str("url", src).Msg("[streams] create new stream")
-	stream := NewStream(src)
-	streams[src] = stream
-	return stream
-}
 
-func Has(src string) bool {
-	return streams[src] != nil
-}
-
-func New(name string, source interface{}) {
-	streams[name] = NewStream(source)
+	return New(src, src)
 }
 
 func Delete(name string) {

cmd/webrtc/candidates.go

@@ -5,7 +5,6 @@ import (
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/AlexxIT/go2rtc/pkg/webrtc"
 	"github.com/pion/sdp/v3"
-	"strings"
 )
 
 var candidates []string
@@ -32,15 +31,11 @@ func addCanditates(answer string) (string, error) {
 	}
 
 	for _, address := range candidates {
-		if strings.HasPrefix(address, "stun:") {
-			ip, err := webrtc.GetPublicIP()
-			if err != nil {
-				log.Warn().Err(err).Msg("[webrtc] public IP")
-				continue
-			}
-			address = ip.String() + address[4:]
-
-			log.Debug().Str("addr", address).Msg("[webrtc] stun public address")
+		var err error
+		address, err = webrtc.LookupIP(address)
+		if err != nil {
+			log.Warn().Err(err).Msg("[webrtc] candidate")
+			continue
 		}
 
 		cand, err := webrtc.NewCandidate(address)

cmd/webrtc/webrtc.go

@@ -69,7 +69,7 @@ func offerHandler(ctx *api.Context, msg *streamer.Message) {
 		return
 	}
 
-	log.Debug().Str("src", src).Msg("[webrtc] new consumer")
+	log.Debug().Str("url", src).Msg("[webrtc] new consumer")
 
 	var err error
 

go.mod

@@ -54,8 +54,4 @@ replace (
 	github.com/brutella/dnssd v1.2.2 => github.com/rblenkinsopp/dnssd v1.2.3-0.20220516082132-0923f3c787a1
 	// RTP tlv8 fix
 	github.com/brutella/hap v0.0.17 => github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657
-	// MSE update
-	github.com/deepch/vdk v0.0.19 => github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e
-	// AES_256_CM_HMAC_SHA1_80 support
-	github.com/pion/srtp/v2 v2.0.10 => github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10
 )

go.sum

@@ -1,9 +1,5 @@
 github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657 h1:FUzXAJfm6sRLJ8T6vfzvy/Hm3aioX8+fbxgx2VZoI78=
 github.com/AlexxIT/hap v0.0.15-0.20220823033740-ce7d1564e657/go.mod h1:c2vEL5pzjRWEx07sa32kTVjzI9bBVlstrwBwKe3DlJ0=
-github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10 h1:4aKRthhmkYcStKuk1hcyvkeNJ/BDx5BTIvYmDO9ZJvg=
-github.com/AlexxIT/srtp/v2 v2.0.10-0.20220608200505-3191d4f19c10/go.mod h1:5TtM9yw6lsH0ppNCehB/EjEUli7VkUgKSPJqWVqbhQ4=
-github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e h1:NAgHHZB+JUN3/J4/yq1q1EAc8xwJ8bb/Qp0AcjkfzAA=
-github.com/AlexxIT/vdk v0.0.18-0.20220616041030-b0d122807b2e/go.mod h1:KqQ/KU3hOc4a62l/jPRH5Hiz5fhTq5cGCl8IqeCxWQI=
 github.com/brutella/dnssd v1.2.3 h1:4fBLjZjPH7SbcHhEcIJhZcC9nOhIDZ0m3rn9bjl1/i0=
 github.com/brutella/dnssd v1.2.3/go.mod h1:JoW2sJUrmVIef25G6lrLj7HS6Xdwh6q8WUIvMkkBYXs=
 github.com/cheekybits/genny v1.0.0/go.mod h1:+tQajlRqAUrPI7DOSpB0XAqZYtQakVtB7wXkRAgjxjQ=
@@ -11,6 +7,8 @@ github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534/go.mod h1
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/deepch/vdk v0.0.19 h1:r6xYyBTtXEIEh+csO0XHT00sI7xLF+hQFkJE9/go5II=
+github.com/deepch/vdk v0.0.19/go.mod h1:7ydHfSkflMZxBXfWR79dMjrT54xzvLxnPaByOa9Jpzg=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
 github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
 github.com/go-chi/chi v1.5.4 h1:QHdzF2szwjqVV4wmByUnTcsbIg7UGaQ0tPF2t5GcAIs=
@@ -103,6 +101,8 @@ github.com/pion/sdp/v3 v3.0.5/go.mod h1:iiFWFpQO8Fy3S5ldclBkpXqmWy02ns78NOKoLLL0
 github.com/pion/srtp v1.5.1 h1:9Q3jAfslYZBt+C69SI/ZcONJh9049JUHZWYRRf5KEKw=
 github.com/pion/srtp v1.5.1/go.mod h1:B+QgX5xPeQTNc1CJStJPHzOlHK66ViMDWTT0HZTCkcA=
 github.com/pion/srtp/v2 v2.0.9/go.mod h1:5TtM9yw6lsH0ppNCehB/EjEUli7VkUgKSPJqWVqbhQ4=
+github.com/pion/srtp/v2 v2.0.10 h1:b8ZvEuI+mrL8hbr/f1YiJFB34UMrOac3R3N1yq2UN0w=
+github.com/pion/srtp/v2 v2.0.10/go.mod h1:XEeSWaK9PfuMs7zxXyiN252AHPbH12NX5q/CFDWtUuA=
 github.com/pion/stun v0.3.5 h1:uLUCBCkQby4S1cf6CGuR9QrVOKcvUwFeemaC865QHDg=
 github.com/pion/stun v0.3.5/go.mod h1:gDMim+47EeEtfWogA37n6qXZS88L5V6LqFcf+DZA2UA=
 github.com/pion/transport v0.6.0/go.mod h1:iWZ07doqOosSLMhZ+FXUTq+TamDoXSllxpbGcfkCmbE=
@@ -121,7 +121,7 @@ github.com/pion/udp v0.1.0/go.mod h1:BPELIjbwE9PRbd/zxI/KYBnbo7B6+oA6YuEaNE8lths
 github.com/pion/udp v0.1.1 h1:8UAPvyqmsxK8oOjloDk4wUt63TzFe9WEJkg5lChlj7o=
 github.com/pion/udp v0.1.1/go.mod h1:6AFo+CMdKQm7UiA0eUPA8/eVCTx8jBIITLZHc9DWX5M=
 github.com/pion/webrtc/v2 v2.2.26/go.mod h1:XMZbZRNHyPDe1gzTIHFcQu02283YO45CbiwFgKvXnmc=
-github.com/pion/webrtc/v3 v3.1.41/go.mod h1:sUcW9SFPEWerDqGOBmdYEMfRvbdd7rgwo4bNzfsXww4=
+github.com/pion/webrtc/v3 v3.1.42/go.mod h1:ffD9DulDrPxyWvDPUIPAOSAWx9GUlOExiJPf7cCcMLA=
 github.com/pion/webrtc/v3 v3.1.43 h1:YT3ZTO94UT4kSBvZnRAH82+0jJPUruiKr9CEstdlQzk=
 github.com/pion/webrtc/v3 v3.1.43/go.mod h1:G/J8k0+grVsjC/rjCZ24AKoCCxcFFODgh7zThNZGs0M=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=

main.go

@@ -4,11 +4,14 @@ import (
 	"github.com/AlexxIT/go2rtc/cmd/api"
 	"github.com/AlexxIT/go2rtc/cmd/app"
 	"github.com/AlexxIT/go2rtc/cmd/debug"
+	"github.com/AlexxIT/go2rtc/cmd/echo"
 	"github.com/AlexxIT/go2rtc/cmd/exec"
 	"github.com/AlexxIT/go2rtc/cmd/ffmpeg"
 	"github.com/AlexxIT/go2rtc/cmd/hass"
 	"github.com/AlexxIT/go2rtc/cmd/homekit"
-	"github.com/AlexxIT/go2rtc/cmd/mse"
+	"github.com/AlexxIT/go2rtc/cmd/ivideon"
+	"github.com/AlexxIT/go2rtc/cmd/mjpeg"
+	"github.com/AlexxIT/go2rtc/cmd/mp4"
 	"github.com/AlexxIT/go2rtc/cmd/ngrok"
 	"github.com/AlexxIT/go2rtc/cmd/rtmp"
 	"github.com/AlexxIT/go2rtc/cmd/rtsp"
@@ -24,20 +27,25 @@ func main() {
 	app.Init()     // init config and logs
 	streams.Init() // load streams list
 
+	api.Init() // init HTTP API server
+
+	echo.Init()
+
 	rtsp.Init()   // add support RTSP client and RTSP server
 	rtmp.Init()   // add support RTMP client
 	exec.Init()   // add support exec scheme (depends on RTSP server)
 	ffmpeg.Init() // add support ffmpeg scheme (depends on exec scheme)
 	hass.Init()   // add support hass scheme
 
-	api.Init() // init HTTP API server
-
 	webrtc.Init()
-	mse.Init()
+	mp4.Init()
+	mjpeg.Init()
 
 	srtp.Init()
 	homekit.Init()
 
+	ivideon.Init()
+
 	ngrok.Init()
 	debug.Init()
 

pkg/README.md

@@ -0,0 +1,5 @@
+## Useful links
+
+- https://www.wowza.com/blog/streaming-protocols
+- https://vimeo.com/blog/post/rtmp-stream/
+- https://sanjeev-pandey.medium.com/understanding-the-mpeg-4-moov-atom-pseudo-streaming-in-mp4-93935e1b9e9a

pkg/h264/README.md

@@ -1,3 +1,22 @@
+# H264
+
+Access Unit (AU) can contain one or multiple NAL Unit:
+
+1. [SEI,] SPS, PPS, IFrame, [IFrame...]
+2. BFrame, [BFrame...]
+3. IFrame, [IFrame...]
+
+## RTP H264
+
+Camera | NALu
+-------|-----
+EZVIZ C3S          | 7f, 8f, 28:28:28 -> 5t, 28:28:28 -> 1t, 1t, 1t, 1t
+Sonoff GK-200MP2-B | 28:28:28 -> 5t, 1t, 1t, 1t
+Dahua IPC-K42      | 7f, 8f, 28:28:28 -> 5t, 28:28:28 -> 1t, 28:28:28 -> 1t
+FFmpeg copy        | 5t, 1t, 1t, 28:28:28 -> 1t, 28:28:28 -> 1t
+FFmpeg h264        | 24 -> 6:5:5:5:5t, 24 -> 1:1:1:1t, 28:28:28 -> 5f, 28:28:28 -> 5f, 28:28:28 -> 5t
+FFmpeg resize      | 6f, 28:28:28 -> 5f, 28... -> 5t, 24 -> 1:1f, 24 -> 1:1t
+
 ## WebRTC
 
 Video codec	    | Media string | Device
@@ -25,3 +44,4 @@ H.264/high      | avc1.6400xx  | FFmpeg superfast
 - [AVC levels](https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels)
 - [AVC profiles table](https://developer.mozilla.org/ru/docs/Web/Media/Formats/codecs_parameter)
 - [Supported Media for Google Cast](https://developers.google.com/cast/docs/media)
+- [Two stream formats, Annex-B, AVCC (H.264) and HVCC (H.265)](https://www.programmersought.com/article/3901815022/)

pkg/h264/avc.go

@@ -12,49 +12,71 @@ func IsAVC(codec *streamer.Codec) bool {
 	return codec.PayloadType == PayloadTypeAVC
 }
 
-func EncodeAVC(raw []byte) (avc []byte) {
-	avc = make([]byte, len(raw)+4)
-	binary.BigEndian.PutUint32(avc, uint32(len(raw)))
-	copy(avc[4:], raw)
+func EncodeAVC(nals ...[]byte) (avc []byte) {
+	var i, n int
+
+	for _, nal := range nals {
+		if i = len(nal); i > 0 {
+			n += 4 + i
+		}
+	}
+
+	avc = make([]byte, n)
+
+	n = 0
+	for _, nal := range nals {
+		if i = len(nal); i > 0 {
+			binary.BigEndian.PutUint32(avc[n:], uint32(i))
+			n += 4 + copy(avc[n+4:], nal)
+		}
+	}
+
 	return
 }
 
 func RepairAVC(track *streamer.Track) streamer.WrapperFunc {
 	sps, pps := GetParameterSet(track.Codec.FmtpLine)
-	sps = EncodeAVC(sps)
-	pps = EncodeAVC(pps)
+	ps := EncodeAVC(sps, pps)
 
 	return func(push streamer.WriterFunc) streamer.WriterFunc {
 		return func(packet *rtp.Packet) (err error) {
-			naluType := NALUType(packet.Payload)
-			switch naluType {
-			case NALUTypeSPS:
-				sps = packet.Payload
-				return
-			case NALUTypePPS:
-				pps = packet.Payload
-				return
+			if NALUType(packet.Payload) == NALUTypeIFrame {
+				packet.Payload = Join(ps, packet.Payload)
 			}
-
-			var clone rtp.Packet
-
-			if naluType == NALUTypeIFrame {
-				clone = *packet
-				clone.Payload = sps
-				if err = push(&clone); err != nil {
-					return
-				}
-
-				clone = *packet
-				clone.Payload = pps
-				if err = push(&clone); err != nil {
-					return
-				}
-			}
-
-			clone = *packet
-			clone.Payload = packet.Payload
-			return push(&clone)
+			return push(packet)
 		}
 	}
 }
+
+func SplitAVC(data []byte) [][]byte {
+	var nals [][]byte
+	for {
+		// get AVC length
+		size := int(binary.BigEndian.Uint32(data)) + 4
+
+		// check if multiple items in one packet
+		if size < len(data) {
+			nals = append(nals, data[:size])
+			data = data[size:]
+		} else {
+			nals = append(nals, data)
+			break
+		}
+	}
+	return nals
+}
+
+func Types(data []byte) []byte {
+	var types []byte
+	for {
+		types = append(types, NALUType(data))
+
+		size := 4 + int(binary.BigEndian.Uint32(data))
+		if size < len(data) {
+			data = data[size:]
+		} else {
+			break
+		}
+	}
+	return types
+}

pkg/h264/helper.go

@@ -2,6 +2,7 @@ package h264
 
 import (
 	"encoding/base64"
+	"encoding/binary"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"strings"
 )
@@ -9,6 +10,7 @@ import (
 const (
 	NALUTypePFrame = 1
 	NALUTypeIFrame = 5
+	NALUTypeSEI    = 6
 	NALUTypeSPS    = 7
 	NALUTypePPS    = 8
 )
@@ -17,6 +19,40 @@ func NALUType(b []byte) byte {
 	return b[4] & 0x1F
 }
 
+// IsKeyframe - check if any NALU in one AU is Keyframe
+func IsKeyframe(b []byte) bool {
+	for {
+		switch NALUType(b) {
+		case NALUTypePFrame:
+			return false
+		case NALUTypeIFrame:
+			return true
+		}
+
+		size := int(binary.BigEndian.Uint32(b)) + 4
+		if size < len(b) {
+			b = b[size:]
+			continue
+		} else {
+			return false
+		}
+	}
+}
+
+func Join(ps, iframe []byte) []byte {
+	b := make([]byte, len(ps)+len(iframe))
+	i := copy(b, ps)
+	copy(b[i:], iframe)
+	return b
+}
+
+func GetProfileLevelID(fmtp string) string {
+	if fmtp == "" {
+		return ""
+	}
+	return streamer.Between(fmtp, "profile-level-id=", ";")
+}
+
 func GetParameterSet(fmtp string) (sps, pps []byte) {
 	if fmtp == "" {
 		return

pkg/h264/rtp.go

@@ -13,90 +13,74 @@ func RTPDepay(track *streamer.Track) streamer.WrapperFunc {
 	depack := &codecs.H264Packet{IsAVC: true}
 
 	sps, pps := GetParameterSet(track.Codec.FmtpLine)
-	sps = EncodeAVC(sps)
-	pps = EncodeAVC(pps)
+	ps := EncodeAVC(sps, pps)
 
-	var buffer []byte
+	buf := make([]byte, 0, 512*1024) // 512K
 
 	return func(push streamer.WriterFunc) streamer.WriterFunc {
 		return func(packet *rtp.Packet) error {
-			//nalUnitType := packet.Payload[0] & 0x1F
 			//fmt.Printf(
-			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d\n",
-			//	track.Codec.Name, nalUnitType, len(packet.Payload), packet.Timestamp,
-			//	packet.PayloadType, packet.SSRC,
+			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d, %v\n",
+			//	track.Codec.Name, packet.Payload[0]&0x1F, len(packet.Payload), packet.Timestamp,
+			//	packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker,
 			//)
 
-			// NALu packets can be split in different ways:
-			// - single type 7 and type 8 packets
-			// - join type 7 and type 8 packet (type 24)
-			// - split type 5 on multiple 28 packets
-			// - split type 5 on multiple separate 28 packets
-			units, err := depack.Unmarshal(packet.Payload)
-			if len(units) == 0 || err != nil {
+			payload, err := depack.Unmarshal(packet.Payload)
+			if len(payload) == 0 || err != nil {
 				return nil
 			}
 
-			for {
-				i := int(binary.BigEndian.Uint32(units)) + 4
-				unitAVC := units[:i]
-
-				unitType := NALUType(unitAVC)
-				switch unitType {
-				case NALUTypeSPS:
-					//println("new SPS")
-					sps = unitAVC
-					return nil
-				case NALUTypePPS:
-					//println("new PPS")
-					pps = unitAVC
+			// Fix TP-Link Tapo TC70: sends SPS and PPS with packet.Marker = true
+			if packet.Marker {
+				switch NALUType(payload) {
+				case NALUTypeSPS, NALUTypePPS:
+					buf = append(buf, payload...)
 					return nil
 				}
-
-				// ffmpeg with `-tune zerolatency` enable option `-x264opts sliced-threads=1`
-				// and every NALU will be sliced to multiple NALUs
-				if !packet.Marker {
-					buffer = append(buffer, unitAVC...)
-					return nil
-				}
-
-				if buffer != nil {
-					buffer = append(buffer, unitAVC...)
-					unitAVC = buffer
-					buffer = nil
-				}
-
-				var clone rtp.Packet
-
-				if unitType == NALUTypeIFrame {
-					clone = *packet
-					clone.Version = RTPPacketVersionAVC
-					clone.Payload = sps
-					if err = push(&clone); err != nil {
-						return err
-					}
-
-					clone = *packet
-					clone.Version = RTPPacketVersionAVC
-					clone.Payload = pps
-					if err = push(&clone); err != nil {
-						return err
-					}
-				}
-
-				clone = *packet
-				clone.Version = RTPPacketVersionAVC
-				clone.Payload = unitAVC
-				if err = push(&clone); err != nil {
-					return err
-				}
-
-				if len(units) == i {
-					return nil
-				}
-
-				units = units[i:]
 			}
+
+			if len(buf) == 0 {
+				switch NALUType(payload) {
+				case NALUTypeIFrame:
+					// fix IFrame without SPS,PPS
+					buf = append(buf, ps...)
+				case NALUTypeSEI:
+					// fix ffmpeg with transcoding first frame
+					i := int(4 + binary.BigEndian.Uint32(payload))
+
+					// check if only one NAL (fix ffmpeg transcoding for Reolink RLC-510A)
+					if i == len(payload) {
+						return nil
+					}
+
+					payload = payload[i:]
+
+					if NALUType(payload) == NALUTypeIFrame {
+						buf = append(buf, ps...)
+					}
+				}
+			}
+
+			// collect all NALs for Access Unit
+			if !packet.Marker {
+				buf = append(buf, payload...)
+				return nil
+			}
+
+			if len(buf) > 0 {
+				payload = append(buf, payload...)
+				buf = buf[:0]
+			}
+
+			//fmt.Printf(
+			//	"[AVC] %v, len: %d, %v\n", Types(payload), len(payload),
+			//	reflect.ValueOf(buf).Pointer() == reflect.ValueOf(payload).Pointer(),
+			//)
+
+			clone := *packet
+			clone.Version = RTPPacketVersionAVC
+			clone.Payload = payload
+			return push(&clone)
 		}
 	}
 }
@@ -110,11 +94,12 @@ func RTPPay(mtu uint16) streamer.WrapperFunc {
 		return func(packet *rtp.Packet) error {
 			if packet.Version == RTPPacketVersionAVC {
 				payloads := payloader.Payload(mtu, packet.Payload)
+				last := len(payloads) - 1
 				for i, payload := range payloads {
 					clone := rtp.Packet{
 						Header: rtp.Header{
 							Version: 2,
-							Marker:  i == len(payloads)-1,
+							Marker:  i == last,
 							//PayloadType:    packet.PayloadType,
 							SequenceNumber: sequencer.NextSequenceNumber(),
 							Timestamp:      packet.Timestamp,

pkg/h265/README.md

@@ -0,0 +1,3 @@
+## Useful links
+
+- https://datatracker.ietf.org/doc/html/rfc7798

pkg/h265/helper.go

@@ -0,0 +1,35 @@
+package h265
+
+import (
+	"encoding/base64"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+)
+
+const (
+	NALUnitTypeIFrame = 19
+)
+
+func NALUnitType(b []byte) byte {
+	return b[4] >> 1
+}
+
+func IsKeyframe(b []byte) bool {
+	return NALUnitType(b) == NALUnitTypeIFrame
+}
+
+func GetParameterSet(fmtp string) (vps, sps, pps []byte) {
+	if fmtp == "" {
+		return
+	}
+
+	s := streamer.Between(fmtp, "sprop-vps=", ";")
+	vps, _ = base64.StdEncoding.DecodeString(s)
+
+	s = streamer.Between(fmtp, "sprop-sps=", ";")
+	sps, _ = base64.StdEncoding.DecodeString(s)
+
+	s = streamer.Between(fmtp, "sprop-pps=", ";")
+	pps, _ = base64.StdEncoding.DecodeString(s)
+
+	return
+}

pkg/h265/rtp.go

@@ -0,0 +1,153 @@
+package h265
+
+import (
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/deepch/vdk/codec/h265parser"
+	"github.com/pion/rtp"
+)
+
+func RTPDepay(track *streamer.Track) streamer.WrapperFunc {
+	vps, sps, pps := GetParameterSet(track.Codec.FmtpLine)
+
+	var buffer []byte
+
+	return func(push streamer.WriterFunc) streamer.WriterFunc {
+		return func(packet *rtp.Packet) error {
+			nut := (packet.Payload[0] >> 1) & 0x3f
+			//fmt.Printf(
+			//	"[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d\n",
+			//	track.Codec.Name, nut, len(packet.Payload), packet.Timestamp,
+			//	packet.PayloadType, packet.SSRC, packet.SequenceNumber,
+			//)
+
+			switch nut {
+			case h265parser.NAL_UNIT_UNSPECIFIED_49:
+				data := packet.Payload
+				switch data[2] >> 6 {
+				case 2: // begin
+					buffer = []byte{
+						(data[0] & 0x81) | (data[2] & 0x3f << 1), data[1],
+					}
+					buffer = append(buffer, data[3:]...)
+					return nil
+				case 0: // continue
+					buffer = append(buffer, data[3:]...)
+					return nil
+				case 1: // end
+					packet.Payload = append(buffer, data[3:]...)
+				}
+			case h265parser.NAL_UNIT_VPS:
+				vps = packet.Payload
+				return nil
+			case h265parser.NAL_UNIT_SPS:
+				sps = packet.Payload
+				return nil
+			case h265parser.NAL_UNIT_PPS:
+				pps = packet.Payload
+				return nil
+			default:
+				//panic("not implemented")
+			}
+
+			var clone rtp.Packet
+
+			nut = (packet.Payload[0] >> 1) & 0x3f
+			if nut >= h265parser.NAL_UNIT_CODED_SLICE_BLA_W_LP && nut <= h265parser.NAL_UNIT_CODED_SLICE_CRA {
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(vps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(sps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				clone = *packet
+				clone.Version = h264.RTPPacketVersionAVC
+				clone.Payload = h264.EncodeAVC(pps)
+				if err := push(&clone); err != nil {
+					return err
+				}
+			}
+
+			clone = *packet
+			clone.Version = h264.RTPPacketVersionAVC
+			clone.Payload = h264.EncodeAVC(packet.Payload)
+
+			return push(&clone)
+		}
+	}
+}
+
+// SafariPay - generate Safari friendly payload for H265
+func SafariPay(mtu uint16) streamer.WrapperFunc {
+	sequencer := rtp.NewRandomSequencer()
+	size := int(mtu - 12) // rtp.Header size
+
+	var buffer []byte
+
+	return func(push streamer.WriterFunc) streamer.WriterFunc {
+		return func(packet *rtp.Packet) error {
+			if packet.Version != h264.RTPPacketVersionAVC {
+				return push(packet)
+			}
+
+			data := packet.Payload
+			data[0] = 0
+			data[1] = 0
+			data[2] = 0
+			data[3] = 1
+
+			var start byte
+
+			nut := (data[4] >> 1) & 0b111111
+			//fmt.Printf("[H265] nut: %2d, size: %6d, data: %16x\n", nut, len(data), data[4:20])
+			switch {
+			case nut >= h265parser.NAL_UNIT_VPS && nut <= h265parser.NAL_UNIT_PPS:
+				buffer = append(buffer, data...)
+				return nil
+			case nut >= h265parser.NAL_UNIT_CODED_SLICE_BLA_W_LP && nut <= h265parser.NAL_UNIT_CODED_SLICE_CRA:
+				buffer = append([]byte{3}, buffer...)
+				data = append(buffer, data...)
+				start = 1
+			default:
+				data = append([]byte{2}, data...)
+				start = 0
+			}
+
+			for len(data) > size {
+				clone := rtp.Packet{
+					Header: rtp.Header{
+						Version:        2,
+						Marker:         false,
+						SequenceNumber: sequencer.NextSequenceNumber(),
+						Timestamp:      packet.Timestamp,
+					},
+					Payload: data[:size],
+				}
+				if err := push(&clone); err != nil {
+					return err
+				}
+
+				data = append([]byte{start}, data[size:]...)
+			}
+
+			clone := rtp.Packet{
+				Header: rtp.Header{
+					Version:        2,
+					Marker:         true,
+					SequenceNumber: sequencer.NextSequenceNumber(),
+					Timestamp:      packet.Timestamp,
+				},
+				Payload: data,
+			}
+			return push(&clone)
+		}
+	}
+}

pkg/ivideon/client.go

@@ -0,0 +1,284 @@
+package ivideon
+
+import (
+	"bytes"
+	"encoding/base64"
+	"encoding/binary"
+	"encoding/json"
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/deepch/vdk/codec/h264parser"
+	"github.com/deepch/vdk/format/fmp4/fmp4io"
+	"github.com/gorilla/websocket"
+	"github.com/pion/rtp"
+	"io"
+	"net/http"
+	"strings"
+	"time"
+)
+
+type Client struct {
+	streamer.Element
+
+	ID string
+
+	conn   *websocket.Conn
+	medias []*streamer.Media
+	tracks map[byte]*streamer.Track
+
+	closed bool
+
+	msg *message
+	t0  time.Time
+
+	buffer chan []byte
+}
+
+func NewClient(id string) *Client {
+	return &Client{ID: id}
+}
+
+func (c *Client) Dial() (err error) {
+	resp, err := http.Get(
+		"https://openapi-alpha.ivideon.com/cameras/" + c.ID +
+			"/live_stream?op=GET&access_token=public&q=2&" +
+			"video_codecs=h264&format=ws-fmp4",
+	)
+
+	data, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return err
+	}
+
+	var v liveResponse
+	if err = json.Unmarshal(data, &v); err != nil {
+		return err
+	}
+
+	if !v.Success {
+		return fmt.Errorf("wrong response: %s", data)
+	}
+
+	c.conn, _, err = websocket.DefaultDialer.Dial(v.Result.URL, nil)
+	if err != nil {
+		return err
+	}
+
+	if err = c.getTracks(); err != nil {
+		_ = c.conn.Close()
+		return err
+	}
+
+	return nil
+}
+
+func (c *Client) Handle() error {
+	c.buffer = make(chan []byte, 5)
+	// add delay to the stream for smooth playing (not a best solution)
+	c.t0 = time.Now().Add(time.Second)
+
+	// processing stream in separate thread for lower delay between packets
+	go c.worker()
+
+	_, data, err := c.conn.ReadMessage()
+	if err != nil {
+		return err
+	}
+
+	track := c.tracks[c.msg.Track]
+	if track != nil {
+		c.buffer <- data
+	}
+
+	// we have one unprocessed msg after getTracks
+	for {
+		_, data, err = c.conn.ReadMessage()
+		if err != nil {
+			return err
+		}
+
+		var msg message
+		if err = json.Unmarshal(data, &msg); err != nil {
+			return err
+		}
+
+		switch msg.Type {
+		case "stream-init":
+			continue
+
+		case "fragment":
+			_, data, err = c.conn.ReadMessage()
+			if err != nil {
+				return err
+			}
+
+			track = c.tracks[msg.Track]
+			if track != nil {
+				c.buffer <- data
+			}
+
+		default:
+			return fmt.Errorf("wrong message type: %s", data)
+		}
+	}
+}
+
+func (c *Client) Close() error {
+	if c.conn == nil {
+		return nil
+	}
+	close(c.buffer)
+	c.closed = true
+	return c.conn.Close()
+}
+
+func (c *Client) getTracks() error {
+	c.tracks = map[byte]*streamer.Track{}
+
+	for {
+		_, data, err := c.conn.ReadMessage()
+		if err != nil {
+			return err
+		}
+
+		var msg message
+		if err = json.Unmarshal(data, &msg); err != nil {
+			return err
+		}
+
+		switch msg.Type {
+		case "stream-init":
+			s := msg.CodecString
+			i := strings.IndexByte(s, '.')
+			if i > 0 {
+				s = s[:i]
+			}
+
+			switch s {
+			case "avc1": // avc1.4d0029
+				// skip multiple identical init
+				if c.tracks[msg.TrackID] != nil {
+					continue
+				}
+
+				codec := streamer.NewCodec(streamer.CodecH264)
+				codec.FmtpLine = "profile-level-id=" + msg.CodecString[i+1:]
+				codec.PayloadType = h264.PayloadTypeAVC
+
+				i = bytes.Index(msg.Data, []byte("avcC")) - 4
+				if i < 0 {
+					return fmt.Errorf("wrong AVC: %s", msg.Data)
+				}
+
+				avccLen := binary.BigEndian.Uint32(msg.Data[i:])
+				data = msg.Data[i+8 : i+int(avccLen)]
+
+				record := h264parser.AVCDecoderConfRecord{}
+				if _, err = record.Unmarshal(data); err != nil {
+					return err
+				}
+
+				codec.FmtpLine += ";sprop-parameter-sets=" +
+					base64.StdEncoding.EncodeToString(record.SPS[0]) + "," +
+					base64.StdEncoding.EncodeToString(record.PPS[0])
+
+				media := &streamer.Media{
+					Kind:      streamer.KindVideo,
+					Direction: streamer.DirectionSendonly,
+					Codecs:    []*streamer.Codec{codec},
+				}
+				c.medias = append(c.medias, media)
+
+				track := &streamer.Track{
+					Direction: streamer.DirectionSendonly,
+					Codec:     codec,
+				}
+				c.tracks[msg.TrackID] = track
+
+			case "mp4a": // mp4a.40.2
+			}
+
+		case "fragment":
+			c.msg = &msg
+			return nil
+
+		default:
+			return fmt.Errorf("wrong message type: %s", data)
+		}
+	}
+}
+
+func (c *Client) worker() {
+	var track *streamer.Track
+	for _, track = range c.tracks {
+		break
+	}
+
+	for data := range c.buffer {
+		moof := &fmp4io.MovieFrag{}
+		if _, err := moof.Unmarshal(data, 0); err != nil {
+			continue
+		}
+
+		moofLen := binary.BigEndian.Uint32(data)
+		_ = moofLen
+
+		mdat := moof.Unknowns[0]
+		if mdat.Tag() != fmp4io.MDAT {
+			continue
+		}
+		i, _ := mdat.Pos() // offset, size
+		data = data[i+8:]
+
+		traf := moof.Tracks[0]
+		ts := uint32(traf.DecodeTime.Time)
+
+		//println("!!!", (time.Duration(ts) * time.Millisecond).String(), time.Since(c.t0).String())
+
+		for _, entry := range traf.Run.Entries {
+			// synchronize framerate for WebRTC and MSE
+			d := time.Duration(ts)*time.Millisecond - time.Since(c.t0)
+			if d < 0 {
+				d = time.Duration(entry.Duration) * time.Millisecond / 2
+			}
+			time.Sleep(d)
+
+			// can be SPS, PPS and IFrame in one packet
+			packet := &rtp.Packet{
+				// ivideon clockrate=1000, RTP clockrate=90000
+				Header:  rtp.Header{Timestamp: ts * 90},
+				Payload: data[:entry.Size],
+			}
+			_ = track.WriteRTP(packet)
+
+			data = data[entry.Size:]
+			ts += entry.Duration
+		}
+
+		if len(data) != 0 {
+			continue
+		}
+	}
+}
+
+type liveResponse struct {
+	Result struct {
+		URL string `json:"url"`
+	} `json:"result"`
+	Success bool `json:"success"`
+}
+
+type message struct {
+	Type string `json:"type"`
+
+	CodecString string `json:"codec_string"`
+	Data        []byte `json:"data"`
+	TrackID     byte   `json:"track_id"`
+
+	Track      byte    `json:"track"`
+	StartTime  float32 `json:"start_time"`
+	Duration   float32 `json:"duration"`
+	IsKey      bool    `json:"is_key"`
+	DataOffset uint32  `json:"data_offset"`
+}

pkg/ivideon/streamer.go

@@ -0,0 +1,31 @@
+package ivideon
+
+import (
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+)
+
+func (c *Client) GetMedias() []*streamer.Media {
+	return c.medias
+}
+
+func (c *Client) GetTrack(media *streamer.Media, codec *streamer.Codec) *streamer.Track {
+	for _, track := range c.tracks {
+		if track.Codec == codec {
+			return track
+		}
+	}
+	panic(fmt.Sprintf("wrong media/codec: %+v %+v", media, codec))
+}
+
+func (c *Client) Start() error {
+	err := c.Handle()
+	if c.closed {
+		return nil
+	}
+	return err
+}
+
+func (c *Client) Stop() error {
+	return c.Close()
+}

pkg/keyframe/consumer.go

@@ -1,72 +0,0 @@
-package keyframe
-
-import (
-	"github.com/AlexxIT/go2rtc/pkg/h264"
-	"github.com/AlexxIT/go2rtc/pkg/mp4"
-	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/pion/rtp"
-)
-
-var annexB = []byte{0, 0, 0, 1}
-
-type Consumer struct {
-	streamer.Element
-	IsMP4 bool
-}
-
-func (k *Consumer) GetMedias() []*streamer.Media {
-	// support keyframe extraction only for one coded...
-	codec := streamer.NewCodec(streamer.CodecH264)
-	return []*streamer.Media{
-		{
-			Kind: streamer.KindVideo, Direction: streamer.DirectionRecvonly,
-			Codecs: []*streamer.Codec{codec},
-		},
-	}
-}
-
-func (k *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
-	// sps and pps without AVC headers
-	sps, pps := h264.GetParameterSet(track.Codec.FmtpLine)
-
-	push := func(packet *rtp.Packet) error {
-		// TODO: remove it, unnecessary
-		if packet.Version != h264.RTPPacketVersionAVC {
-			panic("wrong packet type")
-		}
-
-		switch h264.NALUType(packet.Payload) {
-		case h264.NALUTypeSPS:
-			sps = packet.Payload[4:] // remove AVC header
-		case h264.NALUTypePPS:
-			pps = packet.Payload[4:] // remove AVC header
-		case h264.NALUTypeIFrame:
-			if sps == nil || pps == nil {
-				return nil
-			}
-
-			var data []byte
-
-			if k.IsMP4 {
-				data = mp4.MarshalMP4(sps, pps, packet.Payload)
-			} else {
-				data = append(data, annexB...)
-				data = append(data, sps...)
-				data = append(data, annexB...)
-				data = append(data, pps...)
-				data = append(data, annexB...)
-				data = append(data, packet.Payload[4:]...)
-			}
-
-			k.Fire(data)
-		}
-		return nil
-	}
-
-	if !h264.IsAVC(track.Codec) {
-		wrapper := h264.RTPDepay(track)
-		push = wrapper(push)
-	}
-
-	return track.Bind(push)
-}

pkg/mjpeg/README.md

@@ -0,0 +1,4 @@
+## Useful links
+
+- https://www.rfc-editor.org/rfc/rfc2435
+- https://github.com/GStreamer/gst-plugins-good/blob/master/gst/rtp/gstrtpjpegdepay.c

pkg/mjpeg/consumer.go

@@ -0,0 +1,95 @@
+package mjpeg
+
+import (
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/pion/rtp"
+)
+
+type Consumer struct {
+	streamer.Element
+
+	UserAgent  string
+	RemoteAddr string
+
+	codecs []*streamer.Codec
+	start  bool
+
+	send int
+}
+
+func (c *Consumer) GetMedias() []*streamer.Media {
+	return []*streamer.Media{{
+		Kind:      streamer.KindVideo,
+		Direction: streamer.DirectionRecvonly,
+		Codecs:    []*streamer.Codec{{Name: streamer.CodecJPEG}},
+	}}
+}
+
+func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
+	var header, payload []byte
+
+	push := func(packet *rtp.Packet) error {
+		//fmt.Printf(
+		//	"[RTP] codec: %s, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d, mark: %v\n",
+		//	track.Codec.Name, len(packet.Payload), packet.Timestamp,
+		//	packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker,
+		//)
+
+		// https://www.rfc-editor.org/rfc/rfc2435#section-3.1
+		b := packet.Payload
+
+		// 3.1.  JPEG header
+		t := b[4]
+
+		// 3.1.7.  Restart Marker header
+		if 64 <= t && t <= 127 {
+			b = b[12:] // skip it
+		} else {
+			b = b[8:]
+		}
+
+		if header == nil {
+			var lqt, cqt []byte
+
+			// 3.1.8.  Quantization Table header
+			q := packet.Payload[5]
+			if q >= 128 {
+				lqt = b[4:68]
+				cqt = b[68:132]
+				b = b[132:]
+			} else {
+				lqt, cqt = MakeTables(q)
+			}
+
+			// https://www.rfc-editor.org/rfc/rfc2435#section-3.1.5
+			// The maximum width is 2040 pixels.
+			w := uint16(packet.Payload[6]) << 3
+			h := uint16(packet.Payload[7]) << 3
+
+			// fix 2560x1920 and 2560x1440
+			if w == 512 && (h == 1920 || h == 1440) {
+				w = 2560
+			}
+
+			//fmt.Printf("t: %d, q: %d, w: %d, h: %d\n", t, q, w, h)
+			header = MakeHeaders(t, w, h, lqt, cqt)
+		}
+
+		// 3.1.9.  JPEG Payload
+		payload = append(payload, b...)
+
+		if packet.Marker {
+			b = append(header, payload...)
+			if end := b[len(b)-2:]; end[0] != 0xFF && end[1] != 0xD9 {
+				b = append(b, 0xFF, 0xD9)
+			}
+			c.Fire(b)
+
+			header = nil
+			payload = nil
+		}
+
+		return nil
+	}
+	return track.Bind(push)
+}

pkg/mjpeg/rfc2435.go

@@ -0,0 +1,182 @@
+package mjpeg
+
+// RFC 2435. Appendix A
+
+var jpeg_luma_quantizer = []byte{
+	16, 11, 10, 16, 24, 40, 51, 61,
+	12, 12, 14, 19, 26, 58, 60, 55,
+	14, 13, 16, 24, 40, 57, 69, 56,
+	14, 17, 22, 29, 51, 87, 80, 62,
+	18, 22, 37, 56, 68, 109, 103, 77,
+	24, 35, 55, 64, 81, 104, 113, 92,
+	49, 64, 78, 87, 103, 121, 120, 101,
+	72, 92, 95, 98, 112, 100, 103, 99,
+}
+var jpeg_chroma_quantizer = []byte{
+	17, 18, 24, 47, 99, 99, 99, 99,
+	18, 21, 26, 66, 99, 99, 99, 99,
+	24, 26, 56, 99, 99, 99, 99, 99,
+	47, 66, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+	99, 99, 99, 99, 99, 99, 99, 99,
+}
+
+func MakeTables(q byte) (lqt, cqt []byte) {
+	var factor int
+
+	switch {
+	case q < 1:
+		factor = 1
+	case q > 99:
+		factor = 99
+	default:
+		factor = int(q)
+	}
+
+	if q < 50 {
+		factor = 5000 / factor
+	} else if q > 99 {
+		factor = 200 - factor*2
+	}
+
+	lqt = make([]byte, 64)
+	cqt = make([]byte, 64)
+
+	for i := 0; i < 64; i++ {
+		lq := (int(jpeg_luma_quantizer[i])*factor + 50) / 100
+		cq := (int(jpeg_chroma_quantizer[i])*factor + 50) / 100
+
+		/* Limit the quantizers to 1 <= q <= 255 */
+		switch {
+		case lq < 1:
+			lqt[i] = 1
+		case lq > 255:
+			lqt[i] = 255
+		default:
+			lqt[i] = byte(lq)
+		}
+
+		switch {
+		case cq < 1:
+			cqt[i] = 1
+		case cq > 255:
+			cqt[i] = 255
+		default:
+			cqt[i] = byte(cq)
+		}
+	}
+
+	return
+}
+
+// RFC 2435. Appendix B
+
+var lum_dc_codelens = []byte{
+	0, 1, 5, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0,
+}
+var lum_dc_symbols = []byte{
+	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+}
+var lum_ac_codelens = []byte{
+	0, 2, 1, 3, 3, 2, 4, 3, 5, 5, 4, 4, 0, 0, 1, 0x7d,
+}
+var lum_ac_symbols = []byte{
+	0x01, 0x02, 0x03, 0x00, 0x04, 0x11, 0x05, 0x12,
+	0x21, 0x31, 0x41, 0x06, 0x13, 0x51, 0x61, 0x07,
+	0x22, 0x71, 0x14, 0x32, 0x81, 0x91, 0xa1, 0x08,
+	0x23, 0x42, 0xb1, 0xc1, 0x15, 0x52, 0xd1, 0xf0,
+	0x24, 0x33, 0x62, 0x72, 0x82, 0x09, 0x0a, 0x16,
+	0x17, 0x18, 0x19, 0x1a, 0x25, 0x26, 0x27, 0x28,
+	0x29, 0x2a, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39,
+	0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49,
+	0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59,
+	0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69,
+	0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79,
+	0x7a, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89,
+	0x8a, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98,
+	0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
+	0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6,
+	0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3, 0xc4, 0xc5,
+	0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2, 0xd3, 0xd4,
+	0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xe1, 0xe2,
+	0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea,
+	0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,
+	0xf9, 0xfa,
+}
+var chm_dc_codelens = []byte{
+	0, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
+}
+var chm_dc_symbols = []byte{
+	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+}
+var chm_ac_codelens = []byte{
+	0, 2, 1, 2, 4, 4, 3, 4, 7, 5, 4, 4, 0, 1, 2, 0x77,
+}
+var chm_ac_symbols = []byte{
+	0x00, 0x01, 0x02, 0x03, 0x11, 0x04, 0x05, 0x21,
+	0x31, 0x06, 0x12, 0x41, 0x51, 0x07, 0x61, 0x71,
+	0x13, 0x22, 0x32, 0x81, 0x08, 0x14, 0x42, 0x91,
+	0xa1, 0xb1, 0xc1, 0x09, 0x23, 0x33, 0x52, 0xf0,
+	0x15, 0x62, 0x72, 0xd1, 0x0a, 0x16, 0x24, 0x34,
+	0xe1, 0x25, 0xf1, 0x17, 0x18, 0x19, 0x1a, 0x26,
+	0x27, 0x28, 0x29, 0x2a, 0x35, 0x36, 0x37, 0x38,
+	0x39, 0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,
+	0x49, 0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58,
+	0x59, 0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68,
+	0x69, 0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78,
+	0x79, 0x7a, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
+	0x88, 0x89, 0x8a, 0x92, 0x93, 0x94, 0x95, 0x96,
+	0x97, 0x98, 0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5,
+	0xa6, 0xa7, 0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4,
+	0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3,
+	0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2,
+	0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda,
+	0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9,
+	0xea, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,
+	0xf9, 0xfa,
+}
+
+func MakeHeaders(t byte, w, h uint16, lqt, cqt []byte) []byte {
+	// Appendix A from https://www.rfc-editor.org/rfc/rfc2435
+	p := []byte{0xFF, 0xD8}
+
+	p = MakeQuantHeader(p, lqt, 0)
+	p = MakeQuantHeader(p, cqt, 1)
+
+	if t == 0 {
+		t = 0x21
+	} else {
+		t = 0x22
+	}
+
+	p = append(p,
+		0xFF, 0xC0, 0, 17, 8,
+		byte(h>>8), byte(h&0xFF),
+		byte(w>>8), byte(w&0xFF),
+		3, 0, t, 0, 1, 0x11, 1, 2, 0x11, 1,
+	)
+
+	p = MakeHuffmanHeader(p, lum_dc_codelens, lum_dc_symbols, 0, 0)
+	p = MakeHuffmanHeader(p, lum_ac_codelens, lum_ac_symbols, 0, 1)
+	p = MakeHuffmanHeader(p, chm_dc_codelens, chm_dc_symbols, 1, 0)
+	p = MakeHuffmanHeader(p, chm_ac_codelens, chm_ac_symbols, 1, 1)
+
+	return append(p, 0xFF, 0xDA, 0, 12, 3, 0, 0, 1, 0x11, 2, 0x11, 0, 63, 0)
+}
+
+func MakeQuantHeader(p []byte, qt []byte, tableNo byte) []byte {
+	p = append(p, 0xFF, 0xDB, 0, 67, tableNo)
+	return append(p, qt...)
+}
+
+func MakeHuffmanHeader(p, codelens, symbols []byte, tableNo, tableClass byte) []byte {
+	p = append(p,
+		0xFF, 0xC4, 0,
+		byte(3+len(codelens)+len(symbols)),
+		(tableClass<<4)|tableNo,
+	)
+	p = append(p, codelens...)
+	return append(p, symbols...)
+}

pkg/mp4/README.md

@@ -0,0 +1,23 @@
+## HEVC
+
+Browser     | avc1 | hvc1 | hev1
+------------|------|------|---
+Mac Chrome  | +    | -    | +
+Mac Safari  | +    | +    | -
+iOS 15?     | +    | +    | -
+Mac Firefox | +    | -    | -
+iOS 12      | +    | -    | -
+Android 13  | +    | -    | -
+
+```
+ffmpeg -i input-hev1.mp4 -c:v copy -tag:v hvc1 -c:a copy output-hvc1.mp4
+Stream #0:0(eng): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, progressive), 720x404, 164 kb/s, 29.97 fps,
+Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, progressive), 720x404, 164 kb/s, 29.97 fps,
+```
+
+## Useful links
+
+- https://stackoverflow.com/questions/63468587/what-hevc-codec-tag-to-use-with-fmp4-hvc1-or-hev1
+- https://stackoverflow.com/questions/32152090/encode-h265-to-hvc1-codec
+- https://jellyfin.org/docs/general/clients/codec-support.html
+- https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding

pkg/mp4/const.go

@@ -0,0 +1,94 @@
+package mp4
+
+import (
+	"encoding/binary"
+	"github.com/deepch/vdk/format/mp4/mp4io"
+	"time"
+)
+
+var matrix = [9]int32{0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x40000000}
+var time0 = time.Date(1904, time.January, 1, 0, 0, 0, 0, time.UTC)
+
+func FTYP() []byte {
+	b := make([]byte, 0x18)
+	binary.BigEndian.PutUint32(b, 0x18)
+	copy(b[0x04:], "ftyp")
+	copy(b[0x08:], "iso5")
+	copy(b[0x10:], "iso5")
+	copy(b[0x14:], "avc1")
+	return b
+}
+
+func MOOV() *mp4io.Movie {
+	return &mp4io.Movie{
+		Header: &mp4io.MovieHeader{
+			PreferredRate:     1,
+			PreferredVolume:   1,
+			Matrix:            matrix,
+			NextTrackId:       -1,
+			Duration:          0,
+			TimeScale:         1000,
+			CreateTime:        time0,
+			ModifyTime:        time0,
+			PreviewTime:       time0,
+			PreviewDuration:   time0,
+			PosterTime:        time0,
+			SelectionTime:     time0,
+			SelectionDuration: time0,
+			CurrentTime:       time0,
+		},
+		MovieExtend: &mp4io.MovieExtend{
+			Tracks: []*mp4io.TrackExtend{
+				{
+					TrackId:               1,
+					DefaultSampleDescIdx:  1,
+					DefaultSampleDuration: 40,
+				},
+			},
+		},
+	}
+}
+
+func TRAK() *mp4io.Track {
+	return &mp4io.Track{
+		// trak > tkhd
+		Header: &mp4io.TrackHeader{
+			TrackId:    int32(1), // change me
+			Flags:      0x0007,   // 7 ENABLED IN-MOVIE IN-PREVIEW
+			Duration:   0,        // OK
+			Matrix:     matrix,
+			CreateTime: time0,
+			ModifyTime: time0,
+		},
+		// trak > mdia
+		Media: &mp4io.Media{
+			// trak > mdia > mdhd
+			Header: &mp4io.MediaHeader{
+				TimeScale:  1000,
+				Duration:   0,
+				Language:   0x55C4,
+				CreateTime: time0,
+				ModifyTime: time0,
+			},
+			// trak > mdia > minf
+			Info: &mp4io.MediaInfo{
+				// trak > mdia > minf > dinf
+				Data: &mp4io.DataInfo{
+					Refer: &mp4io.DataRefer{
+						Url: &mp4io.DataReferUrl{
+							Flags: 0x000001, // self reference
+						},
+					},
+				},
+				// trak > mdia > minf > stbl
+				Sample: &mp4io.SampleTable{
+					SampleDesc:    &mp4io.SampleDesc{},
+					TimeToSample:  &mp4io.TimeToSample{},
+					SampleToChunk: &mp4io.SampleToChunk{},
+					SampleSize:    &mp4io.SampleSize{},
+					ChunkOffset:   &mp4io.ChunkOffset{},
+				},
+			},
+		},
+	}
+}

pkg/mp4/consumer.go

@@ -0,0 +1,143 @@
+package mp4
+
+import (
+	"encoding/json"
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
+	"github.com/pion/rtp"
+)
+
+type Consumer struct {
+	streamer.Element
+
+	UserAgent  string
+	RemoteAddr string
+
+	muxer  *Muxer
+	codecs []*streamer.Codec
+	start  bool
+
+	send int
+}
+
+func (c *Consumer) GetMedias() []*streamer.Media {
+	return []*streamer.Media{
+		{
+			Kind:      streamer.KindVideo,
+			Direction: streamer.DirectionRecvonly,
+			Codecs: []*streamer.Codec{
+				{Name: streamer.CodecH264, ClockRate: 90000},
+				{Name: streamer.CodecH265, ClockRate: 90000},
+			},
+		},
+		//{
+		//	Kind:      streamer.KindAudio,
+		//	Direction: streamer.DirectionRecvonly,
+		//	Codecs: []*streamer.Codec{
+		//		{Name: streamer.CodecAAC, ClockRate: 16000},
+		//	},
+		//},
+	}
+}
+
+func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
+	codec := track.Codec
+	switch codec.Name {
+	case streamer.CodecH264:
+		c.codecs = append(c.codecs, track.Codec)
+
+		push := func(packet *rtp.Packet) error {
+			if packet.Version != h264.RTPPacketVersionAVC {
+				return nil
+			}
+
+			if c.muxer == nil {
+				return nil
+			}
+
+			if !c.start {
+				if h264.IsKeyframe(packet.Payload) {
+					c.start = true
+				} else {
+					return nil
+				}
+			}
+
+			buf := c.muxer.Marshal(packet)
+			c.send += len(buf)
+			c.Fire(buf)
+
+			return nil
+		}
+
+		var wrapper streamer.WrapperFunc
+		if h264.IsAVC(codec) {
+			wrapper = h264.RepairAVC(track)
+		} else {
+			wrapper = h264.RTPDepay(track)
+		}
+		push = wrapper(push)
+
+		return track.Bind(push)
+
+	case streamer.CodecH265:
+		c.codecs = append(c.codecs, track.Codec)
+
+		push := func(packet *rtp.Packet) error {
+			if packet.Version != h264.RTPPacketVersionAVC {
+				return nil
+			}
+
+			if !c.start {
+				if h265.IsKeyframe(packet.Payload) {
+					c.start = true
+				} else {
+					return nil
+				}
+			}
+
+			buf := c.muxer.Marshal(packet)
+			c.send += len(buf)
+			c.Fire(buf)
+
+			return nil
+		}
+
+		if !h264.IsAVC(codec) {
+			wrapper := h265.RTPDepay(track)
+			push = wrapper(push)
+		}
+
+		return track.Bind(push)
+	}
+
+	fmt.Printf("[rtmp] unsupported codec: %+v\n", track.Codec)
+
+	return nil
+}
+
+func (c *Consumer) MimeType() string {
+	return c.muxer.MimeType(c.codecs)
+}
+
+func (c *Consumer) Init() ([]byte, error) {
+	if c.muxer == nil {
+		c.muxer = &Muxer{}
+	}
+	return c.muxer.GetInit(c.codecs)
+}
+
+//
+
+func (c *Consumer) MarshalJSON() ([]byte, error) {
+	v := map[string]interface{}{
+		"type":        "MP4 server consumer",
+		"send":        c.send,
+		"remote_addr": c.RemoteAddr,
+		"user_agent":  c.UserAgent,
+	}
+
+	return json.Marshal(v)
+}

pkg/mp4/helpers.go

@@ -1,47 +0,0 @@
-package mp4
-
-import (
-	"errors"
-	"io"
-)
-
-type MemoryWriter struct {
-	buf []byte
-	pos int
-}
-
-func (m *MemoryWriter) Write(p []byte) (n int, err error) {
-	minCap := m.pos + len(p)
-	if minCap > cap(m.buf) { // Make sure buf has enough capacity:
-		buf2 := make([]byte, len(m.buf), minCap+len(p)) // add some extra
-		copy(buf2, m.buf)
-		m.buf = buf2
-	}
-	if minCap > len(m.buf) {
-		m.buf = m.buf[:minCap]
-	}
-	copy(m.buf[m.pos:], p)
-	m.pos += len(p)
-	return len(p), nil
-}
-
-func (m *MemoryWriter) Seek(offset int64, whence int) (int64, error) {
-	newPos, offs := 0, int(offset)
-	switch whence {
-	case io.SeekStart:
-		newPos = offs
-	case io.SeekCurrent:
-		newPos = m.pos + offs
-	case io.SeekEnd:
-		newPos = len(m.buf) + offs
-	}
-	if newPos < 0 {
-		return 0, errors.New("negative result pos")
-	}
-	m.pos = newPos
-	return int64(newPos), nil
-}
-
-func (m *MemoryWriter) Bytes() []byte {
-	return m.buf
-}

pkg/mp4/muxer.go

@@ -1,37 +1,210 @@
 package mp4
 
 import (
-	"github.com/deepch/vdk/av"
+	"encoding/binary"
+	"fmt"
+	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
+	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/deepch/vdk/codec/h264parser"
-	"github.com/deepch/vdk/format/mp4"
-	"time"
+	"github.com/deepch/vdk/codec/h265parser"
+	"github.com/deepch/vdk/format/fmp4/fmp4io"
+	"github.com/deepch/vdk/format/mp4/mp4io"
+	"github.com/deepch/vdk/format/mp4f/mp4fio"
+	"github.com/pion/rtp"
 )
 
-func MarshalMP4(sps, pps, frame []byte) []byte {
-	writer := &MemoryWriter{}
-	muxer := mp4.NewMuxer(writer)
-
-	stream, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
-	if err != nil {
-		panic(err)
-	}
-
-	if err = muxer.WriteHeader([]av.CodecData{stream}); err != nil {
-		panic(err)
-	}
-
-	pkt := av.Packet{
-		CompositionTime: time.Millisecond,
-		IsKeyFrame:      true,
-		Duration:        time.Second,
-		Data:            frame,
-	}
-	if err = muxer.WritePacket(pkt); err != nil {
-		panic(err)
-	}
-	if err = muxer.WriteTrailer(); err != nil {
-		panic(err)
-	}
-
-	return writer.buf
+type Muxer struct {
+	fragIndex uint32
+	dts       uint64
+	pts       uint32
+	data      []byte
+	total     int
+}
+
+func (m *Muxer) MimeType(codecs []*streamer.Codec) string {
+	s := `video/mp4; codecs="`
+
+	for _, codec := range codecs {
+		switch codec.Name {
+		case streamer.CodecH264:
+			s += "avc1." + h264.GetProfileLevelID(codec.FmtpLine)
+		case streamer.CodecH265:
+			// +Safari +Chrome +Edge -iOS15 -Android13
+			s += "hvc1.1.6.L93.B0" // hev1.1.6.L93.B0
+		}
+	}
+
+	return s + `"`
+}
+
+func (m *Muxer) GetInit(codecs []*streamer.Codec) ([]byte, error) {
+	moov := MOOV()
+
+	for _, codec := range codecs {
+		switch codec.Name {
+		case streamer.CodecH264:
+			sps, pps := h264.GetParameterSet(codec.FmtpLine)
+			if sps == nil {
+				// some dummy SPS and PPS not a problem
+				sps = []byte{0x67, 0x42, 0x00, 0x0a, 0xf8, 0x41, 0xa2}
+				pps = []byte{0x68, 0xce, 0x38, 0x80}
+			}
+
+			codecData, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
+			if err != nil {
+				return nil, err
+			}
+
+			width := codecData.Width()
+			height := codecData.Height()
+
+			trak := TRAK()
+			trak.Media.Header.TimeScale = int32(codec.ClockRate)
+			trak.Header.TrackWidth = float64(width)
+			trak.Header.TrackHeight = float64(height)
+
+			trak.Media.Info.Video = &mp4io.VideoMediaInfo{
+				Flags: 0x000001,
+			}
+			trak.Media.Info.Sample.SampleDesc.AVC1Desc = &mp4io.AVC1Desc{
+				DataRefIdx:           1,
+				HorizontalResolution: 72,
+				VorizontalResolution: 72,
+				Width:                int16(width),
+				Height:               int16(height),
+				FrameCount:           1,
+				Depth:                24,
+				ColorTableId:         -1,
+				Conf: &mp4io.AVC1Conf{
+					Data: codecData.AVCDecoderConfRecordBytes(),
+				},
+			}
+
+			trak.Media.Handler = &mp4io.HandlerRefer{
+				SubType: [4]byte{'v', 'i', 'd', 'e'},
+				Name:    []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'm', 'a', 'i', 'n', 0},
+			}
+
+			moov.Tracks = append(moov.Tracks, trak)
+
+		case streamer.CodecH265:
+			vps, sps, pps := h265.GetParameterSet(codec.FmtpLine)
+			if sps == nil {
+				return nil, fmt.Errorf("empty SPS: %#v", codec)
+			}
+
+			codecData, err := h265parser.NewCodecDataFromVPSAndSPSAndPPS(vps, sps, pps)
+			if err != nil {
+				return nil, err
+			}
+
+			width := codecData.Width()
+			height := codecData.Height()
+
+			trak := TRAK()
+			trak.Media.Header.TimeScale = int32(codec.ClockRate)
+			trak.Header.TrackWidth = float64(width)
+			trak.Header.TrackHeight = float64(height)
+
+			trak.Media.Info.Video = &mp4io.VideoMediaInfo{
+				Flags: 0x000001,
+			}
+			trak.Media.Info.Sample.SampleDesc.HV1Desc = &mp4io.HV1Desc{
+				DataRefIdx:           1,
+				HorizontalResolution: 72,
+				VorizontalResolution: 72,
+				Width:                int16(width),
+				Height:               int16(height),
+				FrameCount:           1,
+				Depth:                24,
+				ColorTableId:         -1,
+				Conf: &mp4io.HV1Conf{
+					Data: codecData.AVCDecoderConfRecordBytes(),
+				},
+			}
+
+			trak.Media.Handler = &mp4io.HandlerRefer{
+				SubType: [4]byte{'v', 'i', 'd', 'e'},
+				Name:    []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'm', 'a', 'i', 'n', 0},
+			}
+
+			moov.Tracks = append(moov.Tracks, trak)
+		}
+	}
+
+	data := make([]byte, moov.Len())
+	moov.Marshal(data)
+
+	return append(FTYP(), data...), nil
+}
+
+func (m *Muxer) Rewind() {
+	m.dts = 0
+	m.pts = 0
+}
+
+func (m *Muxer) Marshal(packet *rtp.Packet) []byte {
+	trackID := uint8(1)
+
+	run := &mp4fio.TrackFragRun{
+		Flags:            0x000b05,
+		FirstSampleFlags: uint32(fmp4io.SampleNoDependencies),
+		DataOffset:       0,
+		Entries:          []mp4io.TrackFragRunEntry{},
+	}
+
+	moof := &mp4fio.MovieFrag{
+		Header: &mp4fio.MovieFragHeader{
+			Seqnum: m.fragIndex + 1,
+		},
+		Tracks: []*mp4fio.TrackFrag{
+			{
+				Header: &mp4fio.TrackFragHeader{
+					Data: []byte{0x00, 0x02, 0x00, 0x20, 0x00, 0x00, 0x00, trackID, 0x01, 0x01, 0x00, 0x00},
+				},
+				DecodeTime: &mp4fio.TrackFragDecodeTime{
+					Version: 1,
+					Flags:   0,
+					Time:    m.dts,
+				},
+				Run: run,
+			},
+		},
+	}
+
+	entry := mp4io.TrackFragRunEntry{
+		//Duration: 90000,
+		Size: uint32(len(packet.Payload)),
+	}
+
+	newTime := packet.Timestamp
+	if m.pts > 0 {
+		//m.dts += uint64(newTime - m.pts)
+		entry.Duration = newTime - m.pts
+		m.dts += uint64(entry.Duration)
+	}
+	m.pts = newTime
+
+	// important before moof.Len()
+	run.Entries = append(run.Entries, entry)
+
+	moofLen := moof.Len()
+	mdatLen := 8 + len(packet.Payload)
+
+	// important after moof.Len()
+	run.DataOffset = uint32(moofLen + 8)
+
+	buf := make([]byte, moofLen+mdatLen)
+	moof.Marshal(buf)
+
+	binary.BigEndian.PutUint32(buf[moofLen:], uint32(mdatLen))
+	copy(buf[moofLen+4:], "mdat")
+	copy(buf[moofLen+8:], packet.Payload)
+
+	m.fragIndex++
+
+	m.total += moofLen + mdatLen
+
+	return buf
 }

pkg/mse/consumer.go

@@ -1,131 +0,0 @@
-package mse
-
-import (
-	"encoding/json"
-	"github.com/AlexxIT/go2rtc/pkg/h264"
-	"github.com/AlexxIT/go2rtc/pkg/streamer"
-	"github.com/deepch/vdk/av"
-	"github.com/deepch/vdk/codec/h264parser"
-	"github.com/deepch/vdk/format/mp4f"
-	"github.com/pion/rtp"
-	"time"
-)
-
-const MsgTypeMSE = "mse"
-
-type Consumer struct {
-	streamer.Element
-
-	UserAgent  string
-	RemoteAddr string
-
-	muxer   *mp4f.Muxer
-	streams []av.CodecData
-	start   bool
-
-	send int
-}
-
-func (c *Consumer) GetMedias() []*streamer.Media {
-	return []*streamer.Media{
-		{
-			Kind:      streamer.KindVideo,
-			Direction: streamer.DirectionRecvonly,
-			Codecs: []*streamer.Codec{
-				{Name: streamer.CodecH264, ClockRate: 90000},
-			},
-		}, {
-			Kind:      streamer.KindAudio,
-			Direction: streamer.DirectionRecvonly,
-			Codecs: []*streamer.Codec{
-				{Name: streamer.CodecAAC, ClockRate: 16000},
-			},
-		},
-	}
-}
-
-func (c *Consumer) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.Track {
-	codec := track.Codec
-	switch codec.Name {
-	case streamer.CodecH264:
-		idx := int8(len(c.streams))
-
-		sps, pps := h264.GetParameterSet(codec.FmtpLine)
-		stream, err := h264parser.NewCodecDataFromSPSAndPPS(sps, pps)
-		if err != nil {
-			return nil
-		}
-		c.streams = append(c.streams, stream)
-
-		pkt := av.Packet{Idx: idx, CompositionTime: time.Millisecond}
-
-		ts2time := time.Second / time.Duration(codec.ClockRate)
-
-		push := func(packet *rtp.Packet) error {
-			if packet.Version != h264.RTPPacketVersionAVC {
-				return nil
-			}
-
-			switch h264.NALUType(packet.Payload) {
-			case h264.NALUTypeIFrame:
-				c.start = true
-				pkt.IsKeyFrame = true
-			case h264.NALUTypePFrame:
-				if !c.start {
-					return nil
-				}
-			default:
-				return nil
-			}
-
-			pkt.Data = packet.Payload
-			newTime := time.Duration(packet.Timestamp) * ts2time
-			if pkt.Time > 0 {
-				pkt.Duration = newTime - pkt.Time
-			}
-			pkt.Time = newTime
-
-			for _, buf := range c.muxer.WritePacketV5(pkt) {
-				c.send += len(buf)
-				c.Fire(buf)
-			}
-
-			return nil
-		}
-
-		if !h264.IsAVC(codec) {
-			wrapper := h264.RTPDepay(track)
-			push = wrapper(push)
-		}
-
-		return track.Bind(push)
-	}
-
-	panic("unsupported codec")
-}
-
-func (c *Consumer) Init() {
-	c.muxer = mp4f.NewMuxer(nil)
-	if err := c.muxer.WriteHeader(c.streams); err != nil {
-		return
-	}
-
-	codecs, buf := c.muxer.GetInit(c.streams)
-	c.Fire(&streamer.Message{Type: MsgTypeMSE, Value: codecs})
-
-	c.send += len(buf)
-	c.Fire(buf)
-}
-
-//
-
-func (c *Consumer) MarshalJSON() ([]byte, error) {
-	v := map[string]interface{}{
-		"type":        "MSE server consumer",
-		"send":        c.send,
-		"remote_addr": c.RemoteAddr,
-		"user_agent":  c.UserAgent,
-	}
-
-	return json.Marshal(v)
-}

pkg/rtmp/client.go

@@ -2,7 +2,6 @@ package rtmp
 
 import (
 	"encoding/base64"
-	"encoding/binary"
 	"encoding/hex"
 	"fmt"
 	"github.com/AlexxIT/go2rtc/pkg/h264"
@@ -48,10 +47,14 @@ func (c *Client) Dial() (err error) {
 	for _, stream := range streams {
 		switch stream.Type() {
 		case av.H264:
-			cd := stream.(h264parser.CodecData)
-			fmtp := "sprop-parameter-sets=" +
-				base64.StdEncoding.EncodeToString(cd.RecordInfo.SPS[0]) + "," +
-				base64.StdEncoding.EncodeToString(cd.RecordInfo.PPS[0])
+			info := stream.(h264parser.CodecData).RecordInfo
+
+			fmtp := fmt.Sprintf(
+				"profile-level-id=%02X%02X%02X;sprop-parameter-sets=%s,%s",
+				info.AVCProfileIndication, info.ProfileCompatibility, info.AVCLevelIndication,
+				base64.StdEncoding.EncodeToString(info.SPS[0]),
+				base64.StdEncoding.EncodeToString(info.PPS[0]),
+			)
 
 			codec := &streamer.Codec{
 				Name:        streamer.CodecH264,
@@ -130,22 +133,14 @@ func (c *Client) Handle() (err error) {
 
 		track := c.tracks[int(pkt.Idx)]
 
-		timestamp := uint32(pkt.Time / time.Duration(track.Codec.ClockRate))
+		// convert seconds to RTP timestamp
+		timestamp := uint32(pkt.Time * time.Duration(track.Codec.ClockRate) / time.Second)
 
-		var payloads [][]byte
-		if track.Codec.Name == streamer.CodecH264 {
-			payloads = splitAVC(pkt.Data)
-		} else {
-			payloads = [][]byte{pkt.Data}
-		}
-
-		for _, payload := range payloads {
-			packet := &rtp.Packet{
-				Header:  rtp.Header{Timestamp: timestamp},
-				Payload: payload,
-			}
-			_ = track.WriteRTP(packet)
+		packet := &rtp.Packet{
+			Header:  rtp.Header{Timestamp: timestamp},
+			Payload: pkt.Data,
 		}
+		_ = track.WriteRTP(packet)
 	}
 }
 
@@ -156,21 +151,3 @@ func (c *Client) Close() error {
 	c.closed = true
 	return c.conn.Close()
 }
-
-func splitAVC(data []byte) [][]byte {
-	var nals [][]byte
-	for {
-		// get AVC length
-		size := int(binary.BigEndian.Uint32(data))
-
-		// check if multiple items in one packet
-		if size+4 < len(data) {
-			nals = append(nals, data[:size+4])
-			data = data[size+4:]
-		} else {
-			nals = append(nals, data)
-			break
-		}
-	}
-	return nals
-}

pkg/rtsp/conn.go

@@ -61,10 +61,10 @@ type Conn struct {
 
 	auth     *tcp.Auth
 	conn     net.Conn
+	mode     Mode
 	reader   *bufio.Reader
 	sequence int
-
-	mode Mode
+	uri      string
 
 	tracks   []*streamer.Track
 	channels map[byte]*streamer.Track
@@ -76,24 +76,10 @@ type Conn struct {
 }
 
 func NewClient(uri string) (*Conn, error) {
-	var err error
-
 	c := new(Conn)
-	c.URL, err = url.Parse(uri)
-	if err != nil {
-		return nil, err
-	}
-
-	if strings.IndexByte(c.URL.Host, ':') < 0 {
-		c.URL.Host += ":554"
-	}
-
-	// remove UserInfo from URL
-	c.auth = tcp.NewAuth(c.URL.User)
 	c.mode = ModeClientProducer
-	c.URL.User = nil
-
-	return c, nil
+	c.uri = uri
+	return c, c.parseURI()
 }
 
 func NewServer(conn net.Conn) *Conn {
@@ -104,12 +90,29 @@ func NewServer(conn net.Conn) *Conn {
 	return c
 }
 
+func (c *Conn) parseURI() (err error) {
+	c.URL, err = url.Parse(c.uri)
+	if err != nil {
+		return err
+	}
+
+	if strings.IndexByte(c.URL.Host, ':') < 0 {
+		c.URL.Host += ":554"
+	}
+
+	// remove UserInfo from URL
+	c.auth = tcp.NewAuth(c.URL.User)
+	c.URL.User = nil
+
+	return nil
+}
+
 func (c *Conn) Dial() (err error) {
 	//if c.state != StateClientInit {
 	//	panic("wrong state")
 	//}
-	if c.conn != nil && c.auth != nil {
-		c.auth.Reset()
+	if c.conn != nil {
+		_ = c.parseURI()
 	}
 
 	c.conn, err = net.DialTimeout(
@@ -359,7 +362,21 @@ func (c *Conn) SetupMedia(
 	var res *tcp.Response
 	res, err = c.Do(req)
 	if err != nil {
-		return nil, err
+		// Dahua VTO2111D fail on this step because of backchannel
+		if c.Backchannel {
+			if err = c.Dial(); err != nil {
+				return nil, err
+			}
+			c.Backchannel = false
+			if err = c.Describe(); err != nil {
+				return nil, err
+			}
+			res, err = c.Do(req)
+		}
+
+		if err != nil {
+			return nil, err
+		}
 	}
 
 	if c.Session == "" {
@@ -375,12 +392,16 @@ func (c *Conn) SetupMedia(
 	// we send our `interleaved`, but camera can answer with another
 
 	// Transport: RTP/AVP/TCP;unicast;interleaved=10-11;ssrc=10117CB7
-	// Transport: RTP/AVP/TCP;unicast;destination=192.168.1.123;source=192.168.10.12;interleaved=0
+	// Transport: RTP/AVP/TCP;unicast;destination=192.168.1.111;source=192.168.1.222;interleaved=0
 	// Transport: RTP/AVP/TCP;ssrc=22345682;interleaved=0-1
 	s := res.Header.Get("Transport")
 	// TODO: rewrite
 	if !strings.HasPrefix(s, "RTP/AVP/TCP;") {
-		return nil, fmt.Errorf("wrong transport: %s", s)
+		// Escam Q6 has a bug:
+		// Transport: RTP/AVP;unicast;destination=192.168.1.111;source=192.168.1.222;interleaved=0-1
+		if !strings.Contains(s, ";interleaved=") {
+			return nil, fmt.Errorf("wrong transport: %s", s)
+		}
 	}
 
 	i := strings.Index(s, "interleaved=")

pkg/shell/shell.go

@@ -0,0 +1,41 @@
+package shell
+
+import (
+	"strings"
+)
+
+func QuoteSplit(s string) []string {
+	var a []string
+
+	for len(s) > 0 {
+		is := strings.IndexByte(s, ' ')
+		if is >= 0 {
+			// skip prefix and double spaces
+			if is == 0 {
+				// goto next symbol
+				s = s[1:]
+				continue
+			}
+
+			// check if quote in word
+			if i := strings.IndexByte(s[:is], '"'); i >= 0 {
+				// search quote end
+				if is = strings.Index(s, `" `); is > 0 {
+					is += 1
+				} else {
+					is = -1
+				}
+			}
+		}
+
+		if is >= 0 {
+			a = append(a, strings.ReplaceAll(s[:is], `"`, ""))
+			s = s[is+1:]
+		} else {
+			//add last word
+			a = append(a, s)
+			break
+		}
+	}
+	return a
+}

pkg/srtp/session.go

@@ -90,8 +90,8 @@ func GuessProfile(masterKey []byte) srtp.ProtectionProfile {
 	switch len(masterKey) {
 	case 16:
 		return srtp.ProtectionProfileAes128CmHmacSha1_80
-	case 32:
-		return srtp.ProtectionProfileAes256CmHmacSha1_80
+	//case 32:
+	//	return srtp.ProtectionProfileAes256CmHmacSha1_80
 	}
 	return 0
 }

pkg/streamer/media.go

@@ -5,6 +5,7 @@ import (
 	"github.com/pion/sdp/v3"
 	"strconv"
 	"strings"
+	"unicode"
 )
 
 const (
@@ -24,6 +25,7 @@ const (
 	CodecVP8  = "VP8"
 	CodecVP9  = "VP9"
 	CodecAV1  = "AV1"
+	CodecJPEG = "JPEG" // payloadType: 26
 
 	CodecPCMU = "PCMU" // payloadType: 0
 	CodecPCMA = "PCMA" // payloadType: 8
@@ -35,7 +37,7 @@ const (
 
 func GetKind(name string) string {
 	switch name {
-	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1:
+	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1, CodecJPEG:
 		return KindVideo
 	case CodecPCMU, CodecPCMA, CodecAAC, CodecOpus, CodecG722, CodecMPA:
 		return KindAudio
@@ -128,12 +130,14 @@ type Codec struct {
 func NewCodec(name string) *Codec {
 	name = strings.ToUpper(name)
 	switch name {
-	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1:
+	case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1, CodecJPEG:
 		return &Codec{Name: name, ClockRate: 90000}
 	case CodecPCMU, CodecPCMA:
 		return &Codec{Name: name, ClockRate: 8000}
 	case CodecOpus:
 		return &Codec{Name: name, ClockRate: 48000, Channels: 2}
+	case "MJPEG":
+		return &Codec{Name: CodecJPEG, ClockRate: 90000}
 	}
 
 	panic(fmt.Sprintf("unsupported codec: %s", name))
@@ -154,8 +158,8 @@ func (c *Codec) Clone() *Codec {
 
 func (c *Codec) Match(codec *Codec) bool {
 	return c.Name == codec.Name &&
-		c.ClockRate == codec.ClockRate &&
-		c.Channels == codec.Channels
+		(c.ClockRate == codec.ClockRate || codec.ClockRate == 0) &&
+		(c.Channels == codec.Channels || codec.Channels == 0)
 }
 
 func UnmarshalSDP(rawSDP []byte) ([]*Media, error) {
@@ -242,7 +246,8 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 			ss := strings.Split(attr.Value[i+1:], "/")
 
 			c.Name = strings.ToUpper(ss[0])
-			c.ClockRate = uint32(atoi(ss[1]))
+			// fix tailing space: `a=rtpmap:96 H264/90000 `
+			c.ClockRate = uint32(atoi(strings.TrimRightFunc(ss[1], unicode.IsSpace)))
 
 			if len(ss) == 3 && ss[2] == "2" {
 				c.Channels = 2
@@ -255,6 +260,7 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 	}
 
 	if c.Name == "" {
+		// https://en.wikipedia.org/wiki/RTP_payload_formats
 		switch payloadType {
 		case "0":
 			c.Name = CodecPCMU
@@ -265,6 +271,9 @@ func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
 		case "14":
 			c.Name = CodecMPA
 			c.ClockRate = 44100
+		case "26":
+			c.Name = CodecJPEG
+			c.ClockRate = 90000
 		default:
 			c.Name = payloadType
 		}

pkg/streamer/track.go

@@ -32,6 +32,8 @@ func (t *Track) WriteRTP(p *rtp.Packet) error {
 }
 
 func (t *Track) Bind(w WriterFunc) *Track {
+	t.mx.Lock()
+
 	if t.Sink == nil {
 		t.Sink = map[*Track]WriterFunc{}
 	}
@@ -39,9 +41,10 @@ func (t *Track) Bind(w WriterFunc) *Track {
 	clone := &Track{
 		Codec: t.Codec, Direction: t.Direction, Sink: t.Sink,
 	}
-	t.mx.Lock()
 	t.Sink[clone] = w
+
 	t.mx.Unlock()
+
 	return clone
 }
 

pkg/tcp/auth.go

@@ -80,12 +80,6 @@ func (a *Auth) Write(req *Request) {
 	}
 }
 
-func (a *Auth) Reset() {
-	if a.Method == AuthDigest {
-		a.Method = AuthUnknown
-	}
-}
-
 func Between(s, sub1, sub2 string) string {
 	i := strings.Index(s, sub1)
 	if i < 0 {

pkg/webrtc/api.go

@@ -86,10 +86,6 @@ func RegisterDefaultCodecs(m *webrtc.MediaEngine) error {
 			PayloadType:        98, //123,
 		},
 		// macOS Safari 15.1
-		{
-			RTPCodecCapability: webrtc.RTPCodecCapability{webrtc.MimeTypeH264, 90000, 0, "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f", videoRTCPFeedback},
-			PayloadType:        99,
-		},
 		{
 			RTPCodecCapability: webrtc.RTPCodecCapability{webrtc.MimeTypeH265, 90000, 0, "", videoRTCPFeedback},
 			PayloadType:        100,

pkg/webrtc/conn.go

@@ -59,7 +59,6 @@ func (c *Conn) Init() {
 		}
 
 		fmt.Printf("TODO: webrtc ontrack %+v\n", remote)
-		fmt.Printf("TODO: webrtc ontrack %#v\n", remote)
 	})
 
 	// OK connection:

pkg/webrtc/consumer.go

@@ -3,6 +3,7 @@ package webrtc
 import (
 	"encoding/json"
 	"github.com/AlexxIT/go2rtc/pkg/h264"
+	"github.com/AlexxIT/go2rtc/pkg/h265"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/pion/rtp"
 	"github.com/pion/webrtc/v3"
@@ -51,7 +52,8 @@ func (c *Conn) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.
 			return trackLocal.WriteRTP(packet)
 		}
 
-		if codec.Name == streamer.CodecH264 {
+		switch codec.Name {
+		case streamer.CodecH264:
 			wrapper := h264.RTPPay(1200)
 			push = wrapper(push)
 
@@ -61,6 +63,15 @@ func (c *Conn) AddTrack(media *streamer.Media, track *streamer.Track) *streamer.
 				wrapper = h264.RTPDepay(track)
 			}
 			push = wrapper(push)
+
+		case streamer.CodecH265:
+			// SafariPay because it is the only browser in the world
+			// that supports WebRTC + H265
+			wrapper := h265.SafariPay(1200)
+			push = wrapper(push)
+
+			wrapper = h265.RTPDepay(track)
+			push = wrapper(push)
 		}
 
 		track = track.Bind(push)

pkg/webrtc/helper.go

@@ -1,12 +1,14 @@
 package webrtc
 
 import (
+	"fmt"
 	"github.com/AlexxIT/go2rtc/pkg/streamer"
 	"github.com/pion/ice/v2"
 	"github.com/pion/stun"
 	"github.com/pion/webrtc/v3"
 	"net"
 	"strconv"
+	"strings"
 )
 
 func NewCandidate(address string) (string, error) {
@@ -34,6 +36,31 @@ func NewCandidate(address string) (string, error) {
 	return "candidate:" + cand.Marshal(), nil
 }
 
+func LookupIP(address string) (string, error) {
+	if strings.HasPrefix(address, "stun:") {
+		ip, err := GetPublicIP()
+		if err != nil {
+			return "", err
+		}
+		return ip.String() + address[4:], nil
+	}
+
+	if IsIP(address) {
+		return address, nil
+	}
+
+	i := strings.IndexByte(address, ':')
+	ips, err := net.LookupIP(address[:i])
+	if err != nil {
+		return "", err
+	}
+	if len(ips) == 0 {
+		return "", fmt.Errorf("can't resolve: %s", address)
+	}
+
+	return ips[0].String() + address[i:], nil
+}
+
 // GetPublicIP example from https://github.com/pion/stun
 func GetPublicIP() (net.IP, error) {
 	c, err := stun.Dial("udp", "stun.l.google.com:19302")
@@ -63,6 +90,15 @@ func GetPublicIP() (net.IP, error) {
 	return xorAddr.IP, nil
 }
 
+func IsIP(host string) bool {
+	for _, i := range host {
+		if i >= 'A' {
+			return false
+		}
+	}
+	return true
+}
+
 func MimeType(codec *streamer.Codec) string {
 	switch codec.Name {
 	case streamer.CodecH264:

www/README.md

@@ -51,4 +51,5 @@ pc.ontrack = ev => {
 ## Useful links
 
 - https://www.webrtc-experiment.com/DetectRTC/
-- https://divtable.com/table-styler/
+- https://divtable.com/table-styler/
+- https://www.chromium.org/audio-video/

www/codecs.html

@@ -45,7 +45,9 @@
         'video/mp4; codecs="avc1.640032"',
         'video/mp4; codecs="avc1.640C32"',
         'video/mp4; codecs="avc1.F4001F"',
-        'video/mp4; codecs="hvc1.016000"',
+        'video/mp4; codecs="hvc1.1.6.L93.B0"',
+        'video/mp4; codecs="hev1.1.6.L93.B0"',
+        'video/mp4; codecs="hev1.2.4.L120.B0"',
     ];
 
     const video = document.createElement("video");

www/index.html

@@ -66,7 +66,11 @@
     const links = [
         '<a href="webrtc.html?src={name}">webrtc</a>',
         '<a href="mse.html?src={name}">mse</a>',
-        '<a href="api/frame.mp4?src={name}">frame.mp4</a>',
+        // '<a href="video.html?src={name}">video</a>',
+        '<a href="api/stream.mp4?src={name}">mp4</a>',
+        '<a href="api/frame.mp4?src={name}">frame</a>',
+        `<a href="rtsp://${location.hostname}:8554/{name}">rtsp</a>`,
+        '<a href="api/stream.mjpeg?src={name}">mjpeg</a>',
         '<a href="api/streams?src={name}">info</a>',
     ];
 

www/mse.html

@@ -60,9 +60,7 @@
             console.debug("ws.onmessage", data);
 
             if (data.type === "mse") {
-                sourceBuffer = mediaSource.addSourceBuffer(
-                    `video/mp4; codecs="${data.value}"`
-                );
+                sourceBuffer = mediaSource.addSourceBuffer(data.value);
                 // important: segments supports TrackFragDecodeTime
                 // sequence supports only TrackFragRunEntry Duration
                 sourceBuffer.mode = "segments";

www/video.html

@@ -0,0 +1,53 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <title>go2rtc - WebRTC</title>
+    <style>
+        body {
+            margin: 0;
+            padding: 0;
+        }
+
+        html, body {
+            height: 100%;
+            width: 100%;
+        }
+
+        #video {
+            /* video "container" size */
+            width: 100%;
+            height: 100%;
+            background: black;
+        }
+    </style>
+</head>
+<body>
+<video id="video" autoplay controls playsinline muted></video>
+<!--<video id="video" preload="auto" controls playsinline muted></video>-->
+<script>
+    const baseUrl = location.origin + location.pathname.substr(
+        0, location.pathname.lastIndexOf("/")
+    );
+    const video = document.getElementById('video');
+
+    video.oncanplay = ev => console.log(ev.type, ev);
+    video.onplaying = ev => console.log(ev.type, ev);
+    video.onwaiting = ev => console.log(ev.type, ev);
+    video.onseeking = ev => console.log(ev.type, ev);
+    video.onloadeddata = ev => console.log(ev.type, ev);
+    video.oncanplaythrough = ev => console.log(ev.type, ev);
+    // video.ondurationchange = ev => console.log(ev.type, ev);
+    // video.ontimeupdate = ev => console.log(ev.type, ev);
+    video.onplay = ev => console.log(ev.type, ev);
+    video.onpause = ev => console.log(ev.type, ev);
+    video.onsuspended = ev => console.log(ev.type, ev);
+    video.onemptied = ev => console.log(ev.type, ev);
+    video.onstalled = ev => console.log(ev.type, ev);
+
+    console.log("start");
+
+    video.src = baseUrl + "/api/stream.mp4" + location.search;
+</script>
+</body>
+</html>