apps/go2rtc
1
0
Fork 0
mirror of https://github.com/AlexxIT/go2rtc.git synced 2025-11-01 12:12:40 +08:00
Code Issues Packages Projects Releases Wiki Activity

Improve WebRTC candidates handling

Browse Source
This commit is contained in:
Alex X
2024-05-10 17:26:52 +03:00
parent f7b98044e6
commit 205018c96a
8 changed files with 288 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
internal/ngrok/ngrok.go
View File

@@ -50,7 +50,7 @@ func Init() {
log.Info().Str("addr", address).Msg("[ngrok] add external candidate for WebRTC")
webrtc.AddCandidate(address, "tcp")
webrtc.AddCandidate("tcp", address)
}
}
})

106
internal/webrtc/README.md
View File

@@ -1,13 +1,105 @@
What you should to know about WebRTC:
- It's almost always a **direct [peer-to-peer](https://en.wikipedia.org/wiki/Peer-to-peer) connection** from your browser to go2rtc app
- When you use Home Assistant, Frigate, Nginx, Nabu Casa, Cloudflare and other software - they are only **involved in establishing** the connection, but they are **not involved in transferring** media data
- WebRTC media cannot be transferred inside an HTTP connection
- Usually, WebRTC uses random UDP ports on client and server side to establish a connection
- Usually, WebRTC uses public [STUN](https://en.wikipedia.org/wiki/STUN) servers to establish a connection outside LAN, such servers are only needed to establish a connection and are not involved in data transfer
- Usually, WebRTC will automatically discover all of your local addresses and all of your public addresses and try to establish a connection
If an external connection via STUN is used:
- Uses [UDP hole punching](https://en.wikipedia.org/wiki/UDP_hole_punching) technology to bypass NAT even if you not open your server to the World
- For about 20% of users, the techology will not work because of the [Symmetric NAT](https://tomchen.github.io/symmetric-nat-test/)
- UDP is not suitable for transmitting 2K and 4K high bitrate video over open networks because of the high loss rate
## Default config
```yaml
webrtc:
listen: ":8555/tcp"
ice_servers:
- urls: [ "stun:stun.l.google.com:19302" ]
```
## Config
- supported TCP: fixed port (default), disabled
- supported UDP: random port (default), fixed port
**Important!** This example is not for copypasting!
| Config examples | TCP | UDP |
|-----------------------|-------|--------|
| `listen: ":8555/tcp"` | fixed | random |
| `listen: ":8555"` | fixed | fixed |
| `listen: ""` | no | random |
```yaml
webrtc:
# fix local TCP or UDP or both ports for WebRTC media
listen: ":8555/tcp" # address of your local server
# add additional host candidates manually
# order is important, the first will have a higher priority
candidates:
- 216.58.210.174:8555 # if you have static public IP-address
- stun:8555 # if you have dynamic public IP-address
- home.duckdns.org:8555 # if you have domain
# add custom STUN and TURN servers
# use `ice_servers: []` for remove defaults and leave empty
ice_servers:
- urls: [ stun:stun1.l.google.com:19302 ]
- urls: [ turn:123.123.123.123:3478 ]
username: your_user
credential: your_pass
# optional filter list for auto discovery logic
# some settings only make sense if you don't specify a fixed UDP port
filters:
# list of host candidates from auto discovery to be sent
# including candidates from the `listen` option
# use `candidates: []` to remove all auto discovery candidates
candidates: [ 192.168.1.123 ]
# list of network types to be used for connection
# including candidates from the `listen` option
networks: [ udp4, udp6, tcp4, tcp6 ]
# list of interfaces to be used for connection
# not related to the `listen` option
interfaces: [ eno1 ]
# list of host IP-addresses to be used for connection
# not related to the `listen` option
ips: [ 192.168.1.123 ]
# range for random UDP ports [min, max] to be used for connection
# not related to the `listen` option
udp_ports: [ 50000, 50100 ]
```
By default go2rtc uses **fixed TCP** port and multiple **random UDP** ports for each WebRTC connection - `listen: ":8555/tcp"`.
You can set **fixed TCP** and **fixed UDP** port for all connections - `listen: ":8555"`. This may has lower performance, but it's your choice.
Don't know why, but you can disable TCP port and leave only random UDP ports - `listen: ""`.
## Config filters
Filters allow you to exclude unnecessary candidates. Extra candidates don't make your connection worse or better. But the wrong filter settings can break everything. Skip this setting if you don't understand it.
For example, go2rtc is installed on the host system. And there are unnecessary interfaces. You can keep only the relevant via `interfaces` or `ips` options. You can also exclude IPv6 candidates if your server supports them but your home network does not.
```yaml
webrtc:
listen: ":8555/tcp" # use fixed TCP port and random UDP ports
filters:
ips: [ 192.168.1.2 ] # IP-address of your server
networks: [ udp4, tcp4 ] # skip IPv6, if it's not supported for you
```
For example, go2rtc inside closed docker container (ex. [Frigate](https://frigate.video/)). You shouldn't filter docker interfaces, otherwise go2rtc will not be able to connect anywhere. But you can filter the docker candidates because no one can connect to them.
```yaml
webrtc:
listen: ":8555" # use fixed TCP and UDP ports
candidates: [ 192.168.1.2:8555 ] # add manual host candidate (use docker port forwarding)
filters:
candidates: [] # skip all internal docker candidates
```
## Userful links

105
internal/webrtc/candidates.go
View File

@@ -2,57 +2,60 @@ package webrtc
import (
"net"
"slices"
"strings"
"github.com/AlexxIT/go2rtc/internal/api/ws"
"github.com/AlexxIT/go2rtc/pkg/webrtc"
"github.com/pion/sdp/v3"
pion "github.com/pion/webrtc/v3"
)
type Address struct {
Host string
host string
Port string
Network string
Offset int
Priority uint32
}
func (a *Address) Marshal() string {
host := a.Host
if host == "stun" {
func (a *Address) Host() string {
if a.host == "stun" {
ip, err := webrtc.GetCachedPublicIP()
if err != nil {
return ""
}
host = ip.String()
return ip.String()
}
return a.host
}
switch a.Network {
case "udp":
return webrtc.CandidateManualHostUDP(host, a.Port, a.Offset)
case "tcp":
return webrtc.CandidateManualHostTCPPassive(host, a.Port, a.Offset)
func (a *Address) Marshal() string {
if host := a.Host(); host != "" {
return webrtc.CandidateICE(a.Network, host, a.Port, a.Priority)
}
return ""
}
var addresses []*Address
var filters webrtc.Filters
func AddCandidate(network, address string) {
if network == "" {
AddCandidate("tcp", address)
AddCandidate("udp", address)
return
}
func AddCandidate(address, network string) {
host, port, err := net.SplitHostPort(address)
if err != nil {
return
}
offset := -1 - len(addresses) // every next candidate will have a lower priority
// start from 1, so manual candidates will be lower than built-in
// and every next candidate will have a lower priority
candidateIndex := 1 + len(addresses)
switch network {
case "tcp", "udp":
addresses = append(addresses, &Address{host, port, network, offset})
default:
addresses = append(
addresses, &Address{host, port, "udp", offset}, &Address{host, port, "tcp", offset},
)
}
priority := webrtc.CandidateHostPriority(network, candidateIndex)
addresses = append(addresses, &Address{host, port, network, priority})
}
func GetCandidates() (candidates []string) {
@@ -64,6 +67,38 @@ func GetCandidates() (candidates []string) {
return
}
// FilterCandidate return true if candidate passed the check
func FilterCandidate(candidate *pion.ICECandidate) bool {
if candidate == nil {
return false
}
// host candidate should be in the hosts list
if candidate.Typ == pion.ICECandidateTypeHost && filters.Candidates != nil {
if !slices.Contains(filters.Candidates, candidate.Address) {
return false
}
}
if filters.Networks != nil {
networkType := NetworkType(candidate.Protocol.String(), candidate.Address)
if !slices.Contains(filters.Networks, networkType) {
return false
}
}
return true
}
// NetworkType convert tcp/udp network to tcp4/tcp6/udp4/udp6
func NetworkType(network, host string) string {
if strings.IndexByte(host, ':') >= 0 {
return network + "6"
} else {
return network + "4"
}
}
func asyncCandidates(tr *ws.Transport, cons *webrtc.Conn) {
tr.WithContext(func(ctx map[any]any) {
if candidates, ok := ctx["candidate"].([]string); ok {
@@ -86,30 +121,6 @@ func asyncCandidates(tr *ws.Transport, cons *webrtc.Conn) {
}
}
func syncCanditates(answer string) (string, error) {
if len(addresses) == 0 {
return answer, nil
}
sd := &sdp.SessionDescription{}
if err := sd.Unmarshal([]byte(answer)); err != nil {
return "", err
}
md := sd.MediaDescriptions[0]
for _, candidate := range GetCandidates() {
md.WithPropertyAttribute(candidate)
}
data, err := sd.Marshal()
if err != nil {
return "", err
}
return string(data), nil
}
func candidateHandler(tr *ws.Transport, msg *ws.Message) error {
// process incoming candidate in sync function
tr.WithContext(func(ctx map[any]any) {

5
internal/webrtc/server.go
View File

@@ -178,10 +178,7 @@ func inputWebRTC(w http.ResponseWriter, r *http.Request) {
return
}
answer, err := prod.GetCompleteAnswer()
if err == nil {
answer, err = syncCanditates(answer)
}
answer, err := prod.GetCompleteAnswer(GetCandidates(), FilterCandidate)
if err != nil {
log.Warn().Err(err).Caller().Send()
http.Error(w, err.Error(), http.StatusInternalServerError)

25
internal/webrtc/webrtc.go
View File

@@ -20,6 +20,7 @@ func Init() {
Listen string `yaml:"listen"`
Candidates []string `yaml:"candidates"`
IceServers []pion.ICEServer `yaml:"ice_servers"`
Filters webrtc.Filters `yaml:"filters"`
} `yaml:"webrtc"`
}
@@ -32,20 +33,15 @@ func Init() {
log = app.GetLogger("webrtc")
filters = cfg.Mod.Filters
address, network, _ := strings.Cut(cfg.Mod.Listen, "/")
var candidateHost []string
for _, candidate := range cfg.Mod.Candidates {
if strings.HasPrefix(candidate, "host:") {
candidateHost = append(candidateHost, candidate[5:])
continue
}
AddCandidate(candidate, network)
AddCandidate(network, candidate)
}
// create pionAPI with custom codecs list and custom network settings
serverAPI, err := webrtc.NewServerAPI(address, network, candidateHost)
serverAPI, err := webrtc.NewServerAPI(network, address, &filters)
if err != nil {
log.Error().Err(err).Caller().Send()
return
@@ -55,8 +51,7 @@ func Init() {
clientAPI := serverAPI
if address != "" {
log.Info().Str("addr", address).Msg("[webrtc] listen")
log.Info().Str("addr", cfg.Mod.Listen).Msg("[webrtc] listen")
clientAPI, _ = webrtc.NewAPI()
}
@@ -139,6 +134,9 @@ func asyncHandler(tr *ws.Transport, msg *ws.Message) error {
}
case *pion.ICECandidate:
if !FilterCandidate(msg) {
return
}
_ = sendAnswer.Wait()
s := msg.ToJSON().Candidate
@@ -248,10 +246,7 @@ func ExchangeSDP(stream *streams.Stream, offer, desc, userAgent string) (answer
stream.AddProducer(conn)
}
answer, err = conn.GetCompleteAnswer()
if err == nil {
answer, err = syncCanditates(answer)
}
answer, err = conn.GetCompleteAnswer(GetCandidates(), FilterCandidate)
log.Trace().Msgf("[webrtc] answer\n%s", answer)
if err != nil {

65
pkg/webrtc/api.go
View File

@@ -2,6 +2,7 @@ package webrtc
import (
"net"
"slices"
"github.com/pion/interceptor"
"github.com/pion/webrtc/v3"
@@ -15,7 +16,15 @@ func NewAPI() (*webrtc.API, error) {
return NewServerAPI("", "", nil)
}
func NewServerAPI(address, network string, candidateHost []string) (*webrtc.API, error) {
type Filters struct {
Candidates []string `yaml:"candidates"`
Interfaces []string `yaml:"interfaces"`
IPs []string `yaml:"ips"`
Networks []string `yaml:"networks"`
UDPPorts []uint16 `yaml:"udp_ports"`
}
func NewServerAPI(network, address string, filters *Filters) (*webrtc.API, error) {
// for debug logs add to env: `PION_LOG_DEBUG=all`
m := &webrtc.MediaEngine{}
//if err := m.RegisterDefaultCodecs(); err != nil {
@@ -32,23 +41,55 @@ func NewServerAPI(address, network string, candidateHost []string) (*webrtc.API,
s := webrtc.SettingEngine{}
// fix https://github.com/pion/webrtc/pull/2407
s.SetDTLSInsecureSkipHelloVerify(true)
if filters != nil && filters.Interfaces != nil {
s.SetIncludeLoopbackCandidate(true)
s.SetInterfaceFilter(func(name string) bool {
return slices.Contains(filters.Interfaces, name)
})
} else {
// disable listen on Hassio docker interfaces
s.SetInterfaceFilter(func(name string) bool {
return name != "hassio" && name != "docker0"
})
}
// fix https://github.com/pion/webrtc/pull/2407
s.SetDTLSInsecureSkipHelloVerify(true)
s.SetReceiveMTU(ReceiveMTU)
s.SetNAT1To1IPs(candidateHost, webrtc.ICECandidateTypeHost)
// by default enable IPv4 + IPv6 modes
s.SetNetworkTypes([]webrtc.NetworkType{
webrtc.NetworkTypeUDP4, webrtc.NetworkTypeTCP4,
webrtc.NetworkTypeUDP6, webrtc.NetworkTypeTCP6,
if filters != nil && filters.IPs != nil {
s.SetIncludeLoopbackCandidate(true)
s.SetIPFilter(func(ip net.IP) bool {
return slices.Contains(filters.IPs, ip.String())
})
}
if filters != nil && filters.Networks != nil {
var networkTypes []webrtc.NetworkType
for _, s := range filters.Networks {
if networkType, err := webrtc.NewNetworkType(s); err == nil {
networkTypes = append(networkTypes, networkType)
}
}
s.SetNetworkTypes(networkTypes)
} else {
s.SetNetworkTypes([]webrtc.NetworkType{
webrtc.NetworkTypeUDP4, webrtc.NetworkTypeUDP6,
webrtc.NetworkTypeTCP4, webrtc.NetworkTypeTCP6,
})
}
if filters != nil && len(filters.UDPPorts) == 2 {
_ = s.SetEphemeralUDPPortRange(filters.UDPPorts[0], filters.UDPPorts[1])
}
//if len(hosts) != 0 {
// // support only: host, srflx
// if candidateType, err := webrtc.NewICECandidateType(hosts[0]); err == nil {
// s.SetNAT1To1IPs(hosts[1:], candidateType)
// } else {
// s.SetNAT1To1IPs(hosts, 0) // 0 = host
// }
//}
if address != "" {
if network == "" || network == "tcp" {

51
pkg/webrtc/helpers.go
View File

@@ -273,38 +273,41 @@ func MimeType(codec *core.Codec) string {
panic("not implemented")
}
// 4.1.2.2. Guidelines for Choosing Type and Local Preferences
// The RECOMMENDED values are 126 for host candidates, 100
// for server reflexive candidates, 110 for peer reflexive candidates,
// and 0 for relayed candidates.
const PriorityTypeHostUDP = (1 << 24) * int(126)
const PriorityTypeHostTCP = (1 << 24) * int(126-27)
const PriorityLocalUDP = (1 << 8) * int(65535)
const PriorityLocalTCPPassive = (1 << 8) * int((1<<13)*4+8191)
const PriorityComponentRTP = 1 * int(256-ice.ComponentRTP)
func CandidateManualHostUDP(host, port string, offset int) string {
foundation := crc32.ChecksumIEEE([]byte("host" + host + "udp4"))
priority := PriorityTypeHostUDP + PriorityLocalUDP + PriorityComponentRTP + offset
func CandidateICE(network, host, port string, priority uint32) string {
// 1. Foundation
// 2. Component, always 1 because RTP
// 3. udp or tcp
// 3. "udp" or "tcp"
// 4. Priority
// 5. Host - IP4 or IP6 or domain name
// 6. Port
// 7. typ host
return fmt.Sprintf("candidate:%d 1 udp %d %s %s typ host", foundation, priority, host, port)
// 7. "typ host"
foundation := crc32.ChecksumIEEE([]byte("host" + host + network + "4"))
s := fmt.Sprintf("candidate:%d 1 %s %d %s %s typ host", foundation, network, priority, host, port)
if network == "tcp" {
return s + " tcptype passive"
}
return s
}
func CandidateManualHostTCPPassive(host, port string, offset int) string {
foundation := crc32.ChecksumIEEE([]byte("host" + host + "tcp4"))
priority := PriorityTypeHostTCP + PriorityLocalTCPPassive + PriorityComponentRTP + offset
// Priority = type << 24 + local << 8 + component
// https://www.rfc-editor.org/rfc/rfc8445#section-5.1.2.1
return fmt.Sprintf(
"candidate:%d 1 tcp %d %s %s typ host tcptype passive", foundation, priority, host, port,
)
const PriorityHostUDP uint32 = 0x001F_FFFF |
126<<24 | // udp host
7<<21 // udp
const PriorityHostTCPPassive uint32 = 0x001F_FFFF |
99<<24 | // tcp host
4<<21 // tcp passive
// CandidateHostPriority (lower indexes has a higher priority)
func CandidateHostPriority(network string, index int) uint32 {
switch network {
case "udp":
return PriorityHostUDP - uint32(index)
case "tcp":
return PriorityHostTCPPassive - uint32(index)
}
return 0
}
func UnmarshalICEServers(b []byte) ([]webrtc.ICEServer, error) {

41
pkg/webrtc/server.go
View File

@@ -81,11 +81,42 @@ transeivers:
return c.pc.LocalDescription().SDP, nil
}
func (c *Conn) GetCompleteAnswer() (answer string, err error) {
if _, err = c.GetAnswer(); err != nil {
return
// GetCompleteAnswer - get SDP answer with candidates inside
func (c *Conn) GetCompleteAnswer(candidates []string, filter func(*webrtc.ICECandidate) bool) (string, error) {
var done = make(chan struct{})
c.pc.OnICECandidate(func(candidate *webrtc.ICECandidate) {
if candidate != nil {
if filter == nil || filter(candidate) {
candidates = append(candidates, candidate.ToJSON().Candidate)
}
} else {
done <- struct{}{}
}
})
answer, err := c.GetAnswer()
if err != nil {
return "", err
}
<-webrtc.GatheringCompletePromise(c.pc)
return c.pc.LocalDescription().SDP, nil
<-done
sd := &sdp.SessionDescription{}
if err = sd.Unmarshal([]byte(answer)); err != nil {
return "", err
}
md := sd.MediaDescriptions[0]
for _, candidate := range candidates {
md.WithPropertyAttribute(candidate)
}
b, err := sd.Marshal()
if err != nil {
return "", err
}
return string(b), nil
}