Commit Graph

104 Commits

Author SHA1 Message Date
Ingo Oppermann
b5976f37f0 Start cluster after core API has been started
The core API will be started before the cluster is started in order to
access the cluster endpoints during a cluster upgrade. If TLS is enabled,
possibly stale certificates are loaded into the cache. Otherwise the
leader has to be contacted via the cluster API which might have changed.
2023-07-10 16:24:08 +02:00
Ingo Oppermann
9462ccfd48 Move IAM dependency to rewriter module 2023-07-10 13:20:24 +02:00
Ingo Oppermann
71b613371a Add debug flag to cluster to disable FFmpeg check 2023-07-10 11:07:24 +02:00
Ingo Oppermann
9905ac70f0 Check for equal FFmpeg skills, remove /v1/version 2023-07-06 16:12:06 +02:00
Ingo Oppermann
a8965cdec7 Change core shutdown sequence
Now the cluster, restreamer (processes), and the servers (HTTP, HTTP, RTMP, SRT)
are shutdown first, then any other services. Before, e.g. the session collector
was closed very early before the servers. This would have lead to loss of a few
sessions.
2023-07-04 12:35:08 +02:00
Ingo Oppermann
c5d52dbeb3 Add gops agent
The gops agent can be enabled by providing an address to
CORE_DEBUG_AGENT_ADDRESS where it will listen to requests
from the gops utility.
2023-07-03 21:27:44 +02:00
Ingo Oppermann
adcbd98467 Add CORE_TLS_SECRET configuration
This secret will be used to encrypt automatically obtained secrets at
rest, i.e. in a storage. They will be decrypted on demand. If the
secret is wrong, stored certificates can't be decrypted. For changing
the secret, the stored certificated must be deleted first in order
to obtain new ones that will be encrypted with the new secret.
2023-07-03 16:02:39 +02:00
Ingo Oppermann
6e156d0f3a Allow to acquire certificates in multi-node cluster 2023-06-29 21:15:04 +02:00
Ingo Oppermann
a6d454b03f Add CORE_TLS_STAGING
Set this value to true in order to use the Let's Encrypt staging CA,
otherwise the production CA will be used, which is the default.
2023-06-28 11:55:56 +02:00
Ingo Oppermann
72a0566bc6 Add CORE_CLUSTER_STARTUP_TIMEOUT_SEC configuration value 2023-06-28 11:41:24 +02:00
Ingo Oppermann
dd128ac99b Fix gathering of hostnames, add /v3/cluster/db/kv endpoint 2023-06-27 14:52:32 +02:00
Ingo Oppermann
7fc58454e4 Allow to cancel core startup 2023-06-26 21:02:31 +02:00
Ingo Oppermann
a4b0c4fc36 Add experimental LE certificate retrieval for cluster 2023-06-26 20:38:16 +02:00
Ingo Oppermann
f37896a1e3 Implement certmagic.Storage on cluster 2023-06-23 21:00:45 +02:00
Ingo Oppermann
d3eed2a417 Allow to log each finished session to filesystem
By providing CORE_SESSIONS_SESSION_LOG_PATH_PATTERN (e.g. "/log/%Y-%m-%d.log") all
finished sessions will be logged to a file according to the provided strftime-pattern. The
actual value is calculated from when the session closed. CORE_SESSIONS_PERSIST must
be set. Default: not set.

Set CORE_SESSIONS_SESSION_LOG_BUFFER_SEC to the number of seconds the log should be
buffered in memory before persisted to disk. Default 15 seconds.
2023-06-20 15:19:25 +02:00
Ingo Oppermann
3cc8626ae1 Fix RTMP and SRT URL generation in placeholder 2023-06-15 14:19:02 +02:00
Ingo Oppermann
debcce192d Add the process domain to the bas path in placeholders and cleanup rules 2023-06-14 22:30:00 +02:00
Ingo Oppermann
a728cc7839 Fix clusterNode/proxyNode 2023-06-14 22:05:59 +02:00
Ingo Oppermann
ccf4bee56d WIP: cluster node revamp, add version and config comparison 2023-06-13 10:25:08 +02:00
Ingo Oppermann
7e7d1caca7 Add EmergencyLeaderTimeout parameter, ignore throttling nodes, implement NodeRecoverTimeout, introduce processNodeMap in cluster DB 2023-06-07 22:08:07 +02:00
Ingo Oppermann
bd75a5ad0f Add SyncInterval and NodeRecoverTimeout to cluster config 2023-06-07 10:18:23 +02:00
Ingo Oppermann
fc03bf73a2 Make resource manager a main module and expose more details 2023-06-06 21:28:08 +02:00
Ingo Oppermann
3ac7ead20d Add the resource limits to the metrics 2023-06-06 15:20:59 +02:00
Ingo Oppermann
1f6f7c9f59 Add API endpoint to leave cluster gracefully, hard leave on exit 2023-06-06 11:22:53 +02:00
Ingo Oppermann
a79cfa0c77 Merge branch 'clusteriam' into vod 2023-06-01 15:16:54 +02:00
Ingo Oppermann
f21ca2055e Use IAM for placeholders, simplify proxy nodes 2023-05-31 21:49:25 +02:00
Ingo Oppermann
89cc42c4b9 Add cluster IAM, update dependencies 2023-05-30 21:24:12 +02:00
Ingo Oppermann
e9034aa171 Create identity and access packages for IAM 2023-05-25 16:16:29 +02:00
Ingo Oppermann
710d5c595f Merge branch 'iam' into clusteriam 2023-05-25 11:05:26 +02:00
Ingo Oppermann
10df8d51b1 Simplify IAM user config, only create users and policies if not previously run 2023-05-24 22:28:24 +02:00
Ingo Oppermann
3c89cbb831 Create identities for basic auth access to mount points 2023-05-24 14:29:14 +02:00
Ingo Oppermann
ccac2ffd5d Various updates
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
cd58cf3df7 Fix panics if cluster is not enabled 2023-05-05 11:56:39 +02:00
Ingo Oppermann
7f59c188cf Remove node storage, use raft configuration instead; re-establish file and stream proxying 2023-05-04 19:49:53 +02:00
Ingo Oppermann
d201921a33 Allow to provide complete cluster configuration
Replace CORE_CLUSTER_JOIN_ADDRESS with CORE_CLUSTER_PEERS. This is
a comma separated list of cluster members with their IDs of the form
ID@host:port

On startup the node tries to connect to all the peers. In case of
sudden deaths of a node this will allow to find back into the
cluster. The list in CLUSTER_PEERS is a starting point of known
peers. Other node that are not in that list can still join the
cluster.

File and stream proxy has been moved to the Proxy type.
2023-05-03 16:13:05 +02:00
Ingo Oppermann
4eb0258ba4 Add automatic bootstrap and recovery 2023-05-02 15:49:46 +02:00
Ingo Oppermann
7527f7833b Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
3a1825cf61 Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
386bdc0b6e WIP: add emergency leader, api circuit breaker 2023-04-21 17:19:21 +02:00
Ingo Oppermann
27f19f9188 WIP: internal API 2023-04-17 14:01:20 +02:00
Ingo Oppermann
7643959bf8 WIP: raft leadership 2023-04-13 21:44:24 +02:00
Ingo Oppermann
e6b64dbe97 WIP: raft 2023-04-13 09:42:19 +02:00
Ingo Oppermann
ffdf6d3323 Merge branch 'dev' into cluster 2023-04-12 15:27:38 +02:00
Ingo Oppermann
8326160b28 Add option to set GOMAXPROCS automatically
Use the debug.auto_max_procs config value or the CORE_DEBUG_AUTO_MAX_PROCS
environment variable. Set it to "true" to enable this feature.
2023-04-11 17:12:01 +02:00
Ingo Oppermann
0bd68d6dc3 Fix debug.force_gc behaviour 2023-04-11 11:08:11 +02:00
Ingo Oppermann
507b4f4b9a Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
84817f137a Merge branch 'dev' into iam 2023-04-03 15:44:16 +02:00
Ingo Oppermann
99c571d623 Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b84fdddd81 Add /api/v3/events endpoint 2023-03-23 11:11:47 +01:00
Ingo Oppermann
674c444d17 Merge branch 'dev' into vod 2023-03-17 18:41:16 +01:00