Commit Graph

97 Commits

Author SHA1 Message Date
Ingo Oppermann
6e156d0f3a Allow to acquire certificates in multi-node cluster 2023-06-29 21:15:04 +02:00
Ingo Oppermann
a6d454b03f Add CORE_TLS_STAGING
Set this value to true in order to use the Let's Encrypt staging CA,
otherwise the production CA will be used, which is the default.
2023-06-28 11:55:56 +02:00
Ingo Oppermann
72a0566bc6 Add CORE_CLUSTER_STARTUP_TIMEOUT_SEC configuration value 2023-06-28 11:41:24 +02:00
Ingo Oppermann
dd128ac99b Fix gathering of hostnames, add /v3/cluster/db/kv endpoint 2023-06-27 14:52:32 +02:00
Ingo Oppermann
7fc58454e4 Allow to cancel core startup 2023-06-26 21:02:31 +02:00
Ingo Oppermann
a4b0c4fc36 Add experimental LE certificate retrieval for cluster 2023-06-26 20:38:16 +02:00
Ingo Oppermann
f37896a1e3 Implement certmagic.Storage on cluster 2023-06-23 21:00:45 +02:00
Ingo Oppermann
d3eed2a417 Allow to log each finished session to filesystem
By providing CORE_SESSIONS_SESSION_LOG_PATH_PATTERN (e.g. "/log/%Y-%m-%d.log") all
finished sessions will be logged to a file according to the provided strftime-pattern. The
actual value is calculated from when the session closed. CORE_SESSIONS_PERSIST must
be set. Default: not set.

Set CORE_SESSIONS_SESSION_LOG_BUFFER_SEC to the number of seconds the log should be
buffered in memory before persisted to disk. Default 15 seconds.
2023-06-20 15:19:25 +02:00
Ingo Oppermann
3cc8626ae1 Fix RTMP and SRT URL generation in placeholder 2023-06-15 14:19:02 +02:00
Ingo Oppermann
debcce192d Add the process domain to the bas path in placeholders and cleanup rules 2023-06-14 22:30:00 +02:00
Ingo Oppermann
a728cc7839 Fix clusterNode/proxyNode 2023-06-14 22:05:59 +02:00
Ingo Oppermann
ccf4bee56d WIP: cluster node revamp, add version and config comparison 2023-06-13 10:25:08 +02:00
Ingo Oppermann
7e7d1caca7 Add EmergencyLeaderTimeout parameter, ignore throttling nodes, implement NodeRecoverTimeout, introduce processNodeMap in cluster DB 2023-06-07 22:08:07 +02:00
Ingo Oppermann
bd75a5ad0f Add SyncInterval and NodeRecoverTimeout to cluster config 2023-06-07 10:18:23 +02:00
Ingo Oppermann
fc03bf73a2 Make resource manager a main module and expose more details 2023-06-06 21:28:08 +02:00
Ingo Oppermann
3ac7ead20d Add the resource limits to the metrics 2023-06-06 15:20:59 +02:00
Ingo Oppermann
1f6f7c9f59 Add API endpoint to leave cluster gracefully, hard leave on exit 2023-06-06 11:22:53 +02:00
Ingo Oppermann
a79cfa0c77 Merge branch 'clusteriam' into vod 2023-06-01 15:16:54 +02:00
Ingo Oppermann
f21ca2055e Use IAM for placeholders, simplify proxy nodes 2023-05-31 21:49:25 +02:00
Ingo Oppermann
89cc42c4b9 Add cluster IAM, update dependencies 2023-05-30 21:24:12 +02:00
Ingo Oppermann
e9034aa171 Create identity and access packages for IAM 2023-05-25 16:16:29 +02:00
Ingo Oppermann
710d5c595f Merge branch 'iam' into clusteriam 2023-05-25 11:05:26 +02:00
Ingo Oppermann
10df8d51b1 Simplify IAM user config, only create users and policies if not previously run 2023-05-24 22:28:24 +02:00
Ingo Oppermann
3c89cbb831 Create identities for basic auth access to mount points 2023-05-24 14:29:14 +02:00
Ingo Oppermann
ccac2ffd5d Various updates
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
cd58cf3df7 Fix panics if cluster is not enabled 2023-05-05 11:56:39 +02:00
Ingo Oppermann
7f59c188cf Remove node storage, use raft configuration instead; re-establish file and stream proxying 2023-05-04 19:49:53 +02:00
Ingo Oppermann
d201921a33 Allow to provide complete cluster configuration
Replace CORE_CLUSTER_JOIN_ADDRESS with CORE_CLUSTER_PEERS. This is
a comma separated list of cluster members with their IDs of the form
ID@host:port

On startup the node tries to connect to all the peers. In case of
sudden deaths of a node this will allow to find back into the
cluster. The list in CLUSTER_PEERS is a starting point of known
peers. Other node that are not in that list can still join the
cluster.

File and stream proxy has been moved to the Proxy type.
2023-05-03 16:13:05 +02:00
Ingo Oppermann
4eb0258ba4 Add automatic bootstrap and recovery 2023-05-02 15:49:46 +02:00
Ingo Oppermann
7527f7833b Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
3a1825cf61 Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
386bdc0b6e WIP: add emergency leader, api circuit breaker 2023-04-21 17:19:21 +02:00
Ingo Oppermann
27f19f9188 WIP: internal API 2023-04-17 14:01:20 +02:00
Ingo Oppermann
7643959bf8 WIP: raft leadership 2023-04-13 21:44:24 +02:00
Ingo Oppermann
e6b64dbe97 WIP: raft 2023-04-13 09:42:19 +02:00
Ingo Oppermann
ffdf6d3323 Merge branch 'dev' into cluster 2023-04-12 15:27:38 +02:00
Ingo Oppermann
8326160b28 Add option to set GOMAXPROCS automatically
Use the debug.auto_max_procs config value or the CORE_DEBUG_AUTO_MAX_PROCS
environment variable. Set it to "true" to enable this feature.
2023-04-11 17:12:01 +02:00
Ingo Oppermann
0bd68d6dc3 Fix debug.force_gc behaviour 2023-04-11 11:08:11 +02:00
Ingo Oppermann
507b4f4b9a Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
84817f137a Merge branch 'dev' into iam 2023-04-03 15:44:16 +02:00
Ingo Oppermann
99c571d623 Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b84fdddd81 Add /api/v3/events endpoint 2023-03-23 11:11:47 +01:00
Ingo Oppermann
674c444d17 Merge branch 'dev' into vod 2023-03-17 18:41:16 +01:00
Ingo Oppermann
3149572a64 Fix freeing up S3 mounts 2023-03-17 18:40:20 +01:00
Ingo Oppermann
7e7aadc6cb Allow to bulk delete files based on ListOptions 2023-03-17 15:56:15 +01:00
Ingo Oppermann
4ce8a0eaa3 Add config value for how many minimal process report should be kept in the history
A minimal history is a history entry without log and prelude.

The corresponding config entry is ffmpeg.log.max_minimal_history. This value is
added on top of the ffmpeg.log.max_history value. I.e. the latest max_history
entries contain the log and prelude, and the remaining entries don't have the
log and prelude. In total there are max_minimal_history+max_history history
entries.

If you want no history, set both values to 0.
If you want only full history, set max_minimal_history to 0.
If you want only minimal history, set max_history to 0.
2023-03-16 12:25:06 +01:00
Ingo Oppermann
6e93c1d5a1 Get rid of $localhost pseudo user 2023-03-09 21:10:04 +01:00
Ingo Oppermann
a0ef3ab5ee WIP: add /api/v3/iam/group/* endpoints 2023-03-09 19:03:58 +01:00
Ingo Oppermann
d101a76e9e Change anon user to localhost user only if DisableLocalhost is set 2023-03-08 15:21:55 +01:00
Ingo Oppermann
8755117e92 Add /api/v3/iam/user endpoints 2023-03-07 16:31:58 +01:00