Commit Graph

75 Commits

Author SHA1 Message Date
Ingo Oppermann
89379b2acd Add /v3/cluster/snapshot endpoint 2023-06-27 15:11:31 +02:00
Ingo Oppermann
dd128ac99b Fix gathering of hostnames, add /v3/cluster/db/kv endpoint 2023-06-27 14:52:32 +02:00
Ingo Oppermann
944d487730 Allow to send command to process on the cluster 2023-06-22 21:43:51 +02:00
Ingo Oppermann
dc3e7afc52 Add locks on the cluster DB 2023-06-22 12:22:25 +02:00
Ingo Oppermann
b35ac9ccc3 Allow proxied range requests, /cluster/process mimic /process 2023-06-21 13:14:27 +02:00
Ingo Oppermann
7565427f9a Add GET /v3/iam/user endpoint 2023-06-20 20:27:51 +02:00
Ingo Oppermann
694f51d566 Update API docs 2023-06-20 15:36:40 +02:00
Ingo Oppermann
22fd241b89 Update API docs 2023-06-14 22:37:33 +02:00
Ingo Oppermann
50e943a075 WIP: add session token, missing: writing sessions to log 2023-06-14 12:30:50 +02:00
Ingo Oppermann
82ba3a8f82 WIP: introducing cluster versioning, degraded mode 2023-06-09 21:42:26 +02:00
Ingo Oppermann
1f6f7c9f59 Add API endpoint to leave cluster gracefully, hard leave on exit 2023-06-06 11:22:53 +02:00
Ingo Oppermann
8829b8fff0 Respect domain in cluster DB, allow metadata in process config for cluster 2023-06-05 21:19:11 +02:00
Ingo Oppermann
a79cfa0c77 Merge branch 'clusteriam' into vod 2023-06-01 15:16:54 +02:00
Ingo Oppermann
401156e4d2 Enforce policies 2023-05-31 16:39:43 +02:00
Ingo Oppermann
3a6bb02bfd Fix reloading policies, add API endpoints for observing internal cluster state 2023-05-31 11:23:54 +02:00
Ingo Oppermann
89cc42c4b9 Add cluster IAM, update dependencies 2023-05-30 21:24:12 +02:00
Ingo Oppermann
710d5c595f Merge branch 'iam' into clusteriam 2023-05-25 11:05:26 +02:00
Ingo Oppermann
7a1eb1251b Remove IAM group semantics 2023-05-23 16:05:04 +02:00
Ingo Oppermann
ccac2ffd5d Various updates
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
ead6e64442 Merge branch 'dev' into iam 2023-05-15 21:17:23 +02:00
Ingo Oppermann
6a4c715f75 Fix update process, list process in raft store 2023-05-13 21:21:47 +02:00
Ingo Oppermann
7d2b7b4836 WIP: allow update processes in cluster 2023-05-12 12:59:01 +02:00
Ingo Oppermann
f4015f5cbd Merge branch 'dev' into cluster 2023-05-05 15:56:26 +02:00
Ingo Oppermann
4c3c8d3bd4 Merge branch 'dev' into vod 2023-05-05 12:09:54 +02:00
Ingo Oppermann
ca261a56ee Add looping_runtime to avstream status 2023-05-05 12:03:48 +02:00
Ingo Oppermann
7f59c188cf Remove node storage, use raft configuration instead; re-establish file and stream proxying 2023-05-04 19:49:53 +02:00
Ingo Oppermann
b723f4e5fd Propagate number of logical CPUs in resource usage 2023-04-27 10:58:41 +02:00
Ingo Oppermann
b70382c40b Expose resource usage in process state 2023-04-25 16:20:59 +02:00
Ingo Oppermann
b6a9fa7965 Merge branch 'dev' into cluster 2023-04-13 16:25:53 +02:00
Ingo Oppermann
76abf5474f Fix compilation error 2023-04-13 15:43:59 +02:00
Ingo Oppermann
5d9db55032 Merge branch 'dev' into vod 2023-04-13 15:23:09 +02:00
Ingo Oppermann
d807becc8a Add support for input framerate data from jsonstats patch 2023-04-13 15:22:33 +02:00
Ingo Oppermann
ffdf6d3323 Merge branch 'dev' into cluster 2023-04-12 15:27:38 +02:00
Ingo Oppermann
507b4f4b9a Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
7e9e6fce8d Add number of keyframes and extradata size to process progress data 2023-04-04 20:44:57 +02:00
Ingo Oppermann
84817f137a Merge branch 'dev' into iam 2023-04-03 15:44:16 +02:00
Ingo Oppermann
029bf208e7 Add bandwidth limit to filesystem operations, rename fields 2023-03-30 09:51:29 +02:00
Ingo Oppermann
0a3117bbd0 Add log_patterns to process config
log_patterns allow to filter the FFmpeg log messages based on regular
expressions. Each entry of log_patterns is interpreted as regular
expression and matched against every non-progress log line emitted from
FFmpeg. All matching lines are returned in the matches array of the
report.
2023-03-27 15:50:25 +02:00
Ingo Oppermann
094e666926 Fix swagger documentation for /api/v3/events 2023-03-23 19:46:24 +01:00
Ingo Oppermann
99c571d623 Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b84fdddd81 Add /api/v3/events endpoint 2023-03-23 11:11:47 +01:00
Ingo Oppermann
b21aba5f9d Remove /process/:id/report/:at endpoint, extend /process/:id/report endpoint 2023-03-22 12:31:41 +01:00
Ingo Oppermann
c92f2fd1df Fix JSON name of process timeout 2023-03-21 14:54:33 +01:00
Ingo Oppermann
b63b23527b Add scheduler and timeout to processes
The scheduler allows to define when a process should run. It can be either
a timestamp in RFC3339 format or a crontab expression. If a scheduler is
given, reconnect and the reconnect delay will only apply to processes that
exited as failed.

The timeout allows to define when a process should be gracefully stopped.
It is measured from the actual start of that process including all reconnects
due to failures. If the process finished regularly, the timeout will be
reset.
2023-03-21 14:51:43 +01:00
Ingo Oppermann
a692f88b69 Add DELETE /api/v3/fs/:storage endpoint
Works similar to the GET endpoint. It has the same query parameter (without
"sort" and "order"). A glob pattern has to be provided. To select all
files, use the glob pattern "/**".
2023-03-17 17:05:56 +01:00
Ingo Oppermann
ecfbbe3857 Allow to list files by ranges of size and/or lastmod
The listing options are implemented by the query parameters size_min,
size_max, lastmod_start, and lastmod_end, or by the new ListOptions
type. size_min and size_max expect a number of bytes, lastmod_start
and lastmod_end expect a unix timestamp. All values are inclusive.
2023-03-17 15:15:20 +01:00
Ingo Oppermann
ffaa3d9324 Merge branch 'dev' into vod 2023-03-17 13:56:07 +01:00
Ingo Oppermann
562b7aed92 Use better naming for storage endpoint documentation 2023-03-17 13:55:19 +01:00
Ingo Oppermann
4ce8a0eaa3 Add config value for how many minimal process report should be kept in the history
A minimal history is a history entry without log and prelude.

The corresponding config entry is ffmpeg.log.max_minimal_history. This value is
added on top of the ffmpeg.log.max_history value. I.e. the latest max_history
entries contain the log and prelude, and the remaining entries don't have the
log and prelude. In total there are max_minimal_history+max_history history
entries.

If you want no history, set both values to 0.
If you want only full history, set max_minimal_history to 0.
If you want only minimal history, set max_history to 0.
2023-03-16 12:25:06 +01:00
Ingo Oppermann
a0ef3ab5ee WIP: add /api/v3/iam/group/* endpoints 2023-03-09 19:03:58 +01:00