Commit Graph

59 Commits

Author SHA1 Message Date
Ingo Oppermann
2ff8ce2c44 Simplify sending signals. Closing stdout on kill 2025-07-08 11:58:30 +02:00
Ingo Oppermann
93ba370c3a Close stdout pipe if process stays too long in finished state 2025-07-07 16:57:19 +02:00
Ingo Oppermann
15a0f4dbc5 Expose process ID 2025-07-07 12:09:20 +02:00
Ingo Oppermann
55314e35a8 Add option to disable CPU throttling 2025-06-24 16:52:51 +02:00
Ingo Oppermann
f445d0c992 Add tee progress parsing 2025-06-19 16:11:30 +02:00
Ingo Oppermann
ba77ec0ecf Fix crash while stopping a process that is currently starting 2025-04-10 15:30:40 +02:00
Ingo Oppermann
e359420838 Make reconnect less prone to race conditions 2025-04-09 14:03:00 +02:00
Ingo Oppermann
dfbf55883d Remove commented out line 2024-10-31 22:01:28 +01:00
Ingo Oppermann
d591a2383e Fix GPU index numbering, promote the GPU ID 2024-10-31 14:59:22 +01:00
Ingo Oppermann
2393dbc4c0 Add replacer tests, fix command in state with dynamic placeholders 2024-10-29 16:43:47 +01:00
Ingo Oppermann
de9a30a108 Add internal mock modules 2024-10-29 14:55:55 +01:00
Ingo Oppermann
2ee7fa7e41 Make resources the only direct user of psutil 2024-10-29 12:25:39 +01:00
Ingo Oppermann
fbf62bf7e5 Remove Start() function, rename Stop() to Cancel() 2024-10-28 17:12:31 +01:00
Ingo Oppermann
412fbedea3 Make psutil a submodule of resources, remove default psutil 2024-10-28 16:13:13 +01:00
Ingo Oppermann
b6daea1a02 Remove unused parameter 2024-10-24 15:52:56 +02:00
Ingo Oppermann
2dbe5b5685 Add GPU support 2024-10-24 15:08:26 +02:00
Ingo Oppermann
1c56d53a6b Adjust comments 2024-08-21 17:02:22 +02:00
Ingo Oppermann
70a49f8bdb Process []byte instread of string in parser 2024-07-26 11:31:47 +02:00
Ingo Oppermann
28e1325eb2 Prevent sending RESUME if process is already resumed 2024-07-23 16:07:48 +02:00
Ingo Oppermann
72883d18d4 Remove bottlenecks in process handling, still some rough edges 2024-07-18 17:16:49 +02:00
Ingo Oppermann
b851249b98 Adjust process state if parser doesn't indicate that the process is producing output 2024-04-15 16:43:34 +02:00
Ingo Oppermann
88605a1f1e Put child processes in their own session
The core can be gracefully shutdown by sending a SIGINT to its process.
However, this signal is also propagated to all child processes, i.e. the
forked ffmpeg processes. They will also be stopped and might reconnect.
This is not wanted. The core has to stop these processes.

The child process will now get their own session ID with setsid() before
replacing themselves with ffmpeg. This way they will not receive a SIGINT
that was meant only for the parent.
2023-12-12 21:29:12 +01:00
Ingo Oppermann
8a8ff6d4f4 Add LimitMode and Resources.CPU.IsThrottling to process state 2023-07-17 21:45:19 +02:00
Ingo Oppermann
c8ab8567d9 Return error if onBeforeStart fails 2023-07-12 20:56:03 +02:00
Ingo Oppermann
72d56be9af Write stop/kill reason to process parser 2023-07-12 10:48:33 +02:00
Ingo Oppermann
d652fd213b Check for identical configs on process update 2023-06-01 16:43:17 +02:00
Ingo Oppermann
4a4d33e529 Merge branch 'dev' into vod 2023-05-24 16:33:29 +02:00
Ingo Oppermann
4b2b6a57a1 Fix calling Wait after process has been read 2023-05-24 16:27:55 +02:00
Ingo Oppermann
9545525e46 Extend process reconnect test 2023-05-02 20:53:49 +02:00
Ingo Oppermann
06d7c6d0d4 Isolate order lock, call onExit later 2023-05-02 11:01:31 +02:00
Ingo Oppermann
7cca02f2b5 Add tests 2023-05-01 21:42:01 +02:00
Ingo Oppermann
c23722416b Fix test 2023-05-01 16:39:47 +02:00
Ingo Oppermann
ef138fb90f Limit CPU and memory independently, release CPU throttling incremently 2023-05-01 16:29:18 +02:00
Ingo Oppermann
b7e0afc7a7 Fix tests 2023-04-29 08:26:22 +02:00
Ingo Oppermann
748bccc3d5 WIP: gradually decrease limiting resources 2023-04-29 07:49:37 +02:00
Ingo Oppermann
f6d1cbbec2 WIP: release CPU throttling stepwise 2023-04-28 16:46:59 +02:00
Ingo Oppermann
74f98e8c7e Define system CPU resources in percent 0-100, acquiring resources check with available system resources 2023-04-28 10:27:33 +02:00
Ingo Oppermann
7527f7833b Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
b723f4e5fd Propagate number of logical CPUs in resource usage 2023-04-27 10:58:41 +02:00
Ingo Oppermann
c9bdaae7f7 Call onExit if cmd.Start() fails 2023-04-26 22:17:08 +02:00
Ingo Oppermann
d59158de03 Allow hard and soft limiting a process
A hard limit will kill the process as soon as either CPU or memory
consumption are above a defined limit for a certain amount of time.

A soft limit will throttle the CPU usage if above a defined limit and
kill the process if memory consumption is above a defined limit. The
soft limit can be enabled/disabled on demand.

The default is hard limit.
2023-04-26 16:01:50 +02:00
Ingo Oppermann
a2dab2682f Fix not propagating process limits 2023-04-26 09:49:28 +02:00
Ingo Oppermann
9b17ab2b29 Add global limits in config, fix not using process limits 2023-04-25 21:49:55 +02:00
Ingo Oppermann
74f582e4b4 Remove debug output 2023-04-25 16:25:21 +02:00
Ingo Oppermann
9b6354ab94 Revert commit b58cc8a7ee 2023-04-25 15:57:17 +02:00
Ingo Oppermann
3a1825cf61 Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
3e7e5d5c9c Merge branch 'dev' into vod 2023-04-24 16:14:32 +02:00
Ingo Oppermann
b58cc8a7ee Fix race condition 2023-04-24 16:09:01 +02:00
Ingo Oppermann
b63b23527b Add scheduler and timeout to processes
The scheduler allows to define when a process should run. It can be either
a timestamp in RFC3339 format or a crontab expression. If a scheduler is
given, reconnect and the reconnect delay will only apply to processes that
exited as failed.

The timeout allows to define when a process should be gracefully stopped.
It is measured from the actual start of that process including all reconnects
due to failures. If the process finished regularly, the timeout will be
reset.
2023-03-21 14:51:43 +01:00
Ingo Oppermann
cceb39192a Fix receiving proper current command args, simplifying retrieval of last log line 2023-03-01 20:13:49 +01:00