Commit Graph

51 Commits

Author SHA1 Message Date
Tolsee
2f7b987198 feat: dequeue worker request on timeout (#1540) 2025-05-09 18:01:49 +02:00
Marc
1d74b2caa8 feat: define domain specific workers in php_server and php blocks (#1509)
* add module (php_server directive) based workers

* refactor moduleID to uintptr for faster comparisons

* let workers inherit environment variables and root from php_server

* caddy can shift FrankenPHPModules in memory for some godforsaken reason, can't rely on them staying the same

* remove debugging statement

* fix tests

* refactor moduleID to uint64 for faster comparisons

* actually allow multiple workers per script filename

* remove logging

* utility function

* reuse existing worker with same filename and environment when calling newWorker with a filepath that already has a suitable worker, simply add number of threads

* no cleanup happens between tests, so restore old global worker overwriting logic

* add test, use getWorker(ForContext) function in frankenphp.go as well

* bring error on second global worker with the same filename again

* refactor to using name instead of moduleID

* nicer name

* nicer name

* add more tests

* remove test case already covered by previous test

* revert back to single variable, moduleIDs no longer relevant

* update comment

* figure out the worker to use in FrankenPHPModule::ServeHTTP

* add caddy/config_tests, add --retry 5 to download

* add caddy/config_tests

* sum up logic a bit, put worker thread addition into moduleWorkers parsing, before workers are actually created

* implement suggestions as far as possible

* fixup

* remove tags

* feat: download the mostly static binary when possible (#1467)

* feat: download the mostly static binary when possible

* cs

* docs: remove wildcard matcher from root directive (#1513)

* docs: update README with additional documentation links

Add link to classic mode, efficiently serving large static files and monitoring FrankenPHP

Signed-off-by: Romain Bastide <romain.bastide@orange.com>

* ci: combine dependabot updates for one group to 1 pull-request

* feat: compatibility with libphp.dylib on macOS

* feat: upgrade to Caddy 2.10

* feat: upgrade to Caddy 2.10

* chore: run prettier

* fix: build-static.sh consecutive builds (#1496)

* fix consecutive builds

* use minor version in PHP_VERSION

* install jq in centos container

* fix "arm64" download arch for spc binary

* jq is not available as a rpm download

* linter

* specify php 8.4 default

specify 8.4 so we manually switch to 8.5 when we make sure it works
allows to run without jq installed

* Apply suggestions from code review

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

---------

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* chore: update Go and toolchain version (#1526)

* apply suggestions one be one - scriptpath only

* generate unique worker names by filename and number

* support worker config from embedded apps

* rename back to make sure we don't accidentally add FrankenPHPApp workers to the slice

* fix test after changing error message

* use 🧩 for module workers

* use 🌍 for global workers :)

* revert 1c414cebbc

* revert 4cc8893ced

* apply suggestions

* add dynamic config loading test of module worker

* fix test

* minor changes

---------

Signed-off-by: Romain Bastide <romain.bastide@orange.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
Co-authored-by: Indra Gunawan <hello@indra.my.id>
Co-authored-by: Romain Bastide <romain.bastide@orange.com>
2025-05-05 16:14:19 +02:00
Kévin Dunglas
5b7fc5ec52 chore: make the linter happy (#1537) 2025-05-02 11:43:54 +02:00
Alexander Stecher
8125993001 fix: disallow 2 workers with same filename (#1492)
* Disallows 2 workers with the same filename.

* Adds test.

* Prevent duplicate names.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Alliballibaba <alliballibaba@gmail.com>
2025-04-29 10:18:24 +02:00
Indra Gunawan
87315a19ae feat: introduces worker name option, use label on worker metrics instead (#1376)
* add worker name option and use it in logs and metrics, update tests

* fix missing reference for collector

* update tests

* update docs

* fix conflict

* add missing allowedDirectives

* update tests
2025-03-22 12:32:59 +01:00
Alexander Stecher
9cca12858b feat: maximum wait times (#1445) 2025-03-19 13:21:37 +01:00
Alexander Stecher
f50248a7d2 refactor: removes context on the C side (#1404) 2025-03-10 08:44:03 +01:00
Alexander Stecher
3ba4e257a1 fix: only drain workers on graceful shutdown (#1405)
* Only drains workers on shutdown.

* trigger build

* Marks func as experimental.

---------

Co-authored-by: Alliballibaba <alliballibaba@gmail.com>
2025-02-28 12:10:00 +01:00
Alliballibaba2
072151dfee feat: Adds automatic thread scaling at runtime and php_ini configuration in Caddyfile (#1266)
Adds option to scale threads at runtime

Adds php_ini configuration in Caddyfile
2025-02-19 20:39:33 +01:00
Alliballibaba2
f592e0f47b refactor: decouple worker threads from non-worker threads (#1137)
* Decouple workers.

* Moves code to separate file.

* Cleans up the exponential backoff.

* Initial working implementation.

* Refactors php threads to take callbacks.

* Cleanup.

* Cleanup.

* Cleanup.

* Cleanup.

* Adjusts watcher logic.

* Adjusts the watcher logic.

* Fix opcache_reset race condition.

* Fixing merge conflicts and formatting.

* Prevents overlapping of TSRM reservation and script execution.

* Adjustments as suggested by @dunglas.

* Adds error assertions.

* Adds comments.

* Removes logs and explicitly compares to C.false.

* Resets check.

* Adds cast for safety.

* Fixes waitgroup overflow.

* Resolves waitgroup race condition on startup.

* Moves worker request logic to worker.go.

* Removes defer.

* Removes call from go to c.

* Fixes merge conflict.

* Adds fibers test back in.

* Refactors new thread loop approach.

* Removes redundant check.

* Adds compareAndSwap.

* Refactor: removes global waitgroups and uses a 'thread state' abstraction instead.

* Removes unnecessary method.

* Updates comment.

* Removes unnecessary booleans.

* test

* First state machine steps.

* Splits threads.

* Minimal working implementation with broken tests.

* Fixes tests.

* Refactoring.

* Fixes merge conflicts.

* Formatting

* C formatting.

* More cleanup.

* Allows for clean state transitions.

* Adds state tests.

* Adds support for thread transitioning.

* Fixes the testdata path.

* Formatting.

* Allows transitioning back to inactive state.

* Fixes go linting.

* Formatting.

* Removes duplication.

* Applies suggestions by @dunglas

* Removes redundant check.

* Locks the handler on restart.

* Removes unnecessary log.

* Changes Unpin() logic as suggested by @withinboredom

* Adds suggestions by @dunglas and resolves TODO.

* Makes restarts fully safe.

* Will make the initial startup fail even if the watcher is enabled (as is currently the case)

* Also adds compareAndSwap to the test.

* Adds comment.

* Prevents panic on initial watcher startup.
2024-12-17 11:28:51 +01:00
Rob Landers
1e279bc348 refactor: simplify exponential backoff and refactor env (#1185)
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-11-23 11:29:56 +01:00
Rob Landers
08e99fc85f fix(metrics): handle the case where the worker is already assigned to a thread (#1171) 2024-11-21 13:23:41 +01:00
Kévin Dunglas
2d6a299dbc perf: improve php_server directive (#1180) 2024-11-21 13:22:24 +01:00
Rob Landers
0c123a2563 remove opcache_reset (#1173)
* remove opcache_reset

* reset opcache if the function exists

* simplify the check

* reformat
2024-11-17 19:19:53 +01:00
Kévin Dunglas
843d199469 perf: cache computations in WithRequestDocumentRoot (#1154) 2024-11-13 07:10:53 +01:00
Kévin Dunglas
75dab8f33d chore: bump deps and misc improvements (#1135) 2024-11-04 16:42:15 +01:00
Alexander Stecher
1c3ce114f6 perf: use hot worker threads when possible (#1126)
Co-authored-by: Alliballibaba <alliballibaba@gmail.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-11-04 16:18:44 +01:00
Alexander Stecher
e5ca97308e perf: optimize $_SERVER import (#1106)
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Alliballibaba <alliballibaba@gmail.com>
2024-11-04 15:34:00 +01:00
Kévin Dunglas
d53f909d20 chore: various cleanups 2024-10-23 22:33:58 +02:00
Rob Landers
e812473fe1 implement getenv and putenv in go (#1086)
* implement getenv and putenv in go

* fix typo

* apply formatting

* return a bool

* prevent ENV= from crashing

* optimization

* optimization

* split env workflows and use go_strings

* clean up unused code

* update tests

* remove useless sprintf

* see if this fixes the asan issues

* clean up comments

* check that VAR= works correctly and use actual php to validate the behavior

* move all unpinning to the end of the request

* handle the case where php is not installed

* fix copy-paste

* optimization

* use strings.cut

* fix lint

* override how env is filled

* reuse fullenv

* use corect function
2024-10-18 13:47:11 +02:00
Alexander Stecher
8bbd16d585 Removes worker panic when the watcher is enabled. (#1092)
* Removes the worker panic when the watcher is enabled.

* Only panics on the initial boot.

* Only panics on the initial boot.

* Resets to always panic when watcher is disabled.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
2024-10-14 20:58:26 +02:00
Alexander Stecher
d99b16a158 perf: remove all cgo handles (#1073)
* Removes Cgo handles and adds phpThreads.

* Changes variable name.

* Changes variable name.

* Fixes merge error.

* Removes unnecessary 'workers are done' check.

* Removes panic.

* Replaces int with uint32_t.

* Changes index type to uintptr_t.

* simplify

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-09 07:31:09 +02:00
Kévin Dunglas
e9c075a4a5 feat: add build tag to skip Watcher support (#1076)
* feat: add build tag to skip Watcher support

* fix

* fix

* cleanup
2024-10-08 23:23:53 +02:00
Kévin Dunglas
56d2f99548 chore: make the watcher module internal 2024-10-07 15:37:40 +02:00
Alexander Stecher
8d9b6e755b feat: restart workers when on source changes (#1013)
* Adds filesystem watcher with tests.

* Refactoring.

* Formatting.

* Formatting.

* Switches to absolute path in tests.

* Fixes race condition from merge conflict.

* Fixes race condition.

* Fixes tests.

* Fixes markdown lint errors.

* Switches back to absolute paths.

* Reverts back to relative file paths.

* Fixes golangci-lint issues.

* Uses github.com/dunglas/go-fswatch instead.

* Stops watcher before stopping workers.

* Updates docs.

* Avoids segfault in tests.

* Fixes watcher segmentation violations on shutdown.

* Adjusts watcher latencies and tests.

* Adds fswatch to dockerfiles

* Fixes fswatch in alpine.

* Fixes segfault (this time for real).

* Allows queueing new reload if file changes while workers are reloading.

* Makes tests more consistent.

* Prevents the watcher from getting stuck if there is an error in the worker file itself.

* Reverts changing the image.

* Puts fswatch version into docker-bake.hcl.

* Asserts instead of panicking.

* Adds notice

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Runs fswatch version.

* Removes .json.

* Replaces ms with s.

* Resets the channel after closing it.

* Update watcher_options.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Asserts no error instead.

* Fixes a race condition where events are fired after frankenphp has stopped.

* Updates docs.

* Update watcher_options_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Allows queuing events while watchers are reloading.

* go fmt

* Refactors stopping and draining logic.

* Allows extended watcher configuration with dirs, recursion, symlinks, case-sensitivity, latency, monitor types and regex.

* Updates docs.

* Adds TODOS.

* go fmt.

* Fixes linting errors.

* Also allows wildcards in the longform and adjusts docs.

* Adds debug log.

* Fixes the watcher short form.

* Refactors sessions and options into a struct.

* Fixes an overflow in the 'workersReadyWG' on unexpected terminations.

* Properly logs errors coming from session.Start().

* go fmt.

* Adds --nocache.

* Fixes lint issue.

* Refactors and resolves race condition on worker reload.

* Implements debouncing with a timer as suggested by @withinboredom.

* Starts watcher even if no workers are defined.

* Updates docs with file limit warning.

* Adds watch config unit tests.

* Adjusts debounce timings.

* go fmt.

* Adds fswatch to static builder (test).

* Adds a short grace period between stopping and destroying the watcher sessions.

* Adds caddy test.

* Adjusts sleep time.

* Swap to edant/watcher.

* Fixes watch options and tests.

* go fmt.

* Adds TODO.

* Installs edant/watcher in the bookworm image.

* Fixes linting.

* Refactors the watcher into its own module.

* Adjusts naming.

* ADocker image adjustments and refactoring.

* Testing installation methods.

* Installs via gcc instead.

* Fixes pointer formats.

* Fixes lint issues.

* Fixes arm alpine and updates docs.

* Clang format.

* Fixes dirs.

* Adds watcher version arg.

* Uses static lib version.

* Adds watcher to tests and sanitizers.

* Uses sudo for copying the shared lib.

* Removes unnused func.

* Refactoring.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Adds fpic.

* Fixes linting.

* Skips tests in msan.

* Resets op_cache in every worker thread after termination

* Review fixes part 1.

* Test: installing libstc++ instead of gcc.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Allows '/**/' for global recursion and '**/' for relative recursion.

* Reverts using the ignorelist.

* Calls opcache directly.

* Adds --watch to php-server command

* Properly free CStrings.

* Sorts alphabetically and uses curl instead of git.

* Labeling and formatting.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update caddy/caddy.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update testdata/files/.gitignore

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher-c.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Fixes test and Dockerfile.

* Fixes Dockerfiles.

* Resets go versions.

* Replaces unsafe.pointer with uintptr_t

* Prevents worker channels from being destroyed on reload.

* Minimizes the public api by only passing a []string.

* Adds support for directory patterns and multiple '**' globs.

* Adjusts label.

* go fmt.

* go mod tidy.

* Fixes merge conflict.

* Refactoring and formatting.

* Cleans up unused vars and functions.

* Allows dirs with a dot.

* Makes test nicer.

* Add dir tests.

* Moves the watch directive inside the worker directive.

* Adds debug log on special events.

* Removes line about symlinks.

* Hints at multiple possible --watch flags.

* Adds ./**/*.php as default watch configuration.

* Changes error to a warning.

* Changes the default to './**/*.{php,yaml,yml,twig,env}' and supports the {bracket} pattern.

* Fixes linting.

* Fixes merge conflict and adjust values.

* Adjusts values.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-07 13:17:24 +02:00
Rob Landers
aa585f7da0 handle worker failures gracefully (#1038)
* handle failures gracefully

* fix super-subtle race condition

* address feedback: panic instead of fatal log and make vars into consts

* pass the frankenphp context to worker-ready function

* reset backoff and failures on normal restart

* update docs

* add test and fix race condition

* fail sometimes but do not be pathological about it

* Use title case

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* fix code style in php

* define lifecycle metrics

* ensure we update unregister the metrics and fix tests

* update caddy tests and fix typo

* update docs

* no need for this

---------

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-03 21:53:12 +02:00
Rob Landers
5d43fc2c8d add basic metrics (#966)
* add metrics

* change how counting works

* also replace dots

* check that metrics exist

* rename NullMetrics to nullMetrics

* update go.sum

* register collectors only once

* add tests

* add tests for metrics and fix bugs

* keep old metrics around for test

* properly reset during shutdown

* use the same method as frankenphp

* Revert "keep old metrics around for test"

This reverts commit 1f0df6f6bdaebf32aec346f068d6f42a0b5f4007.

* change to require.NoError

* compile regex only once

* remove name sanitizer

* use require

* parameterize host port because security software sucks

* remove need for renaming workers

* increase number of threads and add tests

* fix where frankenphp configuration was bleeding into later tests

* adds basic docs for metrics

* Add caddy metrics link

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Fix typos

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* address feedback

* change comment to be much more "dangerous"

---------

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-09-26 09:53:37 +02:00
Kévin Dunglas
a16076e082 perf: prevent useless logger allocs 2024-08-27 11:03:01 +02:00
Kévin Dunglas
90a7b98b10 feat: log the number of threads and workers during startup 2024-05-30 11:15:30 +02:00
Kévin Dunglas
3d065eda35 $_SERVER['FRANKENPHP_WORKER'] must not be NULL-terminated 2024-05-21 18:50:02 +02:00
Kévin Dunglas
07a74e5c5a perf: reduce allocs when creating $_SERVER (#540)
* perf: reduce allocs when creating $_SERVER

* improve

* refactor: prevent C allocs when populating $_SERVER

* cs

* remove append()

* simplify

* wip

* cleanup

* add cache

* cleanup otter init

* some fixes

* cleanup

* test with a leak

* remove const?

* add const

* wip

* wip

* allocate dynamic variables in Go memory

* cleanup

* typo

* bump otter

* chore: bump deps
2024-03-12 18:31:30 +01:00
Rob Landers
7830aae549 Ensure we don't modify a shared env (#452)
* Ensure we don't modify a shared env

* Update worker.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update worker.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

---------

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2023-12-31 20:02:11 +01:00
Rob Landers
5bda50cbd7 Fix memory leak (#442)
* do not use caddy context

* ensure all handles are cleaned up

* do not export types

* just panic when double deleting a handle

* set the minimal capacity

* remove micro-opt

* move handle cleanup to just before we return from serveHttp

* ensure we clean up cli scripts

* handle cgi-mode free

* micro-optimization: set initial capacity
2023-12-28 00:16:19 +01:00
Kévin Dunglas
b4aa8038ff feat: detect when worker crashes or terminates normally (#315) 2023-11-21 23:15:07 +01:00
Kévin Dunglas
aa1d968dcf refactor: faster $_SERVER variables creation 2023-11-16 14:40:52 +01:00
Kévin Dunglas
fb63099a88 feat: allow passing env vars to workers (#210) 2023-09-15 12:59:43 +02:00
Kévin Dunglas
8149f191e0 chore: clean multiple errors handling 2023-08-12 21:26:16 +02:00
Kévin Dunglas
307c5fa865 fix: properly cleanup workersReadyWG (#184) 2023-08-12 12:39:13 +02:00
Kévin Dunglas
df976c1708 fix: race condition on shutdown 2023-08-06 17:37:52 +02:00
Kévin Dunglas
75cd310292 feat: add support for PHP timeouts on Linux (#128) 2023-03-05 15:47:20 +01:00
Kévin Dunglas
8e824a7906 perf: remove an extra call to cgo (#117) 2022-11-28 18:06:37 +01:00
Rob Landers
9ef3bd7c47 feat: add fastcgi_finish_request() support (#69)
* Add function to ext

* buggy af, but kinda working

* Get worker and regular mode working

* fix formatting

* add eol

* add eol

* 🤦

* properly generate file

* use sync.Once
2022-11-03 08:36:47 +01:00
Kévin Dunglas
26edeb76e4 fix: prevent panic when restarting crashed worker (#49) 2022-10-21 14:43:16 +02:00
Kévin Dunglas
7ec0043fe8 feat: handle errors in worker mode (#9)
* feat: handle errors in worker mode (not working)

* feat: better exception and error handling
2022-10-11 08:10:46 +02:00
Kévin Dunglas
c45a4c620f feat: improve sessions handling in workers 2022-10-07 16:59:27 +02:00
Kévin Dunglas
4a87ad3609 chore: add tests regarding persistent objects 2022-10-05 15:44:02 +02:00
Kévin Dunglas
d61c96a4c3 feat: use Zap for logging 2022-10-04 17:42:45 +02:00
Kévin Dunglas
796476d537 feat: use threads instead of GoRoutines (#6)
* feat: use threads instead of GoRoutines

* many improvements

* fix some bugs
2022-10-04 14:41:19 +02:00
Kévin Dunglas
84273ec395 fix: fix thread safety issues (#5)
* fix: fix thread safety issues

* some improvements

* wip

* tiny improvements
2022-06-03 17:18:07 +02:00
Kévin Dunglas
09de805aaa fix: memory leaks and refactor worker code (#2) 2022-05-18 21:19:07 +02:00