Files
frankenphp/caddy/caddy_test.go
Alexander Stecher 8d9b6e755b feat: restart workers when on source changes (#1013)
* Adds filesystem watcher with tests.

* Refactoring.

* Formatting.

* Formatting.

* Switches to absolute path in tests.

* Fixes race condition from merge conflict.

* Fixes race condition.

* Fixes tests.

* Fixes markdown lint errors.

* Switches back to absolute paths.

* Reverts back to relative file paths.

* Fixes golangci-lint issues.

* Uses github.com/dunglas/go-fswatch instead.

* Stops watcher before stopping workers.

* Updates docs.

* Avoids segfault in tests.

* Fixes watcher segmentation violations on shutdown.

* Adjusts watcher latencies and tests.

* Adds fswatch to dockerfiles

* Fixes fswatch in alpine.

* Fixes segfault (this time for real).

* Allows queueing new reload if file changes while workers are reloading.

* Makes tests more consistent.

* Prevents the watcher from getting stuck if there is an error in the worker file itself.

* Reverts changing the image.

* Puts fswatch version into docker-bake.hcl.

* Asserts instead of panicking.

* Adds notice

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Runs fswatch version.

* Removes .json.

* Replaces ms with s.

* Resets the channel after closing it.

* Update watcher_options.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Asserts no error instead.

* Fixes a race condition where events are fired after frankenphp has stopped.

* Updates docs.

* Update watcher_options_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Allows queuing events while watchers are reloading.

* go fmt

* Refactors stopping and draining logic.

* Allows extended watcher configuration with dirs, recursion, symlinks, case-sensitivity, latency, monitor types and regex.

* Updates docs.

* Adds TODOS.

* go fmt.

* Fixes linting errors.

* Also allows wildcards in the longform and adjusts docs.

* Adds debug log.

* Fixes the watcher short form.

* Refactors sessions and options into a struct.

* Fixes an overflow in the 'workersReadyWG' on unexpected terminations.

* Properly logs errors coming from session.Start().

* go fmt.

* Adds --nocache.

* Fixes lint issue.

* Refactors and resolves race condition on worker reload.

* Implements debouncing with a timer as suggested by @withinboredom.

* Starts watcher even if no workers are defined.

* Updates docs with file limit warning.

* Adds watch config unit tests.

* Adjusts debounce timings.

* go fmt.

* Adds fswatch to static builder (test).

* Adds a short grace period between stopping and destroying the watcher sessions.

* Adds caddy test.

* Adjusts sleep time.

* Swap to edant/watcher.

* Fixes watch options and tests.

* go fmt.

* Adds TODO.

* Installs edant/watcher in the bookworm image.

* Fixes linting.

* Refactors the watcher into its own module.

* Adjusts naming.

* ADocker image adjustments and refactoring.

* Testing installation methods.

* Installs via gcc instead.

* Fixes pointer formats.

* Fixes lint issues.

* Fixes arm alpine and updates docs.

* Clang format.

* Fixes dirs.

* Adds watcher version arg.

* Uses static lib version.

* Adds watcher to tests and sanitizers.

* Uses sudo for copying the shared lib.

* Removes unnused func.

* Refactoring.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Adds fpic.

* Fixes linting.

* Skips tests in msan.

* Resets op_cache in every worker thread after termination

* Review fixes part 1.

* Test: installing libstc++ instead of gcc.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Allows '/**/' for global recursion and '**/' for relative recursion.

* Reverts using the ignorelist.

* Calls opcache directly.

* Adds --watch to php-server command

* Properly free CStrings.

* Sorts alphabetically and uses curl instead of git.

* Labeling and formatting.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update caddy/caddy.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update testdata/files/.gitignore

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher-c.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Fixes test and Dockerfile.

* Fixes Dockerfiles.

* Resets go versions.

* Replaces unsafe.pointer with uintptr_t

* Prevents worker channels from being destroyed on reload.

* Minimizes the public api by only passing a []string.

* Adds support for directory patterns and multiple '**' globs.

* Adjusts label.

* go fmt.

* go mod tidy.

* Fixes merge conflict.

* Refactoring and formatting.

* Cleans up unused vars and functions.

* Allows dirs with a dot.

* Makes test nicer.

* Add dir tests.

* Moves the watch directive inside the worker directive.

* Adds debug log on special events.

* Removes line about symlinks.

* Hints at multiple possible --watch flags.

* Adds ./**/*.php as default watch configuration.

* Changes error to a warning.

* Changes the default to './**/*.{php,yaml,yml,twig,env}' and supports the {bracket} pattern.

* Fixes linting.

* Fixes merge conflict and adjust values.

* Adjusts values.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-07 13:17:24 +02:00

605 lines
14 KiB
Go

package caddy_test
import (
"bytes"
"fmt"
"github.com/dunglas/frankenphp"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/testutil"
"github.com/stretchr/testify/require"
"net/http"
"strings"
"sync"
"testing"
"github.com/caddyserver/caddy/v2/caddytest"
)
var testPort = "9080"
func TestPHP(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
for i := 0; i < 100; i++ {
wg.Add(1)
go func(i int) {
tester.AssertGetResponse(fmt.Sprintf("http://localhost:"+testPort+"/index.php?i=%d", i), http.StatusOK, fmt.Sprintf("I am by birth a Genevese (%d)", i))
wg.Done()
}(i)
}
wg.Wait()
}
func TestLargeRequest(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
tester.AssertPostResponseBody(
"http://localhost:"+testPort+"/large-request.php",
[]string{},
bytes.NewBufferString(strings.Repeat("f", 1_048_576)),
http.StatusOK,
"Request body size: 1048576 (unknown)",
)
}
func TestWorker(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp {
worker ../testdata/index.php 2
}
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
for i := 0; i < 100; i++ {
wg.Add(1)
go func(i int) {
tester.AssertGetResponse(fmt.Sprintf("http://localhost:"+testPort+"/index.php?i=%d", i), http.StatusOK, fmt.Sprintf("I am by birth a Genevese (%d)", i))
wg.Done()
}(i)
}
wg.Wait()
}
func TestEnv(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp {
worker {
file ../testdata/worker-env.php
num 1
env FOO bar
}
}
}
localhost:`+testPort+` {
route {
php {
root ../testdata
env FOO baz
}
}
}
`, "caddyfile")
tester.AssertGetResponse("http://localhost:"+testPort+"/worker-env.php", http.StatusOK, "bazbar")
}
func TestJsonEnv(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
"admin": {
"listen": "localhost:2999"
},
"apps": {
"frankenphp": {
"workers": [
{
"env": {
"FOO": "bar"
},
"file_name": "../testdata/worker-env.php",
"num": 1
}
]
},
"http": {
"http_port": `+testPort+`,
"https_port": 9443,
"servers": {
"srv0": {
"listen": [
":`+testPort+`"
],
"routes": [
{
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"env": {
"FOO": "baz"
},
"handler": "php",
"root": "../testdata"
}
]
}
]
}
]
}
]
}
],
"match": [
{
"host": [
"localhost"
]
}
],
"terminal": true
}
]
}
}
},
"pki": {
"certificate_authorities": {
"local": {
"install_trust": false
}
}
}
}
}
`, "json")
tester.AssertGetResponse("http://localhost:"+testPort+"/worker-env.php", http.StatusOK, "bazbar")
}
func TestCustomCaddyVariablesInEnv(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp {
worker {
file ../testdata/worker-env.php
num 1
env FOO world
}
}
}
localhost:`+testPort+` {
route {
map 1 {my_customvar} {
default "hello "
}
php {
root ../testdata
env FOO {my_customvar}
}
}
}
`, "caddyfile")
tester.AssertGetResponse("http://localhost:"+testPort+"/worker-env.php", http.StatusOK, "hello world")
}
func TestPHPServerDirective(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp
}
localhost:`+testPort+` {
root * ../testdata
php_server
}
`, "caddyfile")
tester.AssertGetResponse("http://localhost:"+testPort, http.StatusOK, "I am by birth a Genevese (i not set)")
tester.AssertGetResponse("http://localhost:"+testPort+"/hello.txt", http.StatusOK, "Hello")
tester.AssertGetResponse("http://localhost:"+testPort+"/not-found.txt", http.StatusOK, "I am by birth a Genevese (i not set)")
}
func TestPHPServerDirectiveDisableFileServer(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp
order php_server before respond
}
localhost:`+testPort+` {
root * ../testdata
php_server {
file_server off
}
respond "Not found" 404
}
`, "caddyfile")
tester.AssertGetResponse("http://localhost:"+testPort, http.StatusOK, "I am by birth a Genevese (i not set)")
tester.AssertGetResponse("http://localhost:"+testPort+"/hello.txt", http.StatusNotFound, "Not found")
}
func TestMetrics(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
// Make some requests
for i := 0; i < 10; i++ {
wg.Add(1)
go func(i int) {
tester.AssertGetResponse(fmt.Sprintf("http://localhost:"+testPort+"/index.php?i=%d", i), http.StatusOK, fmt.Sprintf("I am by birth a Genevese (%d)", i))
wg.Done()
}(i)
}
wg.Wait()
// Fetch metrics
resp, err := http.Get("http://localhost:2999/metrics")
if err != nil {
t.Fatalf("failed to fetch metrics: %v", err)
}
defer resp.Body.Close()
// Read and parse metrics
metrics := new(bytes.Buffer)
_, err = metrics.ReadFrom(resp.Body)
if err != nil {
t.Fatalf("failed to read metrics: %v", err)
}
cpus := fmt.Sprintf("%d", frankenphp.MaxThreads)
// Check metrics
expectedMetrics := `
# HELP frankenphp_total_threads Total number of PHP threads
# TYPE frankenphp_total_threads counter
frankenphp_total_threads ` + cpus + `
# HELP frankenphp_busy_threads Number of busy PHP threads
# TYPE frankenphp_busy_threads gauge
frankenphp_busy_threads 0
`
require.NoError(t, testutil.GatherAndCompare(prometheus.DefaultGatherer, strings.NewReader(expectedMetrics), "frankenphp_total_threads", "frankenphp_busy_threads"))
}
func TestWorkerMetrics(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp {
worker ../testdata/index.php 2
}
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
// Make some requests
for i := 0; i < 10; i++ {
wg.Add(1)
go func(i int) {
tester.AssertGetResponse(fmt.Sprintf("http://localhost:"+testPort+"/index.php?i=%d", i), http.StatusOK, fmt.Sprintf("I am by birth a Genevese (%d)", i))
wg.Done()
}(i)
}
wg.Wait()
// Fetch metrics
resp, err := http.Get("http://localhost:2999/metrics")
if err != nil {
t.Fatalf("failed to fetch metrics: %v", err)
}
defer resp.Body.Close()
// Read and parse metrics
metrics := new(bytes.Buffer)
_, err = metrics.ReadFrom(resp.Body)
if err != nil {
t.Fatalf("failed to read metrics: %v", err)
}
cpus := fmt.Sprintf("%d", frankenphp.MaxThreads)
// Check metrics
expectedMetrics := `
# HELP frankenphp_total_threads Total number of PHP threads
# TYPE frankenphp_total_threads counter
frankenphp_total_threads ` + cpus + `
# HELP frankenphp_busy_threads Number of busy PHP threads
# TYPE frankenphp_busy_threads gauge
frankenphp_busy_threads 2
# HELP frankenphp_testdata_index_php_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_testdata_index_php_busy_workers gauge
frankenphp_testdata_index_php_busy_workers 0
# HELP frankenphp_testdata_index_php_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_testdata_index_php_total_workers gauge
frankenphp_testdata_index_php_total_workers 2
# HELP frankenphp_testdata_index_php_worker_request_count
# TYPE frankenphp_testdata_index_php_worker_request_count counter
frankenphp_testdata_index_php_worker_request_count 10
# HELP frankenphp_testdata_index_php_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# TYPE frankenphp_testdata_index_php_ready_workers gauge
frankenphp_testdata_index_php_ready_workers 2
# HELP frankenphp_testdata_index_php_worker_crashes Number of PHP worker crashes for this worker
# TYPE frankenphp_testdata_index_php_worker_crashes counter
frankenphp_testdata_index_php_worker_crashes 0
# HELP frankenphp_testdata_index_php_worker_restarts Number of PHP worker restarts for this worker
# TYPE frankenphp_testdata_index_php_worker_restarts counter
frankenphp_testdata_index_php_worker_restarts 0
`
require.NoError(t,
testutil.GatherAndCompare(
prometheus.DefaultGatherer,
strings.NewReader(expectedMetrics),
"frankenphp_total_threads",
"frankenphp_busy_threads",
"frankenphp_testdata_index_php_busy_workers",
"frankenphp_testdata_index_php_total_workers",
"frankenphp_testdata_index_php_worker_request_count",
"frankenphp_testdata_index_php_worker_crashes",
"frankenphp_testdata_index_php_worker_restarts",
"frankenphp_testdata_index_php_ready_workers",
))
}
func TestAutoWorkerConfig(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443
frankenphp {
worker ../testdata/index.php
}
}
localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")
// Make some requests
for i := 0; i < 10; i++ {
wg.Add(1)
go func(i int) {
tester.AssertGetResponse(fmt.Sprintf("http://localhost:"+testPort+"/index.php?i=%d", i), http.StatusOK, fmt.Sprintf("I am by birth a Genevese (%d)", i))
wg.Done()
}(i)
}
wg.Wait()
// Fetch metrics
resp, err := http.Get("http://localhost:2999/metrics")
if err != nil {
t.Fatalf("failed to fetch metrics: %v", err)
}
defer resp.Body.Close()
// Read and parse metrics
metrics := new(bytes.Buffer)
_, err = metrics.ReadFrom(resp.Body)
if err != nil {
t.Fatalf("failed to read metrics: %v", err)
}
cpus := fmt.Sprintf("%d", frankenphp.MaxThreads)
workers := fmt.Sprintf("%d", frankenphp.MaxThreads-1)
// Check metrics
expectedMetrics := `
# HELP frankenphp_total_threads Total number of PHP threads
# TYPE frankenphp_total_threads counter
frankenphp_total_threads ` + cpus + `
# HELP frankenphp_busy_threads Number of busy PHP threads
# TYPE frankenphp_busy_threads gauge
frankenphp_busy_threads ` + workers + `
# HELP frankenphp_testdata_index_php_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_testdata_index_php_busy_workers gauge
frankenphp_testdata_index_php_busy_workers 0
# HELP frankenphp_testdata_index_php_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_testdata_index_php_total_workers gauge
frankenphp_testdata_index_php_total_workers ` + workers + `
# HELP frankenphp_testdata_index_php_worker_request_count
# TYPE frankenphp_testdata_index_php_worker_request_count counter
frankenphp_testdata_index_php_worker_request_count 10
# HELP frankenphp_testdata_index_php_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# TYPE frankenphp_testdata_index_php_ready_workers gauge
frankenphp_testdata_index_php_ready_workers ` + workers + `
# HELP frankenphp_testdata_index_php_worker_crashes Number of PHP worker crashes for this worker
# TYPE frankenphp_testdata_index_php_worker_crashes counter
frankenphp_testdata_index_php_worker_crashes 0
# HELP frankenphp_testdata_index_php_worker_restarts Number of PHP worker restarts for this worker
# TYPE frankenphp_testdata_index_php_worker_restarts counter
frankenphp_testdata_index_php_worker_restarts 0
`
require.NoError(t,
testutil.GatherAndCompare(
prometheus.DefaultGatherer,
strings.NewReader(expectedMetrics),
"frankenphp_total_threads",
"frankenphp_busy_threads",
"frankenphp_testdata_index_php_busy_workers",
"frankenphp_testdata_index_php_total_workers",
"frankenphp_testdata_index_php_worker_request_count",
"frankenphp_testdata_index_php_worker_crashes",
"frankenphp_testdata_index_php_worker_restarts",
"frankenphp_testdata_index_php_ready_workers",
))
}
func TestWorkerWithInactiveWatcher(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port 9080
frankenphp {
worker {
file ../testdata/worker-with-watcher.php
num 1
watch ./**/*.php
}
}
}
localhost:9080 {
root ../testdata
rewrite worker-with-watcher.php
php
}
`, "caddyfile")
tester.AssertGetResponse("http://localhost:9080", http.StatusOK, "requests:1")
tester.AssertGetResponse("http://localhost:9080", http.StatusOK, "requests:2")
}