Files
frankenphp/watcher/watcher.go
Alexander Stecher 8d9b6e755b feat: restart workers when on source changes (#1013)
* Adds filesystem watcher with tests.

* Refactoring.

* Formatting.

* Formatting.

* Switches to absolute path in tests.

* Fixes race condition from merge conflict.

* Fixes race condition.

* Fixes tests.

* Fixes markdown lint errors.

* Switches back to absolute paths.

* Reverts back to relative file paths.

* Fixes golangci-lint issues.

* Uses github.com/dunglas/go-fswatch instead.

* Stops watcher before stopping workers.

* Updates docs.

* Avoids segfault in tests.

* Fixes watcher segmentation violations on shutdown.

* Adjusts watcher latencies and tests.

* Adds fswatch to dockerfiles

* Fixes fswatch in alpine.

* Fixes segfault (this time for real).

* Allows queueing new reload if file changes while workers are reloading.

* Makes tests more consistent.

* Prevents the watcher from getting stuck if there is an error in the worker file itself.

* Reverts changing the image.

* Puts fswatch version into docker-bake.hcl.

* Asserts instead of panicking.

* Adds notice

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Runs fswatch version.

* Removes .json.

* Replaces ms with s.

* Resets the channel after closing it.

* Update watcher_options.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Asserts no error instead.

* Fixes a race condition where events are fired after frankenphp has stopped.

* Updates docs.

* Update watcher_options_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Allows queuing events while watchers are reloading.

* go fmt

* Refactors stopping and draining logic.

* Allows extended watcher configuration with dirs, recursion, symlinks, case-sensitivity, latency, monitor types and regex.

* Updates docs.

* Adds TODOS.

* go fmt.

* Fixes linting errors.

* Also allows wildcards in the longform and adjusts docs.

* Adds debug log.

* Fixes the watcher short form.

* Refactors sessions and options into a struct.

* Fixes an overflow in the 'workersReadyWG' on unexpected terminations.

* Properly logs errors coming from session.Start().

* go fmt.

* Adds --nocache.

* Fixes lint issue.

* Refactors and resolves race condition on worker reload.

* Implements debouncing with a timer as suggested by @withinboredom.

* Starts watcher even if no workers are defined.

* Updates docs with file limit warning.

* Adds watch config unit tests.

* Adjusts debounce timings.

* go fmt.

* Adds fswatch to static builder (test).

* Adds a short grace period between stopping and destroying the watcher sessions.

* Adds caddy test.

* Adjusts sleep time.

* Swap to edant/watcher.

* Fixes watch options and tests.

* go fmt.

* Adds TODO.

* Installs edant/watcher in the bookworm image.

* Fixes linting.

* Refactors the watcher into its own module.

* Adjusts naming.

* ADocker image adjustments and refactoring.

* Testing installation methods.

* Installs via gcc instead.

* Fixes pointer formats.

* Fixes lint issues.

* Fixes arm alpine and updates docs.

* Clang format.

* Fixes dirs.

* Adds watcher version arg.

* Uses static lib version.

* Adds watcher to tests and sanitizers.

* Uses sudo for copying the shared lib.

* Removes unnused func.

* Refactoring.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Adds fpic.

* Fixes linting.

* Skips tests in msan.

* Resets op_cache in every worker thread after termination

* Review fixes part 1.

* Test: installing libstc++ instead of gcc.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Allows '/**/' for global recursion and '**/' for relative recursion.

* Reverts using the ignorelist.

* Calls opcache directly.

* Adds --watch to php-server command

* Properly free CStrings.

* Sorts alphabetically and uses curl instead of git.

* Labeling and formatting.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update caddy/caddy.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update testdata/files/.gitignore

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher-c.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Fixes test and Dockerfile.

* Fixes Dockerfiles.

* Resets go versions.

* Replaces unsafe.pointer with uintptr_t

* Prevents worker channels from being destroyed on reload.

* Minimizes the public api by only passing a []string.

* Adds support for directory patterns and multiple '**' globs.

* Adjusts label.

* go fmt.

* go mod tidy.

* Fixes merge conflict.

* Refactoring and formatting.

* Cleans up unused vars and functions.

* Allows dirs with a dot.

* Makes test nicer.

* Add dir tests.

* Moves the watch directive inside the worker directive.

* Adds debug log on special events.

* Removes line about symlinks.

* Hints at multiple possible --watch flags.

* Adds ./**/*.php as default watch configuration.

* Changes error to a warning.

* Changes the default to './**/*.{php,yaml,yml,twig,env}' and supports the {bracket} pattern.

* Fixes linting.

* Fixes merge conflict and adjust values.

* Adjusts values.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-07 13:17:24 +02:00

146 lines
3.4 KiB
Go

package watcher
// #cgo LDFLAGS: -lwatcher -lstdc++
// #cgo CFLAGS: -Wall -Werror
// #include <stdint.h>
// #include <stdlib.h>
// #include "watcher.h"
import "C"
import (
"errors"
"go.uber.org/zap"
"runtime/cgo"
"sync"
"time"
"unsafe"
)
type watcher struct {
sessions []C.uintptr_t
callback func()
trigger chan struct{}
stop chan struct{}
}
// duration to wait before triggering a reload after a file change
const debounceDuration = 150 * time.Millisecond
var (
// the currently active file watcher
activeWatcher *watcher
// after stopping the watcher we will wait for eventual reloads to finish
reloadWaitGroup sync.WaitGroup
// we are passing the logger from the main package to the watcher
logger *zap.Logger
AlreadyStartedError = errors.New("The watcher is already running")
UnableToStartWatching = errors.New("Unable to start the watcher")
)
func InitWatcher(filePatterns []string, callback func(), zapLogger *zap.Logger) error {
if len(filePatterns) == 0 {
return nil
}
if activeWatcher != nil {
return AlreadyStartedError
}
logger = zapLogger
activeWatcher = &watcher{callback: callback}
err := activeWatcher.startWatching(filePatterns)
if err != nil {
return err
}
reloadWaitGroup = sync.WaitGroup{}
return nil
}
func DrainWatcher() {
if activeWatcher == nil {
return
}
logger.Debug("stopping watcher")
activeWatcher.stopWatching()
reloadWaitGroup.Wait()
activeWatcher = nil
}
func (w *watcher) startWatching(filePatterns []string) error {
w.trigger = make(chan struct{})
w.stop = make(chan struct{})
w.sessions = make([]C.uintptr_t, len(filePatterns))
watchPatterns, err := parseFilePatterns(filePatterns)
if err != nil {
return err
}
for i, watchPattern := range watchPatterns {
watchPattern.trigger = w.trigger
session, err := startSession(watchPattern)
if err != nil {
return err
}
w.sessions[i] = session
}
go listenForFileEvents(w.trigger, w.stop)
return nil
}
func (w *watcher) stopWatching() {
close(w.stop)
for _, session := range w.sessions {
stopSession(session)
}
}
func startSession(w *watchPattern) (C.uintptr_t, error) {
handle := cgo.NewHandle(w)
cDir := C.CString(w.dir)
defer C.free(unsafe.Pointer(cDir))
watchSession := C.start_new_watcher(cDir, C.uintptr_t(handle))
if watchSession != 0 {
logger.Debug("watching", zap.String("dir", w.dir), zap.Strings("patterns", w.patterns))
return watchSession, nil
}
logger.Error("couldn't start watching", zap.String("dir", w.dir))
return watchSession, UnableToStartWatching
}
func stopSession(session C.uintptr_t) {
success := C.stop_watcher(session)
if success == 0 {
logger.Warn("couldn't close the watcher")
}
}
//export go_handle_file_watcher_event
func go_handle_file_watcher_event(path *C.char, eventType C.int, pathType C.int, handle C.uintptr_t) {
watchPattern := cgo.Handle(handle).Value().(*watchPattern)
if watchPattern.allowReload(C.GoString(path), int(eventType), int(pathType)) {
watchPattern.trigger <- struct{}{}
}
}
func listenForFileEvents(triggerWatcher chan struct{}, stopWatcher chan struct{}) {
timer := time.NewTimer(debounceDuration)
timer.Stop()
defer timer.Stop()
for {
select {
case <-stopWatcher:
break
case <-triggerWatcher:
timer.Reset(debounceDuration)
case <-timer.C:
timer.Stop()
scheduleReload()
}
}
}
func scheduleReload() {
logger.Info("filesystem change detected")
reloadWaitGroup.Add(1)
activeWatcher.callback()
reloadWaitGroup.Done()
}