Files
frankenphp/watcher/watch_pattern.go
Alexander Stecher 8d9b6e755b feat: restart workers when on source changes (#1013)
* Adds filesystem watcher with tests.

* Refactoring.

* Formatting.

* Formatting.

* Switches to absolute path in tests.

* Fixes race condition from merge conflict.

* Fixes race condition.

* Fixes tests.

* Fixes markdown lint errors.

* Switches back to absolute paths.

* Reverts back to relative file paths.

* Fixes golangci-lint issues.

* Uses github.com/dunglas/go-fswatch instead.

* Stops watcher before stopping workers.

* Updates docs.

* Avoids segfault in tests.

* Fixes watcher segmentation violations on shutdown.

* Adjusts watcher latencies and tests.

* Adds fswatch to dockerfiles

* Fixes fswatch in alpine.

* Fixes segfault (this time for real).

* Allows queueing new reload if file changes while workers are reloading.

* Makes tests more consistent.

* Prevents the watcher from getting stuck if there is an error in the worker file itself.

* Reverts changing the image.

* Puts fswatch version into docker-bake.hcl.

* Asserts instead of panicking.

* Adds notice

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev-alpine.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update dev.Dockerfile

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Runs fswatch version.

* Removes .json.

* Replaces ms with s.

* Resets the channel after closing it.

* Update watcher_options.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Asserts no error instead.

* Fixes a race condition where events are fired after frankenphp has stopped.

* Updates docs.

* Update watcher_options_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Allows queuing events while watchers are reloading.

* go fmt

* Refactors stopping and draining logic.

* Allows extended watcher configuration with dirs, recursion, symlinks, case-sensitivity, latency, monitor types and regex.

* Updates docs.

* Adds TODOS.

* go fmt.

* Fixes linting errors.

* Also allows wildcards in the longform and adjusts docs.

* Adds debug log.

* Fixes the watcher short form.

* Refactors sessions and options into a struct.

* Fixes an overflow in the 'workersReadyWG' on unexpected terminations.

* Properly logs errors coming from session.Start().

* go fmt.

* Adds --nocache.

* Fixes lint issue.

* Refactors and resolves race condition on worker reload.

* Implements debouncing with a timer as suggested by @withinboredom.

* Starts watcher even if no workers are defined.

* Updates docs with file limit warning.

* Adds watch config unit tests.

* Adjusts debounce timings.

* go fmt.

* Adds fswatch to static builder (test).

* Adds a short grace period between stopping and destroying the watcher sessions.

* Adds caddy test.

* Adjusts sleep time.

* Swap to edant/watcher.

* Fixes watch options and tests.

* go fmt.

* Adds TODO.

* Installs edant/watcher in the bookworm image.

* Fixes linting.

* Refactors the watcher into its own module.

* Adjusts naming.

* ADocker image adjustments and refactoring.

* Testing installation methods.

* Installs via gcc instead.

* Fixes pointer formats.

* Fixes lint issues.

* Fixes arm alpine and updates docs.

* Clang format.

* Fixes dirs.

* Adds watcher version arg.

* Uses static lib version.

* Adds watcher to tests and sanitizers.

* Uses sudo for copying the shared lib.

* Removes unnused func.

* Refactoring.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Adds fpic.

* Fixes linting.

* Skips tests in msan.

* Resets op_cache in every worker thread after termination

* Review fixes part 1.

* Test: installing libstc++ instead of gcc.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Test: using msan ignorelist.

* Allows '/**/' for global recursion and '**/' for relative recursion.

* Reverts using the ignorelist.

* Calls opcache directly.

* Adds --watch to php-server command

* Properly free CStrings.

* Sorts alphabetically and uses curl instead of git.

* Labeling and formatting.

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/sanitizers.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update .github/workflows/tests.yaml

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update caddy/caddy.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update docs/config.md

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update frankenphp_with_watcher_test.go

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update testdata/files/.gitignore

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher-c.h

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Update watcher/watcher.c

Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* Fixes test and Dockerfile.

* Fixes Dockerfiles.

* Resets go versions.

* Replaces unsafe.pointer with uintptr_t

* Prevents worker channels from being destroyed on reload.

* Minimizes the public api by only passing a []string.

* Adds support for directory patterns and multiple '**' globs.

* Adjusts label.

* go fmt.

* go mod tidy.

* Fixes merge conflict.

* Refactoring and formatting.

* Cleans up unused vars and functions.

* Allows dirs with a dot.

* Makes test nicer.

* Add dir tests.

* Moves the watch directive inside the worker directive.

* Adds debug log on special events.

* Removes line about symlinks.

* Hints at multiple possible --watch flags.

* Adds ./**/*.php as default watch configuration.

* Changes error to a warning.

* Changes the default to './**/*.{php,yaml,yml,twig,env}' and supports the {bracket} pattern.

* Fixes linting.

* Fixes merge conflict and adjust values.

* Adjusts values.

---------

Co-authored-by: a.stecher <a.stecher@sportradar.com>
Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>
2024-10-07 13:17:24 +02:00

167 lines
4.7 KiB
Go

package watcher
import (
"go.uber.org/zap"
"path/filepath"
"strings"
)
type watchPattern struct {
dir string
patterns []string
trigger chan struct{}
}
func parseFilePatterns(filePatterns []string) ([]*watchPattern, error) {
watchPatterns := make([]*watchPattern, 0, len(filePatterns))
for _, filePattern := range filePatterns {
watchPattern, err := parseFilePattern(filePattern)
if err != nil {
return nil, err
}
watchPatterns = append(watchPatterns, watchPattern)
}
return watchPatterns, nil
}
// this method prepares the watchPattern struct for a single file pattern (aka /path/*pattern)
// TODO: using '/' is more efficient than filepath functions, but does not work on windows
func parseFilePattern(filePattern string) (*watchPattern, error) {
w := &watchPattern{}
// first we clean the pattern
absPattern, err := filepath.Abs(filePattern)
if err != nil {
return nil, err
}
w.dir = absPattern
// then we split the pattern to determine where the directory ends and the pattern starts
splitPattern := strings.Split(absPattern, "/")
patternWithoutDir := ""
for i, part := range splitPattern {
isFilename := i == len(splitPattern)-1 && strings.Contains(part, ".")
isGlobCharacter := strings.ContainsAny(part, "[*?{")
if isFilename || isGlobCharacter {
patternWithoutDir = filepath.Join(splitPattern[i:]...)
w.dir = filepath.Join(splitPattern[:i]...)
break
}
}
// now we split the pattern according to the recursive '**' syntax
w.patterns = strings.Split(patternWithoutDir, "**")
for i, pattern := range w.patterns {
w.patterns[i] = strings.Trim(pattern, "/")
}
// finally, we remove the trailing slash and add leading slash
w.dir = "/" + strings.Trim(w.dir, "/")
return w, nil
}
func (watchPattern *watchPattern) allowReload(fileName string, eventType int, pathType int) bool {
if !isValidEventType(eventType) || !isValidPathType(pathType, fileName) {
return false
}
return isValidPattern(fileName, watchPattern.dir, watchPattern.patterns)
}
// 0:rename,1:modify,2:create,3:destroy,4:owner,5:other,
func isValidEventType(eventType int) bool {
return eventType <= 3
}
// 0:dir,1:file,2:hard_link,3:sym_link,4:watcher,5:other,
func isValidPathType(pathType int, fileName string) bool {
if pathType == 4 {
logger.Debug("special edant/watcher event", zap.String("fileName", fileName))
}
return pathType <= 2
}
func isValidPattern(fileName string, dir string, patterns []string) bool {
// first we remove the dir from the pattern
if !strings.HasPrefix(fileName, dir) {
return false
}
fileNameWithoutDir := strings.TrimLeft(fileName, dir+"/")
// if the pattern has size 1 we can match it directly against the filename
if len(patterns) == 1 {
return matchBracketPattern(patterns[0], fileNameWithoutDir)
}
return matchPatterns(patterns, fileNameWithoutDir)
}
func matchPatterns(patterns []string, fileName string) bool {
partsToMatch := strings.Split(fileName, "/")
cursor := 0
// if there are multiple patterns due to '**' we need to match them individually
for i, pattern := range patterns {
patternSize := strings.Count(pattern, "/") + 1
// if we are at the last pattern we will start matching from the end of the filename
if i == len(patterns)-1 {
cursor = len(partsToMatch) - patternSize
}
// the cursor will move through the fileName until the pattern matches
for j := cursor; j < len(partsToMatch); j++ {
cursor = j
subPattern := strings.Join(partsToMatch[j:j+patternSize], "/")
if matchBracketPattern(pattern, subPattern) {
cursor = j + patternSize - 1
break
}
if cursor > len(partsToMatch)-patternSize-1 {
return false
}
}
}
return true
}
// we also check for the following bracket syntax: /path/*.{php,twig,yaml}
func matchBracketPattern(pattern string, fileName string) bool {
openingBracket := strings.Index(pattern, "{")
closingBracket := strings.Index(pattern, "}")
// if there are no brackets we can match regularly
if openingBracket == -1 || closingBracket == -1 {
return matchPattern(pattern, fileName)
}
beforeTheBrackets := pattern[:openingBracket]
betweenTheBrackets := pattern[openingBracket+1 : closingBracket]
afterTheBrackets := pattern[closingBracket+1:]
// all bracket entries are checked individually, only one needs to match
// *.{php,twig,yaml} -> *.php, *.twig, *.yaml
for _, pattern := range strings.Split(betweenTheBrackets, ",") {
if matchPattern(beforeTheBrackets+pattern+afterTheBrackets, fileName) {
return true
}
}
return false
}
func matchPattern(pattern string, fileName string) bool {
if pattern == "" {
return true
}
patternMatches, err := filepath.Match(pattern, fileName)
if err != nil {
logger.Error("failed to match filename", zap.String("file", fileName), zap.Error(err))
return false
}
return patternMatches
}