- config Server: change time duration to golib duration to simplify
marshal string form
- adjust test following update of config server
- fix test in socket package to use BDD framework & gherkin form
- adjust documentation & test
Package HTTPServer:
- Fix bug in PortUse & PortNotUse
- Move function PortUse & PortNotUse as alone function
- Add test & documentation
- Unify test & documentation following other packages
- Add: error for invalid address check
- Move: errors string InvalidInstance as golib/error type
- Update: add check of listen address in port use function
- FIX: potential CWE-400 with bufio.ReadBytes & bufio.ReadSlices, with
no limited read buffer
- ADD: test to check overflow buffer with discard or error
- REFACTOR: all buffering package, parsing process
- UPDATE: doc, examples, test following changes
- OPTIMIZE: rework code to optimize process
- REWORK: benchmark to check benefice of optimization
- FIX: wording error
Package IOUtils/Multi:
- REWORK: re-design all package to allow sequential/parallel mode
- UPDATE: package with adaptive mode to allow switch automaticly between
sequential and parallel mode following measurment of sample
- OPTIMIZE: code to maximize bandwith and reduce time of write
- UPDATE: documentation, test and comments
- REWORK: testing organization and benchmark aggregation
Package HttpServer:
- FIX: bug with dial addr rewrite for healtcheck & testing PortUse
Package Logger/HookFile:
- FIX: bug with race condition on aggregator counter file
Other:
- Bump dependencies
- FIX: format / import file
- Add some README file to give missing documentations or update existing documentation file
Package Archive:
- Add some comments to godoc information
- Moving NopWriterCloser interface to ioutils package
Package IOUtils:
- New package NopWriterCloser to implement interfac like NopReader
Package Database:
- KVMap: fix missing function following update of kvdriver
Package Duration:
- Rename BDD testing
Package Context/Gin:
- Moving function New between model & interface file
Package AWS:
- rework Walk function to use more generic with standard walk caller function
- func walk will now no more return and include error (can be catched into the given func)
- func walk will now return a bool to continue or stop the loop
- func walk with many input function will now stop when all given function return false
- func walk will now return error only about main process and not given function
Package errors:
- Add interface error into interface Error
Package IOUtils:
- Moving IOWrapper as subPackage and optimize process + allow thread safe
- STD IO Writer Logger: add capabilities of multi filter slice of byte
- STD IO Writer Logger: add function Add..Filter to allow adding filter and not full setting the pattern list
Package HTTP Server:
- Rework filtering logger message to ignore connections reset by peer in server logger
- Rework filtering logger message to ingore all TLS Handshake error in logger
- rework package to allow convert config to model and retrive config from model
- add sub package to manage cipher, curves, auth client, tls version, certificates, root ca...
- add some small test (can be expande to having more coverage)
- optimize some code
Package httpcli:
- update code following change in certificates
Package httpserver:
- update code following change in certificates
Package Config/Components:
- update code following change in certificates
Package FTPClient:
- update code following change in certificates
Package Nats:
- update code following change in certificates
- replace mutex by atomic
- fix minor bugs & optimize code
Package Size:
- add function 'mul' allowing do a multiplaction on the size value by a float64 given value (could use another size.float64)
- add function 'div' allowing do a division of the size value by a float64 given value (could use another size.float64)
- add function 'add' allowing do an addtion of the size value with a uint64 given value (could use another size.uint64)
- add function 'sub' allowing do an substraction of the size value with a uint64 given value (could use another size.uint64)
Package Encoding:
- rework completly crypt package to new package encoding
- part aes of old crypt become new subpackage encoding/aes
- part hex of old crypt (associated with aes) become new standalone subpackage encoding/hexa
- adding new subpackage mux to use multiplexer/de-multiplexer of io stream
- mux is different of multireader / multiwriter in way as io send from mux in one channel will be received only in the coresponding channel of de-mux
- mux is encoding subpackage becauxse based on hex + cbor marshalling / unmarshalling
- all subpackage can be used directly of by io reader / writer
Package Socket/server
- rework server to use as possible direct io to con
- rework server to optimize & fix managment (context / shutdown...)
- rework server to stop connection befor stopping server
- rework server to not load incoming and send it to handler by allowing handler to manager io input/output directly
- server not more used buff internally (less memory consuption)
- adding counter of opened connection
- adding some testing who's checking server & client for each protocol
Package Socket/client
- rework client to allow connect to server before reading/writing on io stream
- rework client to allow consum direct io stream instead of reading locally / call function
- rework client to allow having less memory consumption by removing internal buffer
- rework client allowing to use root io function (bufio, io copy, ...)
Package Socket/multi
- adding new subpackage multi to simplfy using of 1 reader / x writer
- if no writer is define, using an extended io discard
- allowing asynch instancing, populate writer and calling main writer
- allowing calling writer / reader to expose the main io reader / writer
Package Socket/delim:
- adding new subpackage delim to simplify using of bufio reader with delimiter
- using 1 writer / 1 reader
- allowing root io function like readwritecloser, writeto, ...
- use local buffer to prevent race if reader and writer are not is same process
Package PProf:
- fix missing \n in message
Package HttpServer:
- fix bug with init TLS & http2
Other:
- bump dependencies
- Client: optimize code & memory
- Client: allow multiple send content for one connection
- Server: optmize code
- Server: optimize memory use & stream
Package server:
- Rework chan / cancel process
- Simplify & optimize chan / cancel process
- Add Function Action for Ticker as FuncTicker
- Rename type func Action to FuncAction
Package httpserver:
- Fix error on starting runner
- Log error on starting runner
- Optimize some code
Package config:
- Add log info for each starting / reloading component (if default logger is set)
- Add log error for starting error or not started for each component on start / reload (if default logger is set)
Package monitor:
- review ticker delay for starting puller (pass from 500ms to 50ms)
Other:
- bump dependencies
- Add DNS Mapper to force destination for a fqdn source
- Allow wildcard and multi wildcard for fqdn source
- DNS Mapper create Transport, Dialer & http client
- DNS Mapper allow a config input to customize timeout, limit and TLS config
- DNS Mapper use a gloabl transport connection poller
- DNS Mapper implement DialContext & Dial method for transport
- DNS Mapper use cache to accelerate process
- DNS Mapper cache is only dnsmapper, not DNS cache
- Replace old helper for http client with DNSMapper
- Add default DNSMapper into the main lib of http client
- Allow to overide the default DNS Mapper with a new one
- Add ticker to force clean idle connection every given duration
- "Compatible" with old config, as the default config will be used instead of old client build
- Clean code and fix minor bugs
- Add config component to allow use a global config
- Config Component httpcli can overide default httpcli dns mapper when updated
Package Certificates
- Add function type to impose function that return a slice of rootCA string
- update config/component/tls by replacing mutex with atomic
- optimize some code
Package Config
- expand errors code index to add component httpcli
Package Config/component:
- database: add new validation, that config key existing into viper
- head: add new validation, that config key existing into viper
- http: add new validation, that config key existing into viper
- ldap: add new validation, that config key existing into viper
- log: add new validation, that config key existing into viper
- mail: add new validation, that config key existing into viper
- smtp: add new validation, that config key existing into viper
Package AWS:
- update package following update of httpcli
- use interface for http cli that implement the Do function
- update following the config/component/aws
- rework config/component/aws to use atomic instead of mutex
- update test unit following change
Package Request:
- update following http client update
- use interface of HTTP Client that implement DO function, instead of http client struct pointer
- update config & code following
- apply same modification into config/component/request
- update config/component/request by replacing mutex to atomic
Package Server
- add function Uptime to model to expose the duration since last start
Package Semaphore
- apply change name of constant following bump of lib associated
Package Crypt
- fix bug into reader to remove suffix of EOF if prevent
Package Errors:
- expand index module to add DNS Mapper client
Package HTTP Server:
- update monitor to apply changes
Package Socket:
- add additional check if reading error: if buffer has data, send it to handler before break run
Other:
- bump dependencies
- fix security arbitrary path
- fix linter
Package AWS:
- implement resolver v2
Package Cobra:
- fix linter
Package Config/component:
- fix linter
Package Context/Config:
- Add function to set context
Package Database/KV...:
- Fix error
- Fix collision pointer
- Fix models
- Fix circular dependencies
- Add function Delete on driver, table and item
- Add function new on drvier to prevent collision data when create item on table get / walk
Package Duration:
- Add type Duration based on time.Duration to allow transform duration to string instead of int64 nanosecond
- Add function to parse in json, yaml, toml, text, cbor
- Add function to allow convert type into mapstructure (spf13 viper, cobra...)
Package File/Perm:
- Add type Perm based on os.FileMode to allow marshall / unmashall it into octal form instead of string representation (-rwxrwxrwx)
- Add function to marshall / unmarshall in json, yaml, toml, text, cbor
- Add function to allow convert type into mapstructure (spf13 viper, cobra...)
Package File/progress:
- Fix linter
Package HTTPServer :
- Fix linter
- Fix security by adding a default value if not set on config
Package ioutils:
- Fix Linter
Package LDAP:
- Add Clone function
Package logger/hookfile:
- Fix linter
Package nats:
- Fix linter
Package Network/Protocol:
- Fix bug with quote / Dbl Quote on unmarshall
Package Password:
- Replace password with crypto rand instead of math rand
Package Size:
- Fix potential overflow
- Add function to format value into Int32, Int, Uint32, Uint, Float32
- Add function to parse Float64 into type Size
Package Socket:
- change config uint32 to golib Size, time.Duration to golib Duration
- add TLS managment to server TCP, discard for UDP & Unix file Local Domain
- add function Info Server to print information of server when listen is starting
Package Errors:
- add function to check & cast error interface into golib Error interface
- update CodeError type to simplify management & error creation
- add function to simplify call of Error function from a generic error interface
- remove some useless function from Error interface
All Other Packages:
- apply change of package Errors into all other packages
- Rework logger
- Remove deprecated functions
- Split Logger to sub package (fields, level, config, ...)
- Optimize dependencies
- Rework Hookfile: fix error like FD still opened
- Rework Hooksyslog: use same model like Hookfile, use network/protocol instead of self lib
- Rework HookStd: use independent hook for std out & std err
- Fix std config make generic options for files & syslog
- Apply formatter to hook instead of main logger entry
- optimize code
Package ioutils:
- rework PathCheckCreate funct: optimize code & fix some error
Package Network:
- create sub package protocol for all network protocl use
- add encode function
Package httpcli:
- remove file network
- use package network/protocol instead of network file
Package archive:
- apply change following rework of logger
Package aws:
- apply change following rework of logger
Package cluster:
- apply change following rework of logger
Package cobra:
- apply change following rework of logger
Package Config Component:
- apply change following rework of logger to component log
- fix logger for monitoring
- fix component following fix of package request / monitoring
Package context:
- apply change following rework of logger
Package database:
- apply change following rework of logger
Package httpserver:
- apply change following rework of logger
Package ldap:
- apply change following rework of logger
Package monitor:
- apply change following rework of logger
- fix logger for monitoring
- fix minor bugs
Package nats:
- apply change following rework of logger
Package nutsdb:
- apply change following rework of logger
Package request:
- apply change following rework of logger
- fix minor bug
- fix missing logger for monitoring
- add one line for healthcheck (info or error)
Package router:
- apply change following rework of logger
Package static:
- apply change following rework of logger
Package status:
- apply change following rework of logger
- fix bug with mandatory component
Package viper:
- apply change following rework of logger
Other:
- bump dependencies
- github action workflow fix
- Add interface Errors to expose func for collection of errors
Package Server:
- Add runner packages for start/stop & ticker process launch in goroutines
- Runner Start/Stop : register a start and a stop function called into a managment instance of goroutines, expose package Server interface
- Runner Ticker : register a func periodycly called into a managment instance of goroutine with a time ticker, expose package server interface
- Add tools function to run x times a function or periodicly with a check function
Package HttpServer:
- Remove old run managment instance and use new Start&Stop Runner instead self runner
- Replace the poller in server start / stop with package server tools function
- Adjust code following this change
- Add in healthcheck a test of dialing to server binding network to perform the healthcheck
- Remove WaitNotify funct (cannot having multiple of waitnotify in same app) : keep waitnotify function in package config
Package Monitor:
- Remove old running system to use package server/runner/ticker
- Adjust code following
Package Cobra :
- Fix minor bug with writing configure
- Optimize code
Bump dependencies
- rework & optimize listing of dependencies to start or reload component
Package HttpServer:
- fix bug with writing on channel without listen channel
Bump dependencies
Package Monitoring :
- use packag dedicated to monitor component
- each monitor work as standalone server to monitor health
- collect metrics to export them to prometheus exporter
Package Prometheus :
- review to simplify use for API and not API metrics
- optimize code
Package Status :
- Rework to use Monitor package
- Rework to use native json / text Marshaller interface
Context :
- rework context config (context var) to use sync map and sync RWMutex (WORM)
- move gin context to dedicated sub package (dependancies of logger make circular dependencies)
- optimize code
Config :
- rework to optimize sync / collect of component
- rework status to monitor
- remove monitor managment from config to each component
- add a func to set default logger to implement inherit default logger options
- optimize code
IOUtils :
- isolate logger / closer interface as a usable & public interface & instance
- this interface / instance allow to collect io.closer over a context to close all if context is done
Logger :
- rework to use context.config map
- rework to use ioutils closer
- rework to allow options to inherit a default options, or the last version of options
- optimize code
Size :
- Add package Size to calculate and manipulate size Byte or bit
- Add encoding : Text/JSON/Yaml/Toml...
- Add option to défine default unit : Byte or bit
Other :
- adjust following code
- optimize code
- limit use of atomic value
- rework to use RWMutex instead of sync.Mutex to maximize capabilities of read instead of write
- remove 32bit build for CI/CD
- add darwin/arm64 build for CI/CD
Bump Dependencies
- Change reloading component method : try to reload all component and store errors, report list of errors but don't break the reloading process
Package Logger :
- Fix bug with entry logger filtering
Package Viper:
- reword message log before reloading config file (watchFS)
- add message log after reloading config file (watchFS)
Package Status :
- Fix DATA Race with status/info
- Add component key into log error message for health
- Add component key into function to use it into info (name)
- Reword health message : no OK/KO (still into status info), add error message reporting
Package httpserver :
- status info : apply update status component, use key in name
- status info : optimize code
Package Request :
- Fix error in url path operation
- Status info : optimize code
- Status info : apply update component, add endpoint hostname with name
Package Static :
- apply status component update
Other :
- Bump dependencies
- Fix too many open file for logger file
- Fix invalid FD for log file
- Fix bug with field into entry
- Fix race detection on testing
Package Status :
- Add capability to send short into query to received a short output without any result of component
- Add Connection Header as Close to router status return of Get
- Fix race detection into health with concurrent read/write data
- Fix race detection with update of main router
Package HTTPServer :
- Add option to disable keepalive for server
- Fix config tag error
- Fix logger usage / close
Package Config :
- Fix component using logger to implement a close
- Fix logger initialization :
- use local var to setup new logger based on logger clone if loger still existing
- if error while configuring new logger, do not change logger
- closing old logger before replace it with new
Package Router :
- Fix missing ignore error return
- Fix golib logger not closed
Other :
- Bump dependencies
- Config Model : add a config model with a golib RouerStatus Config Model to use a AWS connection for API with a request status health check
- Config Interface : add function GetAccessKey to retrieve the accesskey value currently used into the current connection
- function Walk (Object, version, ...) : fix bug with pointer of string not initialized
Package Config :
- interface Component : add status router pointer into the Init function to allow used a global router status for all component status registration as router status component
Package Status :
- move Status Config as an sub package of Package Status
Package Errors :
- add CamelCase const
Package Logger :
- fix following bump dependencies
Global :
- bump dependencies
- change init of errors files : change function never call vy panic to prevent an error code collision
- Bucket : add function Walk based on list but run a given function for each found bucket
- Object : add function Walk & WalkPrefix based on ListPrefix to run a given function for each returned object
- Object : add function WalkVersion & WalkVersionPrefix based on ListVersion to run a given function for each returned object version
Package logger:
- Fix bug with compat : a nil error not clean when sent to logger
Package Context:
- Fix bug with context/config : the load key return an atomic value instead of the interface stored
- major dependancies are :
- golib/context config
- golib/viper
- interface config extend :
- golib/context config interface
- component list
- interface component list :
this interface implement all function to manage a collection of component. All component are registred with they config key.
A component must implement basic function like start, stop, reload, defaultConfig...
The main config json is set by calling all component config with the config key attached
Each component have some status status like isStarted, isRunning, ...
Each component must also declare his dependencies with other component. As that when start or reload is called,
the component is sure that dependencies are still started or reloaded.
- They are 4 component for now : log, tls, headers and http server
- The config package will start a new context / cancelfunc on init to be sure to stop cleanly all component and process
Add Package Viper :
- this package is an helper to the config package with the spf13 viper lib
- this package can watch any change of a config file or can be connected to a remote config cluster like ETCD
Add Package Cobra :
- this package is an helper to make a CLI with flag / command
- this package is based on spf13 cobra has all method to be connected to viper golib
# Refactor Logger :
- Remove logger as globals to use dedicated pointer
- Add compat file to allow retro-compatibility (function are deprecated)
- Add field management to allow add custom information
- Add option struct to manage options
- Add entry struct to manage all log entry
- Add multithreading capabilities (with race prevention)
- Allow to log to file + stdout/stderr + syslog
- Add io.WriteCloser compatibility interface
- Add logrus hook to manage option by hook instance
- Add capabilities to custom each logfile/syslog/standard independently
- Add small test suite
# Update Packages for logger new options :
- archive
- cluster
- context
- httpcli
- httpserver
- ldap
- tests
# Other :
- bump dependancies
- update minio server for aws test
* validators 'required' needs bool to be a ptr (if not, 'false' value will be declared as missing...), but required was not necessary for boolean...
Co-authored-by: lgambini <lucas.gambini.ext@orange.com>
- Pkg HTTP Server : add option to force TLS as mandatory to start and server
- Pkg HTTP Server : add function to listen server with multiple handler (map[key string]http.handler)
- Pkg HTTP Server : add option to define an handler key to associate one server with one handler
- Pkg Error : change the register part to allow identify if a code is free or still defined
- Pkg Error : apply change of register into all modules
- Make some optimization
- Refactor ErrorType, list errors managment, codeError
- Add interface Error with error interface implement
- Add type CodeError assign typiclly to const that represent code of error
- Add func to registry func to retrieve message from an uint16 codeError (typicaly a switch of each codeError const)
- Add default errorCode with default errorMessage if no one code or message is found
- Add modeError to manage how to manage compatibility between Error interface and error interface
- Add Error interface that allow parent link (parent as error or Error interface), code and trace management
- Add trace finder to allow find func/file/line caller when Error is call
- Add http 2 transport in httpcli
- Add http 2 transport in httpserver
- Add function to get client http with timeout management in httpcli
- Add function to get Error if occurs of http client in httpcli
- Add test for smtp package
- Chg return error by returning Error in all packages
- Chg package njs-archive by archive
- Chg package njs-certif by certificates
- Chg package njs-console by console
- Chg package njs-crypt by crypt
- Chg package njs-errors by errors
- Chg package njs-httpcli by httpcli
- Chg package njs-httpserver by httpserver
- Chg package njs-ioutils by ioutils
- Chg package njs-ldap by ldap
- Chg package njs-logger by logger
- Chg package njs-password by password
- Chg package njs-progress by progress
- Chg package njs-router by router
- Chg package njs-semaphore by semaphore
- Chg package njs-smtp by smtp
- Chg package njs-static by static
- Chg package njs-status by status
- Chg package njs-version by version
- Fix dependancies gopkg by github/go-ldap for go module compatibility
- Fix gin Abort call by gin Abort with Error in static package
- Fix issue #18 in status package : replace partner by component
- Fix go vet error
- Del deprecated function
- Del useless function & files
- Bump dependancies
- Apply CHG in README.md
