527 Commits

Author SHA1 Message Date
nabbar
c4b5f11efc Rework ioutils/gzipreader/fileprogress
Package ioutils:
- remove file progress from ioutils and rework it to package
  file/progress

Package file/progress:
- simplify call / use of file progress
- optimize code
- use atomic to function progress
- isolation part of code
- make interface more compatible with *os/File / io interface

Package archive/gzipreader
- create package to expose a io.reader interface from a no gzipped io.reader
- add interface GZipReader to expose metrics like rate of compression

Package archive:
- apply following change
- add minor internal change into errors files

Package artifact:
- apply following change
- add minor internal change into errors files

Package aws:
- apply following change
- removing minio server from repo

Package mail:
- apply following change
- add minor internal change into errors files

Package nutsdb:
- apply following change
- add minor internal change into errors files

Package static:
- apply following change

Other:
- bump dependencies
- ci/cd : add a wget command to dl minio server for testing
- add aws/minio to gitignore
2023-08-22 19:42:46 +02:00
nabbar
3a2c52195f Rework Package Database
Package Database/gorm:
- move package database using gorm, to package database/gorm
- update config component following

Package Database/KVItem:
- create new package to manage item of KV Db
- allow to track change on item model
- allow to store only if changed
- embedded driver function to load / store to make instance portable

Package Database/KVTable:
- create new package to manage a table (list of map key : value)
- embedded driver instance to make instance portable
- allow to walk, list, get items with KVItem instance

Package Database/KVDriver:
- create new package to wrap db KV function to KVTable awaiting function
- Allow to skip function Walk replace by a n internal call of function List and Get

Package Database/Map:
- create new package to wrap KV function usage of map[comparable]any to KVDriver model
- Use json marshall/unmarshall between map[comparable]any to model any
2023-08-22 19:35:33 +02:00
nabbar
ffb5724c2c Minor Fix & Go1.21
Package Socket:
- fix bug with timeout

Package AWS:
- update config model following validator update

Other:
- bump dependencies
- go1.21
2023-08-22 19:35:16 +02:00
nabbar
bb5c6a6860 Package request:
- options: fix validator tag for struct, following bump dependencies
v1.11.6
2023-08-08 13:47:36 +02:00
nabbar
43178b5b4a Bump dependencies v1.11.5 2023-08-08 08:24:17 +02:00
nabbar
0b1449fa47 Package logger:
- fix missing ticker stop

Package monitor:
- fix missing ticker stop

Package server:
- fix missing ticker stop
2023-08-06 14:33:36 +02:00
nabbar
eb5c4219d3 Package Status:
- fix listmandatory: transform into sync.map to prevent race detection
2023-08-06 14:30:52 +02:00
nabbar
212cfdc0cf Package Socket:
- rework client / server to offer tcp & udp server/client in addition of unix socket file
- add config model
- add aggregation function
2023-07-30 16:03:55 +02:00
nabbar
aae7b9ea0e Package Status:
- add strictly health function to check status with OK only
- change healthy function to check status not KO only (so warn or OK)

Bump dependencies
v1.11.4
2023-07-29 14:07:52 +02:00
nabbar
ec30365fb6 Package Shell:
- fix info command & autocompletion

Other:
- bump dependencies
v1.11.3
2023-07-28 13:33:22 +02:00
Nicolas JUHEL
43dbab6811 Merge pull request #165 from nabbar/fix_2023-07-24
Fix 2023-07

Package Status:
- Rework for the management of mandatory components to qualify the global status.
- Added qualification to components: "should", "must", "one of".
- These qualifications will influence the overall state but not the unitary state of each component.

Package Socket:
- Creation package to open and listen local unix file socket
- Allow to specify a handler who's needing a conn instance as entry
- Create a client sub package to allow negociate with the local unix file socket

Package Shell:
- adding sub package for command and command collection

Package Monitor:
- add function to expose monitor command for shell

Package Config:
- add function to expose monitor command for shell

Package Network:
- Add "IP" protocol

Package Httpcli:
- Rework the construction of the client
- Add capability to specify proxy url

Package Cobra:
- add function to add flag to global command like verbose or config

Package Log:
- fix seg fault when calling a nil instance
- remove println in hookfile / hooksyslog

Other:
- bump dependencies
2023-07-26 21:34:04 +02:00
Nicolas JUHEL
7cd4eadc50 Package Status:
- Rework for the management of mandatory components to qualify the global status.
- Added qualification to components: "should", "must", "one of".
- These qualifications will influence the overall state but not the unitary state of each component.

Package Socket:
- Creation package to open and listen local unix file socket
- Allow to specify a handler who's needing a conn instance as entry
- Create a client sub package to allow negociate with the local unix file socket

Package Shell:
- adding sub package for command and command collection

Package Monitor:
- add function to expose monitor command for shell

Package Config:
- add function to expose monitor command for shell

Package Network:
- Add "IP" protocol

Pakcage Httpcli:
- Rework the construction of the client
- Add capability to specify proxy url

Package Cobra:
- add function to add flag to global command like verbose or config

Package Log:
- fix seg fault when calling a nil instance
- remove println in hookfile / hooksyslog
2023-07-26 21:11:39 +02:00
Nicolas JUHEL
bb2f7f55c7 Merge pull request #164 from nabbar/fix_2307
Fix Logger & Status
v1.11.2
2023-07-05 11:34:19 +02:00
nabbar
7eda9dbf0e Package Logger :
- fix buffer wrtting / cleanning
- add recovery to catch fatal error

Package Server :
- add recovery to catch fatal error

Package Status :
- add IsHealthy function with cache value to prevent checking monitor on all call

Bump dependencies
2023-07-05 11:14:38 +02:00
nabbar
33f9fe3eb9 bump dependencies 2023-06-26 09:19:50 +02:00
Nicolas JUHEL
274d64a079 Merge pull request #163 from nabbar/LoggerRework
Rework Logger / Rework Network Protocol L4 / Fix minor bugs

    Package Logger:
    - Rework logger
    - Remove deprecated functions
    - Split Logger to sub package (fields, level, config, ...)
    - Optimize dependencies
    - Rework Hookfile: fix error like FD still opened
    - Rework Hooksyslog: use same model like Hookfile, use network/protocol instead of self lib
    - Rework HookStd: use independent hook for std out & std err
    - Fix std config make generic options for files & syslog
    - Apply formatter to hook instead of main logger entry
    - optimize code
    
    Package ioutils:
    - rework PathCheckCreate funct: optimize code & fix some error
    
    Package Network:
    - create sub package protocol for all network protocl use
    - add encode function
    
    Package httpcli:
    - remove file network
    - use package network/protocol instead of network file
    
    Package archive:
    - apply change following rework of logger
    
    Package aws:
    - apply change following rework of logger
    
    Package cluster:
    - apply change following rework of logger
    
    Package cobra:
    - apply change following rework of logger
    
    Package Config Component:
    - apply change following rework of logger to component log
    - fix logger for monitoring
    - fix component following fix of package request / monitoring
    
    Package context:
    - apply change following rework of logger
    
    Package database:
    - apply change following rework of logger
    
    Package httpserver:
    - apply change following rework of logger
    
    Package ldap:
    - apply change following rework of logger
    
    Package monitor:
    - apply change following rework of logger
    - fix logger for monitoring
    - fix minor bugs
    
    Package nats:
    - apply change following rework of logger
    
    Package nutsdb:
    - apply change following rework of logger
    
    Package request:
    - apply change following rework of logger
    - fix minor bug
    - fix missing logger for monitoring
    - add one line for healthcheck (info or error)
    
    Package router:
    - apply change following rework of logger
    
    Package static:
    - apply change following rework of logger
    
    Package status:
    - apply change following rework of logger
    - fix bug with mandatory component
    
    Package viper:
    - apply change following rework of logger
    
    Other:
    - bump dependencies
    - github action workflow fix
v1.11.1
2023-06-26 08:55:35 +02:00
nabbar
f31950daef Package Logger:
- Rework logger
- Remove deprecated functions
- Split Logger to sub package (fields, level, config, ...)
- Optimize dependencies
- Rework Hookfile: fix error like FD still opened
- Rework Hooksyslog: use same model like Hookfile, use network/protocol instead of self lib
- Rework HookStd: use independent hook for std out & std err
- Fix std config make generic options for files & syslog
- Apply formatter to hook instead of main logger entry
- optimize code

Package ioutils:
- rework PathCheckCreate funct: optimize code & fix some error

Package Network:
- create sub package protocol for all network protocl use
- add encode function

Package httpcli:
- remove file network
- use package network/protocol instead of network file

Package archive:
- apply change following rework of logger

Package aws:
- apply change following rework of logger

Package cluster:
- apply change following rework of logger

Package cobra:
- apply change following rework of logger

Package Config Component:
- apply change following rework of logger to component log
- fix logger for monitoring
- fix component following fix of package request / monitoring

Package context:
- apply change following rework of logger

Package database:
- apply change following rework of logger

Package httpserver:
- apply change following rework of logger

Package ldap:
- apply change following rework of logger

Package monitor:
- apply change following rework of logger
- fix logger for monitoring
- fix minor bugs

Package nats:
- apply change following rework of logger

Package nutsdb:
- apply change following rework of logger

Package request:
- apply change following rework of logger
- fix minor bug
- fix missing logger for monitoring
- add one line for healthcheck (info or error)

Package router:
- apply change following rework of logger

Package static:
- apply change following rework of logger

Package status:
- apply change following rework of logger
- fix bug with mandatory component

Package viper:
- apply change following rework of logger

Other:
- bump dependencies
- github action workflow fix
2023-06-26 08:36:29 +02:00
Nicolas JUHEL
abe84d7498 Merge pull request #161 from nabbar/aws-mpu-rework
Rework Aws/MPU

Package AWS:
- rework MultipartUpload process & helper
- update test to use lib size
- update object multipart to use new helper
    
Package IO Utils :
- add truncate & sync to FileProgress
- fix error on open file mode for FileProgress
    
Package Console :
- fix interface used for color buffer
    
Package Cobra :
- add function to print message on write config to use custom message instead of internal message. If the function is not set, the default message will be print.
    
Other:
- fix golangci-lint config to remove crazy linter (use only golang group compliance linter)
- bump dependencies
- fix issue #160
v1.11.0
2023-06-02 10:42:55 +02:00
nabbar
1ab80f338e Package AWS:
- rework MultipartUpload process & helper
- update test to use lib size
- update object multipart to use new helper

Package IO Utils :
- add truncate & sync to FileProgress
- fix error on open file mode for FileProgress

Package Console :
- fix interface used for color buffer

Package Cobra :
- add function to print message on write config to use custom message instead of internal message. If the function is not set, the default message will be print.

Other:
- fix golangci-lint config to remove crazy linter (use only golang group compliance linter)
- bump dependencies
2023-06-02 10:26:35 +02:00
nabbar
7316c249e2 Bump dependencies 2023-05-15 14:48:57 +02:00
nabbar
5d447406d6 Package AWS:
- objects / list Access All : add username param to allow list access and return a slice of access struct for the specify user, if username is empty, send the request without username
- sizepart : remove sizepart struct and replace it by package golib/size

Package Logger:
- fields : fix error with field & sync/map

Bump dependencies
v1.10.13
2023-04-21 10:08:48 +02:00
nabbar
03014192e1 Package Config:
- component aws : fix creation client with httpcli config was not init before client creation

Other:
- bump dependencies
v1.10.12
2023-04-18 17:29:43 +02:00
nabbar
e0e9aad81b Package HTTPCli :
- Fix testing suite name
Other :
- fix linter config
- bump dependencies
v1.10.11
2023-04-17 09:56:15 +02:00
nabbar
4be6357413 Package HttpCli :
- Fix ForceIP bug
- Add BDD test
2023-04-15 01:35:14 +02:00
nabbar
c86e1dc1b5 Package httpserver:
- fix bug with context deadline

Package Database:
  - fix darwin/arm64 import C for gorm/sqlite (using shared lib sqlite)

Bump dependencies
v1.10.10
2023-04-11 15:39:24 +02:00
Nicolas JUHEL
bd04ad86e6 Merge pull request #159 from nabbar/runner
Refactor Runner & Package Server

Package Errors:
- Add interface Errors to expose func for collection of errors

Package Server:
- Add runner packages for start/stop & ticker process launch in goroutines
- Runner Start/Stop : register a start and a stop function called into a managment instance of goroutines, expose package Server interface
- Runner Ticker : register a func periodycly called into a managment instance of goroutine with a time ticker, expose package server interface
- Add tools function to run x times a function or periodicly with a check function

Package HttpServer:
- Remove old run managment instance and use new Start&Stop Runner instead self runner
- Replace the poller in server start / stop with package server tools function
- Adjust code following this change
- Add in healthcheck a test of dialing to server binding network to perform the healthcheck
- Remove WaitNotify funct (cannot having multiple of waitnotify in same app) : keep waitnotify function in package config

Package Monitor:
- Remove old running system to use package server/runner/ticker
- Adjust code following

Package Cobra :
- Fix minor bug with writing configure
- Optimize code

Bump dependencies
v1.10.9
2023-04-06 13:31:29 +02:00
nabbar
e047c986a6 Package Errors:
- Add interface Errors to expose func for collection of errors

Package Server:
- Add runner packages for start/stop & ticker process launch in goroutines
- Runner Start/Stop : register a start and a stop function called into a managment instance of goroutines, expose package Server interface
- Runner Ticker : register a func periodycly called into a managment instance of goroutine with a time ticker, expose package server interface
- Add tools function to run x times a function or periodicly with a check function

Package HttpServer:
- Remove old run managment instance and use new Start&Stop Runner instead self runner
- Replace the poller in server start / stop with package server tools function
- Adjust code following this change
- Add in healthcheck a test of dialing to server binding network to perform the healthcheck
- Remove WaitNotify funct (cannot having multiple of waitnotify in same app) : keep waitnotify function in package config

Package Monitor:
- Remove old running system to use package server/runner/ticker
- Adjust code following

Package Cobra :
- Fix minor bug with writing configure
- Optimize code

Bump dependencies
2023-04-06 13:17:38 +02:00
Nicolas JUHEL
7ea47dd800 Merge pull request #158 from nabbar/fix_230330
Fix & Optimize

Package Config:
- rework & optimize listing of dependencies to start or reload component 

Package HttpServer:
- fix bug with writing on channel without listen channel

Other :
- bump dependencies
v1.10.8
2023-03-30 18:45:58 +02:00
nabbar
16bcc244dc Package Config:
- rework & optimize listing of dependencies to start or reload component

Package HttpServer:
- fix bug with writing on channel without listen channel

Bump dependencies
2023-03-30 18:06:00 +02:00
nabbar
c1ff964cd7 PKG Minotor:
- fix minor bug with range of valid value for sli
v1.10.6.1 v1.10.7
2023-03-15 15:51:57 +01:00
nabbar
b31fd548a3 PKG Minotor:
- fix divied by zero
- fix twice type of data mixed in same metrics
v1.10.6
2023-03-15 15:40:39 +01:00
Nicolas JUHEL
27fc6a0c24 Merge pull request #157 from nabbar/pkg_monitor_metrics
dependencies & some metrics
v1.10.5
2023-03-15 10:27:35 +01:00
nabbar
1739ee9e72 Package Monitor:
- Pool : add some metrics for SLIs
Bump dependencies:
- fix protobuf CVE-2023-24535
2023-03-15 10:03:11 +01:00
Nicolas JUHEL
bb4e94454e Merge pull request #155 from nabbar/console
Package Console: 
- adding padding center

Other: 
- bump dependencies
v1.10.4
2023-03-13 09:13:22 +01:00
nabbar
88f135c79e Package Console:
- adding padding center

Other:
- bump dependencies
2023-03-13 08:51:35 +01:00
Nicolas JUHEL
54462f03e8 Merge pull request #154 from nabbar/fix_2023-03-07
Fix Package Monitor

- create function to init prom metrics
- add logger to log errors
- add function to trigger a duration interval between 2 collect metrics
- fix use unic prom metrics for all component
- fix encoding info text
- fix status duration
- bump dependencies
v1.10.3
2023-03-08 16:40:58 +01:00
nabbar
6e005a75c5 Bump dependencies 2023-03-08 16:19:45 +01:00
nabbar
ef47435dcb Package Monitor:
- create function to init prom metrics
- add logger to log errors
- add function to trigger a duration interval between 2 collect metrics
- fix use unic prom metrics for all component
- fix encoding info text
- fix status duration
2023-03-08 16:11:09 +01:00
Nicolas JUHEL
a7b7657143 Merge pull request #153 from nabbar/fix_2023-03-01
Fix 2023-03-01

Package Config/Component AWS :
- rework monitoring for status config to use Golib Request health system & config
- if Enable is true on the health config, use a request instance for healtcheck, otherwise use the aws check as healthcheck

Package AWS/Config :
- adjust code for config request health

Package Request :
- expose healthcheck into interface

Bump dependencies
v1.10.2
2023-03-01 12:26:36 +01:00
nabbar
c72199d029 Fix 2023-03-01
Package Config/Component AWS :
- rework monitoring for status config to use Golib Request health system & config
- if Enable is true on the health config, use a request instance for healtcheck, otherwise use the aws check as healthcheck

Package AWS/Config :
- adjust code for config request health

Package Request :
- expose healthcheck into interface

Bump dependencies
2023-03-01 12:08:12 +01:00
Nicolas JUHEL
7a5650aad0 Merge pull request #152 from nabbar/fix_2023-02-28
Fix 2023-02

## Package Status:
  - Fix text formatting

## Package HTTPServer:
  - fix TLS check, config & loading for server

## Bump dependancies
  - dependencies based on go 1.20
  - change nutsdb owner repos
v1.10.1
2023-02-28 15:19:08 +01:00
nabbar
854c2fa6e7 Package Status:
- Fix text formatting

Package HTTPServer:
  - fix TLS check, config & loading for server

Bump dependancies
  - change nutsdb owner repos
2023-02-28 15:00:34 +01:00
Nicolas JUHEL
443935a407 Merge pull request #150 from nabbar/monitoring
Rework Monitoring, Prometheus, Status system

Package Monitoring :
- use packag dedicated to monitor component
- each monitor work as standalone server to monitor health
- collect metrics to export them to prometheus exporter

Package Prometheus :
- review to simplify use for API and not API metrics
- optimize code

Package Status :
- Rework to use Monitor package
- Rework to use native json / text Marshaller interface

Context :
- rework context config (context var) to use sync map and sync RWMutex (WORM)
- move gin context to dedicated sub package (dependancies of logger make circular dependencies)
- optimize code

Config :
- rework to optimize sync / collect of component
- rework status to monitor
- remove monitor managment from config to each component
- add a func to set default logger to implement inherit default logger options
- optimize code

IOUtils :
- isolate logger / closer interface as a usable & public interface & instance
- this interface / instance allow to collect io.closer over a context to close all if context is done

Logger :
- rework to use context.config map
- rework to use ioutils closer
- rework to allow options to inherit a default options, or the last version of options
- optimize code

Size :
- Add package Size to calculate and manipulate size Byte or bit
- Add encoding : Text/JSON/Yaml/Toml...
- Add option to défine default unit : Byte or bit

Other :
- adjust following code
- optimize code
- limit use of atomic value
- rework to use RWMutex instead of sync.Mutex to maximize capabilities of read instead of write
- remove 32bit build for CI/CD
- add darwin/arm64 build for CI/CD

Bump Dependencies
v1.10.0
2023-02-21 16:46:31 +01:00
nabbar
e3239db998 Rework Monitoring, Prometheus, Status system
Package Monitoring :
- use packag dedicated to monitor component
- each monitor work as standalone server to monitor health
- collect metrics to export them to prometheus exporter

Package Prometheus :
- review to simplify use for API and not API metrics
- optimize code

Package Status :
- Rework to use Monitor package
- Rework to use native json / text Marshaller interface

Context :
- rework context config (context var) to use sync map and sync RWMutex (WORM)
- move gin context to dedicated sub package (dependancies of logger make circular dependencies)
- optimize code

Config :
- rework to optimize sync / collect of component
- rework status to monitor
- remove monitor managment from config to each component
- add a func to set default logger to implement inherit default logger options
- optimize code

IOUtils :
- isolate logger / closer interface as a usable & public interface & instance
- this interface / instance allow to collect io.closer over a context to close all if context is done

Logger :
- rework to use context.config map
- rework to use ioutils closer
- rework to allow options to inherit a default options, or the last version of options
- optimize code

Size :
- Add package Size to calculate and manipulate size Byte or bit
- Add encoding : Text/JSON/Yaml/Toml...
- Add option to défine default unit : Byte or bit

Other :
- adjust following code
- optimize code
- limit use of atomic value
- rework to use RWMutex instead of sync.Mutex to maximize capabilities of read instead of write
- remove 32bit build for CI/CD
- add darwin/arm64 build for CI/CD

Bump Dependencies
2023-02-21 16:30:40 +01:00
nabbar
771a66eff4 - Package AWS : fix error with context while cloning
- Package Errors : fix error with slice of error not returning all parent
- Package Logger :
    - fix log error slice
    - following bump hclog : implement GetLevel + adjust SetLevel to report different between hc log debug & trace following EnableTrace in options
- Bump dependencies
v1.9.20
2022-12-07 12:41:17 +01:00
Nicolas JUHEL
170899fb4f Merge pull request #149 from nabbar/pkg_aws_lock
Package AWS - Lock

## Package AWS :
  - Bucket ADD CreateWithLock : create bucket with Object Lock enable
  - Bucket ADD GetLock : retrieve object lock bucket configuration
  - Bucket ADD SetLock : set object lock bucket configuration
  - Object ADD GetLegalHold : retrieve object lock status for object/version
  - Object ADD SetLegalHold : set object lock flag for object/version
  - Interface ADD SetClientS3 : replace current client S3 with a new one given as parameters
  - Interface ADD SetClientIam : replace current client IAM with a new one given as parameters
  - Interface ADD HTTPCli : return the current setting of func to return a http.client
  - Interface ADD mutex : lock any accessing struct properties to prevent race

## Other :
  - bump dependencies
v1.9.19
2022-12-02 09:56:46 +01:00
nabbar
980012d921 Package AWS :
- Bucket ADD CreateWithLock : create bucket with Object Lock enable
  - Bucket ADD GetLock : retrieve object lock bucket configuration
  - Bucket ADD SetLock : set object lock bucket configuration
  - Object ADD GetLegalHold : retrieve object lock status for object/version
  - Object ADD SetLegalHold : set object lock flag for object/version
  - Interface ADD SetClientS3 : replace current client S3 with a new one given as parameters
  - Interface ADD SetClientIam : replace current client IAM with a new one given as parameters
  - Interface ADD HTTPCli : return the current setting of func to return a http.client
  - Interface ADD mutex : lock any accessing struct properties to prevent race

Other :
  - bump dependencies
2022-12-02 09:35:29 +01:00
Nicolas JUHEL
70f9a14d1e Merge pull request #148 from nabbar/fix_status
Fix Bug & Optimization

Config Component :
- Change reloading component method : try to reload all component and store errors, report list of errors but don't break the reloading process

Package Logger :
- Fix bug with entry logger filtering

Package Viper:
- reword message log before reloading config file (watchFS)
- add message log after reloading config file (watchFS)

Package Status :
- Fix DATA Race with status/info
- Add component key into log error message for health
- Add component key into function to use it into info (name)
- Reword health message : no OK/KO (still into status info), add error message reporting

Package httpserver :
- status info : apply update status component, use key in name
- status info : optimize code

Package Request :
- Fix error in url path operation
- Status info : optimize code
- Status info : apply update component, add endpoint hostname with name

Package Static :
- apply status component update

Other:
- Bump dependencies
v1.9.18
2022-11-17 17:18:04 +01:00
nabbar
52f0d6fa04 Config Component :
- Change reloading component method : try to reload all component and store errors, report list of errors but don't break the reloading process

Package Logger :
- Fix bug with entry logger filtering

Package Viper:
- reword message log before reloading config file (watchFS)
- add message log after reloading config file (watchFS)

Package Status :
- Fix DATA Race with status/info
- Add component key into log error message for health
- Add component key into function to use it into info (name)
- Reword health message : no OK/KO (still into status info), add error message reporting

Package httpserver :
- status info : apply update status component, use key in name
- status info : optimize code

Package Request :
- Fix error in url path operation
- Status info : optimize code
- Status info : apply update component, add endpoint hostname with name

Package Static :
- apply status component update

Other :
- Bump dependencies
2022-11-17 16:54:51 +01:00
Nicolas JUHEL
aedad09eb7 Merge pull request #147 from nabbar/fix_os_path_operation
- Fix local path operations by using a wrong package (diff between package path and path/filepath)
- Bump dependencies
v1.9.17
2022-11-15 14:54:04 +01:00