Commit Graph

41 Commits

Author SHA1 Message Date
nabbar
c72199d029 Fix 2023-03-01
Package Config/Component AWS :
- rework monitoring for status config to use Golib Request health system & config
- if Enable is true on the health config, use a request instance for healtcheck, otherwise use the aws check as healthcheck

Package AWS/Config :
- adjust code for config request health

Package Request :
- expose healthcheck into interface

Bump dependencies
2023-03-01 12:08:12 +01:00
nabbar
e3239db998 Rework Monitoring, Prometheus, Status system
Package Monitoring :
- use packag dedicated to monitor component
- each monitor work as standalone server to monitor health
- collect metrics to export them to prometheus exporter

Package Prometheus :
- review to simplify use for API and not API metrics
- optimize code

Package Status :
- Rework to use Monitor package
- Rework to use native json / text Marshaller interface

Context :
- rework context config (context var) to use sync map and sync RWMutex (WORM)
- move gin context to dedicated sub package (dependancies of logger make circular dependencies)
- optimize code

Config :
- rework to optimize sync / collect of component
- rework status to monitor
- remove monitor managment from config to each component
- add a func to set default logger to implement inherit default logger options
- optimize code

IOUtils :
- isolate logger / closer interface as a usable & public interface & instance
- this interface / instance allow to collect io.closer over a context to close all if context is done

Logger :
- rework to use context.config map
- rework to use ioutils closer
- rework to allow options to inherit a default options, or the last version of options
- optimize code

Size :
- Add package Size to calculate and manipulate size Byte or bit
- Add encoding : Text/JSON/Yaml/Toml...
- Add option to défine default unit : Byte or bit

Other :
- adjust following code
- optimize code
- limit use of atomic value
- rework to use RWMutex instead of sync.Mutex to maximize capabilities of read instead of write
- remove 32bit build for CI/CD
- add darwin/arm64 build for CI/CD

Bump Dependencies
2023-02-21 16:30:40 +01:00
nabbar
771a66eff4 - Package AWS : fix error with context while cloning
- Package Errors : fix error with slice of error not returning all parent
- Package Logger :
    - fix log error slice
    - following bump hclog : implement GetLevel + adjust SetLevel to report different between hc log debug & trace following EnableTrace in options
- Bump dependencies
2022-12-07 12:41:17 +01:00
nabbar
980012d921 Package AWS :
- Bucket ADD CreateWithLock : create bucket with Object Lock enable
  - Bucket ADD GetLock : retrieve object lock bucket configuration
  - Bucket ADD SetLock : set object lock bucket configuration
  - Object ADD GetLegalHold : retrieve object lock status for object/version
  - Object ADD SetLegalHold : set object lock flag for object/version
  - Interface ADD SetClientS3 : replace current client S3 with a new one given as parameters
  - Interface ADD SetClientIam : replace current client IAM with a new one given as parameters
  - Interface ADD HTTPCli : return the current setting of func to return a http.client
  - Interface ADD mutex : lock any accessing struct properties to prevent race

Other :
  - bump dependencies
2022-12-02 09:35:29 +01:00
nabbar
eb8097fad8 Fix local path operations by using a wrong package (diff between package path and path/filepath)
Bump dependencies
2022-11-15 14:37:49 +01:00
Nicolas JUHEL
5187e7eae2 Package AWS :
- User : Add/update policy managment (list, walk, attach, detach)
- Group : Add/update policy managment (list, walk, attach, detach)
- Role : Add/update policy managment (list, walk, attach, detach)
- Policy : add capability to not list default verison policy
2022-10-28 15:16:53 +02:00
Nicolas JUHEL
7139797df2 Package AWS :
- Fix bug while check key len : min size was 3, AWS reference API S3 say min len is 1
Update :
- Bump dependencies
2022-10-25 15:31:31 +02:00
Nicolas JUHEL
573af6ce3a Package AWS :
- Config Model : add a config model with a golib RouerStatus Config Model to use a AWS connection for API with a request status health check
  - Config Interface : add function GetAccessKey to retrieve the accesskey value currently used into the current connection
  - function Walk (Object, version, ...) : fix bug with pointer of string not initialized

Package Config :
  - interface Component : add status router pointer into the Init function to allow used a global router status for all component status registration as router status component

Package Status :
  - move Status Config as an sub package of Package Status

Package Errors :
  - add CamelCase const

Package Logger :
  - fix following bump dependencies

Global :
  - bump dependencies
  - change init of errors files : change function never call vy panic to prevent an error code collision
2022-09-13 12:31:04 +02:00
Nicolas JUHEL
55aa0c3637 Package AWS:
- Bucket : add function Walk based on list but run a given function for each found bucket
- Object : add function Walk & WalkPrefix based on ListPrefix to run a given function for each returned object
- Object : add function WalkVersion & WalkVersionPrefix based on ListVersion to run a given function for each returned object version
Package logger:
- Fix bug with compat : a nil error not clean when sent to logger
Package Context:
- Fix bug with context/config : the load key return an atomic value instead of the interface stored
2022-08-10 11:11:34 +02:00
Nicolas JUHEL
f0a5a894f2 Pakcage AWS :
+ Add Tag function for object
  + Add Lock function for object
  + Add Attributes function for object
  + Add lifecycle function for bucket
  + Add function to get client in AWS interface
Package Certificates :
  + Add clean certifs function in config
Bump dependencies
2022-08-03 12:13:15 +02:00
Nicolas JUHEL
90ceb19d2f - Chg :
- Package AWS : extend aws helper to manage accessKey
	- Package Cobra : refactor configure function to allow use independant part of the command
	- Package ldap : fix error
	- Package config
		- interface : add shutdown method
		- components:
			- request : syntax error in default json
			- mail : syntax error default json
			- ldap :
				- allow to access loaded config
				- mutex lock circular
			- tls : allow to access loaded config
	- Package request :
		- add error managment
		- fix error with circular mutex
		- fix bug with empty response body
		- fix some other errors
	- Package status : default label in component default config
	- Bump dependancies
	- Bump yaml to v3 (in waiting fix for CVE-2022-28948)
	- force dependancies gogo/protobuff to v1.3.2+
2022-07-25 16:00:36 +02:00
Nicolas JUHEL
01308cc8f2 Optimize errors for package aws & logger
Bump dependancies
2022-04-28 16:19:55 +02:00
Nicolas JUHEL
48b47ec97e Package AWS :
- fix missing commit : version function
- fix missing commut : deleteall function
- add function to load bucket replication configuration
- add AWS feature to create new client for a config based on current client
- add Config feature to change the credentials of current config
2022-04-22 14:37:59 +02:00
Nicolas JUHEL
017c24f2df - Package AWS : fix missing field for clone function
- Bump dependancies
2022-04-21 15:51:08 +02:00
Nicolas JUHEL
f073bca2d3 Package AWS :
- Fix issue #136 :
  - add function ForceSignerOptions to pass options funct to signer
  - if not option exist, use default signer v4, else specify the options function list to the new signer
- fix error with multiple ptr identifier on model
- add function to list and cancel multipart uploads
- add function to list, get and delete version object
- add function to delete all objects following the S3 API DeleteObjects function (to send array of object key / version) in one request

Package Config :
- modify component interface :
    - add cpt as custom function parameters to prevent new component still not stored when running custom function
    - re-order mutex on start / reload process

Change :
- Bump dependancies
2022-04-20 12:21:26 +02:00
Nicolas JUHEL
b1f5e42db7 Package HttpCli :
- slipt package between pure httpcli and request to become new package Request
- Add options to httpcli allowed to create an http client based on options

Package AWS :
- add option for delete object : check boolean to make or not the check (head) before delete
- add exclusion of NoSuchKey error if check is false

Package Request :
- integer option capabilities to configure the request inteface
- use httpcli option to get httpcli (keep custom function to get an *http.client)
- extend function to mode manage the request
2022-04-01 13:33:21 +02:00
Nicolas JUHEL
6645bcc05e Refactor Package HTTPCli:
- Reword Const (timeout + errors)
- Remove old request interface
- Add new request interface to make a request, send it, parse a result model and check response status
- Add function to skip DNS resolution for http client connection

Change
- Bump dependancies
2022-03-31 13:11:00 +02:00
Nicolas JUHEL
312ce42dc3 Fix issue #132 2022-03-29 08:36:19 +02:00
Nicolas JUHEL
425962f8c0 Add packages :
- package databse : use gorm lib to  implement this helpers
- package fpt client : expose a helper to post / read file from ftp server

Add component into package Config :
- add AWS custom / standard
- add database
- add Nats cluster
- add NutsDB cluster
- add smtp client
- fix some errors into tls, http component

Fix / Update :
- bump dependancies
- adapt some fix into aws config, smtp, http server, nats, nutsdb, status packages
- add gorm logger into logger pacakge
- fix issue #125
- fix issue #126
- fix issue #127
2022-03-16 15:08:33 +01:00
Nicolas JUHEL
7fbcf39c6a AWS : add method to update metadata (copy object)
AWS : add method for website object (update redirection & acl)
AWS : fix bug with missing content-type (all defined as application/octet-stream)
2022-01-17 18:06:28 +01:00
Nicolas JUHEL
74be9a73c4 AWS Test suite : fix cancel context 2022-01-14 13:10:52 +01:00
Nicolas JUHEL
10b2d5e06b - Add Bucket ACL
- Add Sub method for Set Bucket ACL to simplify small call
2022-01-13 17:29:56 +01:00
Nicolas JUHEL
032ddb30a4 PKG AWS :
- add bucket website configuration method
  - add bucket CORS configuration method
  - chg bucket method structure file
  - bump ginkgo to v2

PKG Logger :
  - bump ginkgo to v2

Change :
  - bump dependancies
  - change unit test command to be compliance with ginkgo v2
2022-01-13 10:45:43 +01:00
Nicolas JUHEL
dbb443eb65 - Fix issue #94
- Fix linter
- Remove useless code/deadcode
2021-04-13 14:21:07 +02:00
Nicolas JUHEL
1e02472227 Update error constant & unify file error.go name 2021-04-12 17:57:24 +02:00
Nicolas JUHEL
db1b2035d3 Fix error in CompareUpdate : policy version document is url encoded... 2021-03-09 18:04:03 +01:00
Nicolas JUHEL
5cff4d62dc Fix error in CompareUpdate : calling policy update with policy name instead of policy arn 2021-03-09 17:41:26 +01:00
Nicolas JUHEL
52b661f251 Packages AWS :
- Fix List Policy Version : if marker empty don't send it

Upd :
- Bump dependancies
2021-03-09 16:49:31 +01:00
Nicolas JUHEL
456ddd0c93 Fix Package AWS :
- Fix function update (use Version function instead of implement into the update function)
- Add func VersionList to list all version id with contents
- Add func VersionAdd to add new version into the policy as default version
- Add func VersionDel to delete one version of the policy
2021-03-05 16:10:38 +01:00
Nicolas JUHEL
5e8e0679cf Fix AWS : missing new function in interface 2021-03-05 12:32:22 +01:00
Nicolas JUHEL
db309c5a1b Package AWS:
- Policy : add function Get to retrieve Policy Type with all information of the policy
- Policy : add function GetVersion to retrieve PolicyVersion Type and policy document
- Policy : add function CompareUpdate to retrieve the policy, compare doc and if different update the policy with the new doc
2021-03-05 11:17:49 +01:00
Nicolas JUHEL
1249f319bc Bump dependancies
- AWS SDK to release v1.0.0
  - other dependancies
Fix Packages :
  - AWS : fix validator function, rules, config model
  - Certificates : fix func NewFrom
  - HTTPServer: fix IsRunning
Fix other :
  - Fix CI/CD job to prevent alert on files modified
  - Fix missing licence comment header
2021-01-25 08:33:18 +01:00
Nicolas JUHEL
92d8d8bf26 Fix bytes reader instead of io reader 2020-12-11 15:28:47 +01:00
Nicolas JUHEL
cd72f5f0eb - Refactor context for ldap + dialer
- Add new package for context :
  * create a context based on gin.context struct
  * create a isolated context from parent context
2020-12-09 06:24:28 +01:00
Nicolas JUHEL
964586fa47 Bump dependancies 2020-12-09 04:28:03 +01:00
Nicolas JUHEL
cbc0a35ea7 Fix Issue #73 and other bugs:
- Boolean value for EndpointOption
- Optimize CustomConfig IsHTTPS function
- Bucket Creation return
- Bucket Create contraint Region Location
- CustomConfig missing Signing Region
- some AWS errors

Fix Github/Action :
- Add some linter/test as blocked task
2020-11-25 11:51:20 +01:00
Nicolas JUHEL
0290b217c5 Bump aws to release v0.29.0 2020-11-02 10:30:45 +01:00
Nicolas JUHEL
3671701836 Fix linter + working dir for test 2020-10-02 12:51:23 +02:00
Nicolas JUHEL
94f90d7e22 Bump to sdk aws go v2 at release v0.26.0...
Bump dependancies
Refactor / Fix aws to work with sdk-aws-go-v2 at release v0.26.0...
2020-10-02 12:43:08 +02:00
Nicolas JUHEL
670d28670e AWS : export model for Custom Config 2020-09-28 16:17:23 +02:00
Nicolas JUHEL
8def6ec437 PKG Aws : add aws package to simplify uses of aws sdk + aimplify custom config for custom endpoint or region like use of minio server 2020-09-11 13:56:17 +02:00