zishuo/golib
1
0
Fork 0
mirror of https://github.com/nabbar/golib.git synced 2025-09-26 20:01:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

README:

Browse Source 
- Add some README file to give missing documentations or update existing documentation file

Package Archive:
- Add some comments to godoc information
- Moving NopWriterCloser interface to ioutils package

Package IOUtils:
- New package NopWriterCloser to implement interfac like NopReader

Package Database:
- KVMap: fix missing function following update of kvdriver

Package Duration:
- Rename BDD testing

Package Context/Gin:
- Moving function New between model & interface file

Package AWS:
- rework Walk function to use more generic with standard walk caller function
- func walk will now no more return and include error (can be catched into the given func)
- func walk will now return a bool to continue or stop the loop
- func walk with many input function will now stop when all given function return false
- func walk will now return error only about main process and not given function

Package errors:
- Add interface error into interface Error

Package IOUtils:
- Moving IOWrapper as subPackage and optimize process + allow thread safe
This commit is contained in:
nabbar
2025-05-24 22:28:37 +02:00
parent 0f4ac42db9
commit 9e8179374b
61 changed files with 9022 additions and 1074 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@@ -1,14 +1,16 @@
![Go](https://github.com/nabbar/golib/workflows/Go/badge.svg)
# golib : custom lib for go
snyk project : https://app.snyk.io/org/nabbar/project/2f55a2b8-6015-4db1-b859-c2bc3b7548a7
[![Known Vulnerabilities](https://snyk.io/test/github/nabbar/golib/badge.svg)](https://snyk.io/test/github/nabbar/golib)
[![Go](https://github.com/nabbar/golib/workflows/Go/badge.svg)](https://github.com/nabbar/golib)
[![GoDoc](https://pkg.go.dev/badge/github.com/nabbar/golib)](https://pkg.go.dev/github.com/nabbar/golib)
[![Go Report Card](https://goreportcard.com/badge/github.com/nabbar/golib)](https://goreportcard.com/report/github.com/nabbar/golib)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
## using in source code
first get the source dependancies
```shell script
go get -u github.com/nabbar/golib/...
go get github.com/nabbar/golib/...
```
second, import the needed lib in your code
@@ -17,8 +19,30 @@ import "github.com/nabbar/golib/version"
```
## Details of packages :
* [package archive](archive/README.md)
* [package artifact](artifact/README.md)
* [package atomic](atomic/README.md)
* [package aws](aws/README.md)
* [package certificates](certificates/README.md)
* [pacakge cobra](cobra/README.md)
* [package config](config/README.md)
* [package console](console/README.md)
* [package context](context/README.md)
* [package database](database/README.md)
* [pacakge duration](duration/README.md)
* [package encoding](encoding/README.md)
* [package errors](errors/README.md)
* [package file](file/README.md)
* [package ftpclient](ftpclient/README.md)
* [package httpcli](httpcli/README.md)
* [package httpserver](httpserver/README.md)
* [package ioutil](ioutil/README.md)
* [package ldap](ldap/README.md)
* [package logger](logger/README.md)
* [package mail](mail/README.md)
* [package mailer](mailer/README.md)
* [package mailPooler](mailPooler/README.md)
* [package monitor](monitor/README.md)
* [package network](network/README.md)
* [package password](password/README.md)
* [package router](router/README.md)
@@ -29,5 +53,5 @@ import "github.com/nabbar/golib/version"
# Build tags
To build static, pure go, some packages need to use tags osusergo and netgo, like this
```bash
go build -a -tags osusergo,netgo -installsuffix cgo ...
go build -a -tags "osusergo netgo" -installsuffix cgo ...
```

626
archive/README.md
View File

@@ -1,114 +1,576 @@
# Package Archive
This package will try to do all uncompress/unarchive and extract one file or all file.
This package will expose 2 functions :
- ExtractFile : for one file extracted
- ExtractAll : to extract all file
# Golib Archive
## Example of implementation
**Golang utilities for archive and compression management**
### Example of one file extracted
---
To use `ExtractFile` function, you will need this parameters :
- `src` : is the source file into a `ioutils.FileProgress` struct to expose an `os.File` pointer with interface `io.WriteTo`, `io.ReadFrom`, `io.ReaderAt` and progress capabilities
- `dst` : is the source file into a `ioutils.FileProgress` struct to expose an `os.File` pointer with interface `io.WriteTo`, `io.ReadFrom`, `io.ReaderAt` and progress capabilities
- `filenameContain` : is a `string` to search in the file name to find it and extract it. This string will be search into the `strings.Contains` function
- `filenameRegex` : is a regex pattern `string` to search in the file name any match and extract it. This string will be search into the `regexp.MatchString` function
## Overview
You can implement this function as it. This example is available in [`test/test-archive`](../test/test-archive/main.go) folder.
```go
import (
"io"
"io/ioutil"
This package provides tools to manipulate archive files (ZIP, TAR, etc.), compress/decompress streams (GZIP, BZIP2, etc.), and helpers to simplify common file operations in Go.
"github.com/nabbar/golib/archive"
"github.com/nabbar/golib/ioutils"
)
---
const fileName = "fullpath to my archive file"
## Installation
func main() {
var (
src ioutils.FileProgress
dst ioutils.FileProgress
err errors.Error
)
Add the dependency to your Go project:
// register closing function in output function callback
defer func() {
if src != nil {
_ = src.Close()
}
if dst != nil {
_ = dst.Close()
}
}()
// open archive with a ioutils NewFileProgress function
if src, err = ioutils.NewFileProgressPathOpen(fileName); err != nil {
panic(err)
}
// open a destination with a ioutils NewFileProgress function, as a temporary file
if dst, err = ioutils.NewFileProgressTemp(); err != nil {
panic(err)
}
// call the extract file function
if err = archive.ExtractFile(tmp, rio, "path/to/my/file/into/archive", "archive name regex"); err != nil {
panic(err)
}
}
```shell
go get github.com/nabbar/golib/archive
```
Or in your `go.mod`:
### Example of all files extracted
To use `ExtractAll` function, you will need this parameters :
- `src` : is the source file into a `ioutils.FileProgress` struct to expose an `os.File` pointer with interface `io.WriteTo`, `io.ReadFrom`, `io.ReaderAt` and progress capabilities
- `originalName` : is a `string` to define the originalName of the archive. This params is used to create a unique file created into the outputPath if the archive is not an archive or just compressed with a not catalogued compress type like gzip or bzip2.
- `outputPath` : is a `string` to precise the destination output directory (full path). All extracted file will be extracted with this directory as base of path.
- `defaultDirPerm` : is a `os.FileMode` to precise the permission of directory. This parameters is usefull if the output directory is not existing.
You can implement this function as it. This example is available in [`test/test-archive-all`](../test/test-archive-all/main.go) folder.
```go
require github.com/nabbar/golib/archive vX.Y.Z
```
---
## Package Structure
- **archive**: Create, extract, and list archives (ZIP, TAR, etc.), see [Subpackage `archive`](#archive-subpackage) for details.
- **compress**: Compress/decompress files or streams (GZIP, BZIP2, etc.), see [Subpackage `compress`](#compress-subpackage) for details.
- **helper**: Utility functions for file and stream manipulation, including a unified interface for compression and decompression, see [Subpackage `helper`](#helper-subpackage) for details.
---
## Quick Start
### Extract any type of Archive / Compressed File
```go
import "github.com/nabbar/golib/archive"
in, err := os.Open("archive.zip")
if err != nil {
panic(err)
}
defer in.Close()
err := archive.ExtractAll("archive.zip", "archive", "destination_folder")
if err != nil {
panic(err)
}
```
### Detect Archive Type
```go
package main
import (
"io"
"io/ioutil"
"fmt"
"os"
"github.com/nabbar/golib/archive"
"github.com/nabbar/golib/ioutils"
libarc "github.com/nabbar/golib/archive"
arcarc "github.com/nabbar/golib/archive/archive"
arctps "github.com/nabbar/golib/archive/archive/types"
)
const fileName = "fullpath to my archive file"
func main() {
var (
src ioutils.FileProgress
tmp ioutils.FileProgress
out string
in *os.File
alg arcarc.Algorithm
arc arctps.Reader
lst []string
err error
)
// open archive with a ioutils NewFileProgress function
if src, err = ioutils.NewFileProgressPathOpen(fileName); err != nil {
in, err = os.Open("archive.zip")
if err != nil {
panic(err)
}
// create an new temporary file to use his name as output path
if tmp, err = ioutils.NewFileProgressTemp(); err != nil {
panic(err)
} else {
// get the filename of the temporary file
out = tmp.FilePath()
defer in.Close()
// close the temporary file will call the delete temporary file
_ = tmp.Close()
alg, arc, _, err = libarc.DetectArchive(in)
if err != nil {
panic(err)
}
if err = archive.ExtractAll(src, path.Base(src.FilePath()), out, 0775); err != nil {
defer arc.Close()
fmt.Println("Archive type detected:", alg.String())
lst, err = arc.List()
if err != nil {
panic(err)
}
for _, f := range lst {
fmt.Println("File in archive:", f)
}
}
```
### Detect Compression Type
```go
package main
import (
"fmt"
"io"
"os"
libarc "github.com/nabbar/golib/archive"
arccmp "github.com/nabbar/golib/archive/compress"
)
func main() {
var (
in *os.File
alg arccmp.Algorithm
rdr io.ReadCloser
err error
)
in, err = os.Open("archive.gz")
if err != nil {
panic(err)
}
defer in.Close()
alg, rdr, err = libarc.DetectCompression(in)
if err != nil {
panic(err)
}
defer rdr.Close()
fmt.Println("Compression type detected:", alg.String())
// Read the decompressed data
_, err = io.Copy(io.Discard, rdr)
if err != nil {
panic(err)
}
}
```
---
# `archive` Subpackage
The `archive` subpackage provides tools to manage archive files (such as ZIP and TAR), including creation, extraction, listing, and detection of archive types. It is designed to work with multiple files and directories, and supports both reading and writing operations.
## Features
- Create archives (ZIP, TAR) from files or directories
- Extract archives to a destination folder
- List the contents of an archive
- Detect archive type from a stream
- Read and write archives using unified interfaces
- Support for streaming and random access (when possible)
---
## Main Types
- **Algorithm**: Enum for supported archive formats (`Tar`, `Zip`, `None`)
- **Reader/Writer**: Interfaces for reading and writing archive entries
---
## API Overview
### Detect Archive Type
Detect the archive algorithm from an `io.ReadCloser` and get a compatible reader.
```go
import (
"os"
"github.com/nabbar/golib/archive/archive"
)
file, _ := os.Open("archive.tar")
alg, reader, closer, err := archive.Detect(file)
if err != nil {
// handle error
}
defer closer.Close()
fmt.Println("Archive type:", alg.String())
```
### Create an Archive
Create a new archive (ZIP or TAR) and add files/directories.
```go
import (
"os"
"github.com/nabbar/golib/archive/archive"
)
out, _ := os.Create("myarchive.zip")
writer, err := archive.Zip.Writer(out)
if err != nil {
// handle error
}
// Use writer to add files/directories (see Writer interface)
```
### Extract an Archive
Extract all files from an archive to a destination.
```go
import (
"os"
"github.com/nabbar/golib/archive/archive"
)
in, _ := os.Open("myarchive.tar")
alg, reader, closer, err := archive.Detect(in)
if err != nil {
// handle error
}
defer closer.Close()
// Use reader to walk through files and extract them
```
### List Archive Contents
List all files in an archive.
```go
import (
"os"
"github.com/nabbar/golib/archive/archive"
)
in, _ := os.Open("archive.zip")
alg, reader, closer, err := archive.Detect(in)
if err != nil {
// handle error
}
defer closer.Close()
files, err := reader.List()
if err != nil {
// handle error
}
for _, f := range files {
fmt.Println(f)
}
```
---
## Error Handling
All functions return an `error` value. Always check and handle errors when working with archives.
---
## Notes
- The archive type is detected by reading the file header.
- For ZIP archives, random access is required (`io.ReaderAt`).
- For TAR archives, streaming is supported.
- Use the `Walk` method (if available) to iterate over archive entries efficiently.
---
For more details, refer to the GoDoc or the source code in `archive/archive`.
---
# `compress` Subpackage
The `compress` subpackage provides utilities to handle compression and decompression of single files or data streams using various algorithms. It supports GZIP, BZIP2, LZ4, XZ, and offers a unified interface for detection, reading, and writing compressed data.
## Features
- Detect compression algorithm from a stream
- Compress and decompress data using GZIP, BZIP2, LZ4, XZ
- Unified `Reader` and `Writer` interfaces for all supported algorithms
- Simple API for marshaling/unmarshaling algorithm types
- Support for both streaming and random access (when possible)
---
## Supported Algorithms
- `None` (no compression)
- `Gzip`
- `Bzip2`
- `LZ4`
- `XZ`
---
## Main Types
- **Algorithm**: Enum for supported compression formats
- **Reader/Writer**: Interfaces for reading from and writing to compressed streams
---
## API Overview
### Detect Compression Algorithm
Detect the compression algorithm from an `io.Reader` and get a compatible decompressor.
```go
import (
"os"
"github.com/nabbar/golib/archive/compress"
)
file, _ := os.Open("file.txt.gz")
alg, reader, err := compress.Detect(file)
if err != nil {
// handle error
}
defer reader.Close()
fmt.Println("Compression type:", alg.String())
```
### Compress Data
Create a compressed file using a specific algorithm.
```go
import (
"os"
"github.com/nabbar/golib/archive/compress"
)
in, _ := os.Open("file.txt")
out, _ := os.Create("file.txt.gz")
writer, err := compress.Gzip.Writer(out)
if err != nil {
// handle error
}
defer writer.Close()
_, err = io.Copy(writer, in)
```
### Decompress Data
Decompress a file using the detected algorithm.
```go
import (
"os"
"github.com/nabbar/golib/archive/compress"
)
in, _ := os.Open("file.txt.gz")
alg, reader, err := compress.Detect(in)
if err != nil {
// handle error
}
defer reader.Close()
out, _ := os.Create("file.txt")
_, err = io.Copy(out, reader)
```
### Parse and Marshal Algorithm
Convert between string and `Algorithm` type.
```go
alg := compress.Parse("gzip")
fmt.Println(alg.String()) // Output: gzip
```
---
## Error Handling
All functions return an `error` value. Always check and handle errors when working with compression streams.
---
## Notes
- The algorithm is detected by reading the file header.
- For writing, use the `Writer` method of the chosen algorithm.
- For reading, use the `Reader` method or `Detect` for auto-detection.
- The package supports both marshaling to/from text and JSON for the `Algorithm` type.
---
For more details, refer to the GoDoc or the source code in `archive/compress`.
---
# `helper` Subpackage
The `helper` subpackage provides advanced utilities to simplify compression and decompression workflows for files and streams. It offers a unified interface to handle both compression and decompression using various algorithms, and can be used as a drop-in `io.ReadWriteCloser` for flexible data processing.
## Features
- Unified `Helper` interface for compressing and decompressing data
- Supports all algorithms from the `compress` subpackage (GZIP, BZIP2, LZ4, XZ, None)
- Can be used as a reader or writer, depending on the operation
- Handles both file and stream sources/destinations
- Thread-safe buffer management for streaming operations
- Error handling for invalid sources or operations
---
## Main Types
- **Helper**: Interface implementing `io.ReadWriteCloser` for compression/decompression
- **Operation**: Enum (`Compress`, `Decompress`) to specify the desired operation
---
## API Overview
### Create a Helper
Create a new helper for compression or decompression, using a specific algorithm and source (reader or writer).
```go
package main
import (
"io"
"os"
"strings"
"github.com/nabbar/golib/archive/compress"
"github.com/nabbar/golib/archive/helper"
)
func main() {
// For compression (writing compressed data)
out, err := os.Create("file.txt.gz")
if err != nil {
panic(err)
}
defer out.Close()
h, err := helper.NewWriter(compress.Gzip, helper.Compress, out)
if err != nil {
panic(err)
}
defer h.Close()
_, err = io.Copy(h, strings.NewReader("data to compress"))
if err != nil {
panic(err)
}
// For decompression (reading decompressed data)
in, err := os.Open("file.txt.gz")
if err != nil {
panic(err)
}
defer in.Close()
h, err = helper.NewReader(compress.Gzip, helper.Decompress, in)
if err != nil {
panic(err)
}
defer h.Close()
_, err = io.Copy(os.Stdout, h)
if err != nil {
panic(err)
}
// For compression (writing compressed data)
out, err := os.Create("file.txt.gz")
if err != nil {
panic(err)
}
defer out.Close()
h, err := helper.NewWriter(compress.Gzip, helper.Compress, out)
if err != nil {
panic(err)
}
defer h.Close()
_, err = io.copy(h, strings.NewReader("data to compress"))
if err != nil {
panic(err)
}
// For decompression (reading decompressed data)
in, err := os.Open("file.txt.gz")
if err != nil {
panic(err)
}
defer in.Close()
h, err = helper.NewReader(compress.Gzip, helper.Decompress, in)
if err != nil {
panic(err)
}
defer h.Close()
_, err = io.Copy(os.Stdout, h)
if err != nil {
panic(err)
}
}
```
### Use as Reader or Writer
You can use the helper as a standard `io.Reader`, `io.Writer`, or `io.ReadWriteCloser` depending on the operation.
```go
// Compress data from a file to another file
in, _ := os.Open("file.txt")
out, _ := os.Create("file.txt.gz")
h, _ := helper.New(compress.Gzip, helper.Compress, out)
defer h.Close()
io.Copy(h, in)
// Decompress data from a file to another file
in, _ := os.Open("file.txt.gz")
out, _ := os.Create("file.txt")
h, _ := helper.New(compress.Gzip, helper.Decompress, in)
defer h.Close()
io.Copy(out, h)
```
### Error Handling
All helper constructors and methods return errors. Typical errors include invalid source, closed resource, or invalid operation.
---
## Notes
- The `Helper` interface adapts to the source type: use a `Reader` for decompression, a `Writer` for compression.
- The `New` function auto-detects the source type and operation.
- Thread-safe buffers are used internally for streaming and chunked operations.
- For advanced use, you can directly use `NewReader` or `NewWriter` to specify the direction.
---
For more details, refer to the GoDoc or the source code in `archive/helper`.

2
archive/archive/interface.go
View File

@@ -85,7 +85,7 @@ func Detect(r io.ReadCloser) (Algorithm, arctps.Reader, io.ReadCloser, error) {
if z, e := Zip.Reader(bfr); e != nil {
return None, nil, nil, e
} else {
return Zip, z, r, nil
return Zip, z, bfr, nil
}
default:

4
archive/compress/interface.go
View File

@@ -30,6 +30,7 @@ import (
"io"
)
// Parse is a convenience function to parse a string and return the corresponding Algorithm.
func Parse(s string) Algorithm {
var alg = None
if e := alg.UnmarshalText([]byte(s)); e != nil {
@@ -39,6 +40,8 @@ func Parse(s string) Algorithm {
}
}
// Detect is a convenience function to detect the compression algorithm used in
// the provided io.Reader and return the compression read closer associated.
func Detect(r io.Reader) (Algorithm, io.ReadCloser, error) {
var (
err error
@@ -55,6 +58,7 @@ func Detect(r io.Reader) (Algorithm, io.ReadCloser, error) {
}
}
// DetectOnly is a function that detects the compression algorithm used in the provided io.Reader
func DetectOnly(r io.Reader) (Algorithm, io.ReadCloser, error) {
var (
err error

42
archive/helper/compressor.go
View File

@@ -29,8 +29,27 @@ import (
"bytes"
"io"
"sync/atomic"
arccmp "github.com/nabbar/golib/archive/compress"
iotnwc "github.com/nabbar/golib/ioutils/nopwritecloser"
)
func makeCompressWriter(algo arccmp.Algorithm, src io.Writer) (h Helper, err error) {
wc, ok := src.(io.WriteCloser)
if !ok {
wc = iotnwc.New(src)
}
if wc, err = algo.Writer(wc); err != nil {
return nil, err
} else {
return &compressWriter{
dst: wc,
}, nil
}
}
type compressWriter struct {
dst io.WriteCloser
}
@@ -47,7 +66,28 @@ func (o *compressWriter) Close() error {
return o.dst.Close()
}
// compressor handles data compression in chunks.
func makeCompressReader(algo arccmp.Algorithm, src io.Reader) (h Helper, err error) {
rc, ok := src.(io.ReadCloser)
if !ok {
rc = io.NopCloser(src)
}
var (
buf = bytes.NewBuffer(make([]byte, 0))
wrt io.WriteCloser
)
wrt, err = algo.Writer(iotnwc.New(buf))
return &compressReader{
src: rc,
wrt: wrt,
buf: buf,
clo: new(atomic.Bool),
}, nil
}
type compressReader struct {
src io.ReadCloser
wrt io.WriteCloser

37
archive/helper/decompressor.go
View File

@@ -34,8 +34,25 @@ import (
"time"
arccmp "github.com/nabbar/golib/archive/compress"
iotnwc "github.com/nabbar/golib/ioutils/nopwritecloser"
)
func makeDeCompressReader(algo arccmp.Algorithm, src io.Reader) (h Helper, err error) {
rc, ok := src.(io.ReadCloser)
if !ok {
rc = io.NopCloser(src)
}
if rc, err = algo.Reader(rc); err != nil {
return nil, err
} else {
return &deCompressReader{
src: rc,
}, nil
}
}
const workBufSizeDeCompressWrite = 32 * 1024 // 32kB for buffer
type deCompressReader struct {
@@ -129,6 +146,26 @@ func (o *bufNoEOF) writeBuff(p []byte) (n int, err error) {
return o.b.Write(p)
}
func makeDeCompressWriter(algo arccmp.Algorithm, src io.Writer) (h Helper, err error) {
wc, ok := src.(io.WriteCloser)
if !ok {
wc = iotnwc.New(src)
}
return &deCompressWriter{
alg: algo,
wrt: wc,
buf: &bufNoEOF{
m: sync.Mutex{},
b: bytes.NewBuffer(make([]byte, 0)),
c: new(atomic.Bool),
},
clo: new(atomic.Bool),
run: new(atomic.Bool),
}, nil
}
type deCompressWriter struct {
alg arccmp.Algorithm
wrt io.WriteCloser

88
archive/helper/interface.go
View File

@@ -26,13 +26,9 @@
package helper
import (
"bytes"
"errors"
"io"
"sync"
"sync/atomic"
libarc "github.com/nabbar/golib/archive"
arccmp "github.com/nabbar/golib/archive/compress"
)
@@ -48,6 +44,10 @@ type Helper interface {
io.ReadWriteCloser
}
// New creates a new Helper instance based on the provided algorithm, operation, and source.
// Algo is the compression algorithm to use, which can be one of the predefined algorithms in the arccmp package.
// Operation defines the type of operation to perform with the archive helper. It can be either Compress or Decompress.
// The source can be an io.Reader for reading or an io.Writer for writing.
func New(algo arccmp.Algorithm, ope Operation, src any) (h Helper, err error) {
if r, k := src.(io.Reader); k {
return NewReader(algo, ope, r)
@@ -58,6 +58,9 @@ func New(algo arccmp.Algorithm, ope Operation, src any) (h Helper, err error) {
return nil, ErrInvalidSource
}
// NewReader creates a new Helper instance for reading data from the provided io.Reader.
// It uses the specified compression algorithm and operation type (Compress or Decompress).
// The returned Helper can be used to read compressed or decompressed data based on the operation specified.
func NewReader(algo arccmp.Algorithm, ope Operation, src io.Reader) (Helper, error) {
switch ope {
case Compress:
@@ -69,6 +72,9 @@ func NewReader(algo arccmp.Algorithm, ope Operation, src io.Reader) (Helper, err
return nil, ErrInvalidOperation
}
// NewWriter creates a new Helper instance for writing data to the provided io.Writer.
// It uses the specified compression algorithm and operation type (Compress or Decompress).
// The returned Helper can be used to write compressed or decompressed data based on the operation specified.
func NewWriter(algo arccmp.Algorithm, ope Operation, dst io.Writer) (Helper, error) {
switch ope {
case Compress:
@@ -79,77 +85,3 @@ func NewWriter(algo arccmp.Algorithm, ope Operation, dst io.Writer) (Helper, err
return nil, ErrInvalidOperation
}
func makeCompressWriter(algo arccmp.Algorithm, src io.Writer) (h Helper, err error) {
wc, ok := src.(io.WriteCloser)
if !ok {
wc = libarc.NopWriteCloser(src)
}
if wc, err = algo.Writer(wc); err != nil {
return nil, err
} else {
return &compressWriter{
dst: wc,
}, nil
}
}
func makeCompressReader(algo arccmp.Algorithm, src io.Reader) (h Helper, err error) {
rc, ok := src.(io.ReadCloser)
if !ok {
rc = io.NopCloser(src)
}
var (
buf = bytes.NewBuffer(make([]byte, 0))
wrt io.WriteCloser
)
wrt, err = algo.Writer(libarc.NopWriteCloser(buf))
return &compressReader{
src: rc,
wrt: wrt,
buf: buf,
clo: new(atomic.Bool),
}, nil
}
func makeDeCompressReader(algo arccmp.Algorithm, src io.Reader) (h Helper, err error) {
rc, ok := src.(io.ReadCloser)
if !ok {
rc = io.NopCloser(src)
}
if rc, err = algo.Reader(rc); err != nil {
return nil, err
} else {
return &deCompressReader{
src: rc,
}, nil
}
}
func makeDeCompressWriter(algo arccmp.Algorithm, src io.Writer) (h Helper, err error) {
wc, ok := src.(io.WriteCloser)
if !ok {
wc = libarc.NopWriteCloser(src)
}
return &deCompressWriter{
alg: algo,
wrt: wc,
buf: &bufNoEOF{
m: sync.Mutex{},
b: bytes.NewBuffer(make([]byte, 0)),
c: new(atomic.Bool),
},
clo: new(atomic.Bool),
run: new(atomic.Bool),
}, nil
}

17
archive/interface.go
View File

@@ -33,18 +33,35 @@ import (
arccmp "github.com/nabbar/golib/archive/compress"
)
// ParseCompression is a convenience function to parse a string and return the corresponding Compress/Algorithm.
// It returns the compression algorithm.
// If the parsing fails, it returns the None algorithm.
// Note: The returned algorithm should be used to create a compression reader.
func ParseCompression(s string) arccmp.Algorithm {
return arccmp.Parse(s)
}
// DetectCompression is a convenience function to detect the compression
// algorithm used in the provided io.Reader and return the compression read
// closer associated.
// It returns the compression algorithm, the compression reader, and an error if any.
// Note: The returned read closer should be closed by the caller to release resources.
func DetectCompression(r io.Reader) (arccmp.Algorithm, io.ReadCloser, error) {
return arccmp.Detect(r)
}
// ParseArchive is a convenience function to parse a string and return the corresponding Archive/Algorithm.
// It returns the archive algorithm.
// If the parsing fails, it returns the None algorithm.
func ParseArchive(s string) arcarc.Algorithm {
return arcarc.Parse(s)
}
// DetectArchive is a convenience function to detect the archive algorithm used in
// the provided io.ReadCloser and return the archive read closer associated.
// It returns the archive algorithm, the archive reader, and the original read closer.
// If the detection fails, it returns an error.
// Note: The returned read closer should be closed by the caller to release resources.
func DetectArchive(r io.ReadCloser) (arcarc.Algorithm, arctps.Reader, io.ReadCloser, error) {
return arcarc.Detect(r)
}

108
artifact/README.md Normal file
View File

@@ -0,0 +1,108 @@
# Golib Artifact
**Golang utilities for artifact version management retrieve / download**
---
## Overview
The `artifact` package provides tools to list, search, retrieve or downlaod version of artifacts such as files, binaries...
---
## Installation
Add the dependency to your Go project:
```shell
go get github.com/nabbar/golib/artifact
```
Or in your `go.mod`:
```go
require github.com/nabbar/golib/artifact vX.Y.Z
```
---
## Features
- List and search artifacts
- Retrieve artifacts by name and version
- Retrieve minor / major versions of artifacts
- Retrieve latest version of artifacts
- Retrieve download URL of artifacts
- Support for local and remote storage backends
---
## Quick Start
```go
package main
import (
"context"
"fmt"
"net/http"
"os"
"github.com/nabbar/golib/artifact"
"github.com/hashicorp/go-version"
"github.com/nabbar/golib/artifact/github"
)
func main() {
var (
err error
art artifact.Client
lst version.Collection
)
// Create a new artifact manager
art, err = github.NewGithub(context.Background(), &http.Client{}, "repo")
if err != nil {
fmt.Println("Error creating artifact manager:", err)
return
}
// List Versions
lst, err = art.ListReleases()
if err != nil {
fmt.Println("Error retrieving artifact:", err)
return
}
for _, ver := range lst {
fmt.Printf("Version: %s\n", ver.String())
link, e := art.GetArtifact("linux", "", ver)
if e != nil {
fmt.Println("No linux version found")
continue
}
fmt.Printf("Donwload Linux: %s\n", link)
}
}
```
---
## Error Handling
All functions return an `error` value. Always check and handle errors when storing, retrieving, or verifying artifacts.
---
## Contributing
Contributions are welcome! Please submit issues or pull requests on [GitHub](https://github.com/nabbar/golib).
---
## License
MIT © Nicolas JUHEL

174
atomic/README.md Normal file
View File

@@ -0,0 +1,174 @@
# golib/atomic
This package provides generic, thread-safe atomic values and maps for Go, making it easier to work with concurrent data structures. It offers atomic value containers, atomic maps (with both `any` and typed values), and utility functions for safe casting and default value management.
## Features
- **Generic atomic values**: Store, load, swap, and compare-and-swap any type safely.
- **Atomic maps**: Thread-safe maps with generic or typed values.
- **Default value management**: Set default values for atomic operations.
- **Safe type casting utilities**.
## Installation
Add to your `go.mod`:
```
require github.com/nabbar/golib/atomic vX.Y.Z
```
## Usage
### Atomic Value
Create and use an atomic value for any type:
```go
import "github.com/nabbar/golib/atomic"
type MyStruct struct {
Field1 string
Field2 int
}
val := atomic.NewValue[MyStruct]()
v1 := MyStruct{
Field1: "Hello",
Field2: 42,
}
val.Store(v1)
v1 = val.Load()
fmt.Println(v1.Field1) // Output: Hello
fmt.Println(v1.Field2) // Output: 42
v2 := MyStruct{
Field1: "World",
Field2: 100,
}
swapped := val.CompareAndSwap(v1, v2) // swapped == true
old := val.Swap(MyStruct{
Field1: "New",
Field2: 200,
}) // old == v2
fmt.Println(old.Field1) // Output: World
fmt.Println(old.Field2) // Output: 100
```
Set default values for load/store:
```go
val.SetDefaultLoad(0)
val.SetDefaultStore(-1)
val.Store(0) // Will store -1 instead of 0
v := val.Load() // If empty, returns 0
```
### Atomic Map (any value)
```go
m := atomic.NewMapAny[string]()
m.Store("foo", 123)
v, ok := m.Load("foo") // v == 123, ok == true
m.Delete("foo")
```
### Atomic Map (typed value)
```go
mt := atomic.NewMapTyped[string, int]()
mt.Store("bar", 456)
v, ok := mt.Load("bar") // v == 456, ok == true
mt.Delete("bar")
```
### Range Over Map
```go
mt.Range(func(key string, value int) bool {
fmt.Printf("%s: %d\n", key, value)
return true // continue iteration
})
```
### Safe Casting
```go
v, ok := atomic.Cast[int](anyValue)
if ok {
// v is of type int
}
empty := atomic.IsEmpty[string](anyValue)
```
## Interfaces
### Atomic Value
```go
type Value[T any] interface {
SetDefaultLoad(def T)
SetDefaultStore(def T)
Load() (val T)
Store(val T)
Swap(new T) (old T)
CompareAndSwap(old, new T) (swapped bool)
}
```
### Atomic Map
```go
// Map is a generic interface for atomic maps with any value type but typed key.
type Map[K comparable] interface {
Load(key K) (value any, ok bool)
Store(key K, value any)
LoadOrStore(key K, value any) (actual any, loaded bool)
LoadAndDelete(key K) (value any, loaded bool)
Delete(key K)
Swap(key K, value any) (previous any, loaded bool)
CompareAndSwap(key K, old, new any) bool
CompareAndDelete(key K, old any) (deleted bool)
Range(f func(key K, value any) bool)
}
// MapTyped is a specialized version of Map for typed values in add of type key from Map.
type MapTyped[K comparable, V any] interface {
Load(key K) (value V, ok bool)
Store(key K, value V)
LoadOrStore(key K, value V) (actual V, loaded bool)
LoadAndDelete(key K) (value V, loaded bool)
Delete(key K)
Swap(key K, value V) (previous V, loaded bool)
CompareAndSwap(key K, old, new V) bool
CompareAndDelete(key K, old V) (deleted bool)
Range(f func(key K, value V) bool)
}
```
See `atomic/interface.go` for full details.
## Error Handling
All operations are safe for concurrent use. Type assertions may fail if you use the wrong type; always check the returned boolean.
## License
MIT © Nicolas JUHEL
Generated With © Github Copilot.

552
aws/README.md Normal file
View File

@@ -0,0 +1,552 @@
# golib/aws
A modular and extensible Go library for AWS S3 and IAM, providing high-level helpers for buckets, objects, users, groups, roles, policies, and advanced uploaders.
---
## Table of Contents
- [Overview](#overview)
- [Installation](#installation)
- [Basic Usage](#basic-usage)
- [Detailed AWS Usage](#detailed-aws-usage)
- [Bucket](#bucket)
- [Object](#object)
- [User](#user)
- [Group](#group)
- [Role](#role)
- [Policy](#policy)
- [Advanced Uploads: Pusher & Multipart](#advanced-uploads-pusher--multipart)
- [Pusher (Recommended)](#pusher-recommended)
- [Multipart (Deprecated)](#multipart-deprecated)
- [Error Handling](#error-handling)
- [License](#license)
---
## Overview
This library provides a set of high-level interfaces and helpers to interact with AWS S3 and IAM resources, focusing on ease of use, extensibility, and robust error handling.
It abstracts the complexity of the AWS SDK for Go, while allowing advanced configuration and custom workflows.
---
## Installation
```sh
go get github.com/nabbar/golib/aws
```
---
## Basic Usage
### Simple S3 Object Upload
```go
package main
import (
"context"
"os"
"github.com/nabbar/golib/aws/pusher"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
func main() {
cfg := &pusher.Config{
FuncGetClientS3: func() *s3.Client { /* return your S3 client */ },
ObjectS3Options: pusher.ConfigObjectOptions{
Bucket: aws.String("my-bucket"),
Key: aws.String("my-object.txt"),
},
PartSize: 5 * 1024 * 1024, // 5MB
BufferSize: 4096,
CheckSum: true,
}
ctx := context.Background()
p, err := pusher.New(ctx, cfg)
if err != nil { panic(err) }
defer p.Close()
file, _ := os.Open("localfile.txt")
defer file.Close()
_, err = p.ReadFrom(file)
if err != nil { panic(err) }
err = p.Complete()
if err != nil { panic(err) }
}
```
### Simple Configuration & Usage
```go
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"github.com/nabbar/golib/aws"
"github.com/nabbar/golib/aws/configAws"
)
func main() {
// Create a new AWS configuration
cfg := configAws.NewConfig("my-new-bucket", "us-west-2", "your-access-key-id", "your-secret-access-key")
// Create a new AWS client with default configuration
cli, err := aws.New(context.Background(), cfg, &http.Client{})
if err != nil {
panic(err)
}
// Use the client to access S3 buckets, objects, IAM users, etc.
err := cli.Bucket().Create("my-new-bucket")
if err != nil {
panic(err)
}
// List all buckets
bck, err := cli.Bucket().List()
if err != nil {
panic(err)
}
for _, bucket := range bck {
fmt.Println("Bucket:", bucket)
}
// Upload an object to the bucket
file, err := os.Open("path/to/your/file.txt")
if err != nil {
panic(err)
}
defer file.Close()
err = cli.Object().Put("file.txt", file)
if err != nil {
panic(err)
}
fmt.Println("File uploaded successfully!")
// Download the object
obj, err := cli.Object().Get("file.txt")
if err != nil {
panic(err)
} else if obj == nil {
panic("Object not found")
} else if obj.ContentLength == nil || *obj.ContentLength == 0 {
panic("Object is empty")
} else if obj.Body == nil {
panic("Object is a directory")
}
defer obj.Body.Close()
outFile, err := os.Create("downloaded_file.txt")
if err != nil {
panic(err)
}
defer outFile.Close()
_, err = io.Copy(outFile, obj.Body)
if err != nil {
panic(err)
}
fmt.Println("File downloaded successfully!")
// Clean up: delete the bucket
err = cli.Bucket().Delete()
if err != nil {
panic(err)
}
fmt.Println("Bucket deleted successfully!")
}
```
---
## Detailed AWS Usage
### Bucket
Manage S3 buckets: create, delete, list, and set policies.
**Interface:**
```go
type Bucket interface {
Create(name string, opts ...Option) error
Delete(name string) error
List() ([]string, error)
SetPolicy(name string, policy Policy) error
// ...
}
```
**Example:**
```go
bck := cli.Bucket().Create("")
lst, err := cli.Bucket().List()
err = cli.Bucket().Delete()
```
#### List Buckets
List all S3 buckets in the account, optionally filtering by prefix:
```go
bck, err := cli.Bucket().List("prefix-")
if err != nil {
// handle error
}
for _, b := range bck {
fmt.Println(b)
}
```
#### Walk through S3 buckets
Walk through all S3 buckets, allowing custom processing for each bucket:
```go
err := cli.Bucket().Walk(func(bucket BucketInfo) bool {
fmt.Printf("Bucket: %s, Created: %s\n", bucket.Name, bucket.CreationDate)
return true // true to continue walking, false to stop
})
if err != nil {
// handle error
}
```
---
### Object
Manage S3 objects: upload, download, delete, copy, and metadata.
**Interface:**
```go
type Object interface {
Put(bucket, key string, data io.Reader, opts ...Option) error
Get(bucket, key string) (io.ReadCloser, error)
Delete(bucket, key string) error
Copy(srcBucket, srcKey, dstBucket, dstKey string) error
// ...
}
```
**Example:**
```go
err := cli.Object().Put("file.txt", fileReader)
reader, err := cli.Object().Get("file.txt")
err = cli.Object().Delete("file.txt")
```
#### List Object Versions
List all versions of an object in a bucket, or walk through all objects in a bucket.
```go
// Lister toutes les versions d’un objet dans un bucket
versions, err := cli.Object().VersionList("file.txt", "", "")
if err != nil {
// gérer l’erreur
}
for _, v := range versions {
fmt.Printf("Key: %s, VersionId: %s, IsLatest: %v\n", v.Key, v.VersionId, v.IsLatest)
}
```
#### Walk through S3 objects
Walk through all objects in a bucket, optionally filtering by prefix and delimiter:
```go
// Walk sur tous les objets d’un bucket
err := cli.Object().WalkPrefix("prefix/", func(info ObjectInfo) bool {
fmt.Printf("Object: %s, Size: %d\n", info.Key, info.Size)
return true // true to continue walking, false to stop
})
if err != nil {
// gérer l’erreur
}
```
---
### User
Manage IAM users: create, delete, credentials, and group membership.
**Interface:**
```go
type User interface {
Create(name string) error
Delete(name string) error
AddToGroup(user, group string) error
AttachPolicy(user string, policy Policy) error
// ...
}
```
**Example:**
```go
usr := awsClient.User()
err := usr.Create("alice")
err = usr.AddToGroup("alice", "admins")
```
#### List IAM Users
List all IAM users in the account, optionally filtering by prefix:
```go
usr, err := usr.List("prefix-")
if err != nil {
// handle error
}
```
#### Walk through IAM users
Walk through all IAM users, allowing custom processing for each user:
```go
err := usr.Walk(func(user UserInfo) bool {
fmt.Printf("User: %s, ARN: %s\n", user.Name, user.Arn)
return true // true to continue walking, false to stop
})
if err != nil {
// handle error
}
```
---
### Group
Manage IAM groups: create, delete, add/remove users, attach policies.
**Interface:**
```go
type Group interface {
Create(name string) error
Delete(name string) error
AddUser(group, user string) error
RemoveUser(group, user string) error
AttachPolicy(group string, policy Policy) error
// ...
}
```
**Example:**
```go
grp := awsClient.Group()
err := grp.Create("admins")
err = grp.AddUser("admins", "alice")
```
#### List IAM Groups
List all IAM groups in the account, optionally filtering by prefix:
```go
grp, err := grp.List("prefix-")
if err != nil {
// handle error
}
```
#### Walk through IAM groups
Walk through all IAM groups, allowing custom processing for each group:
```go
err := grp.Walk(func(group GroupInfo) bool {
fmt.Printf("Group: %s, ARN: %s\n", group.Name, group.Arn)
return true // true to continue walking, false to stop
})
if err != nil {
// handle error
}
```
---
### Role
Manage IAM roles: create, delete, attach policies.
**Interface:**
```go
type Role interface {
Create(name string, assumePolicy Policy) error
Delete(name string) error
AttachPolicy(role string, policy Policy) error
// ...
}
```
**Example:**
```go
role := awsClient.Role()
err := role.Create("my-role", assumePolicy)
err = role.AttachPolicy("my-role", policy)
```
#### List IAM Roles
List all IAM roles in the account, optionally filtering by prefix:
```go
rol, err := role.List("prefix-")
if err != nil {
// handle error
}
```
#### Walk through IAM roles
Walk through all IAM roles, allowing custom processing for each role:
```go
err := role.Walk(func(role RoleInfo) bool {
fmt.Printf("Role: %s, ARN: %s\n", role.Name, role.Arn)
return true // true to continue walking, false to stop
})
if err != nil {
// handle error
}
```
---
### Policy
Manage IAM policies: create, delete, attach/detach.
**Interface:**
```go
type Policy interface {
Create(name string, document string) error
Delete(name string) error
Attach(entity string) error
Detach(entity string) error
// ...
}
```
**Example:**
```go
pol := awsClient.Policy()
err := pol.Create("readonly", policyJSON)
err = pol.Attach("my-role")
```
#### List IAM Policies
List all IAM policies in the account, optionally filtering by prefix:
```go
pol, err := pol.List("prefix-")
if err != nil {
// handle error
}
```
#### Walk through IAM policies
Walk through all IAM policies, allowing custom processing for each policy:
```go
err := pol.Walk(func(policy PolicyInfo) bool {
fmt.Printf("Policy: %s, ARN: %s\n", policy.Name, policy.Arn)
return true // true to continue walking, false to stop
})
if err != nil {
// handle error
}
```
---
## Advanced Uploads: Pusher & Multipart
### Pusher (Recommended)
Modern, high-level API for uploading objects to S3, supporting both single and multipart uploads, with advanced configuration and callback support.
**Interface:**
```go
type Pusher interface {
io.WriteCloser
io.ReaderFrom
Abort() error
Complete() error
CopyFromS3(bucket, object, versionId string) error
// ... (see pusher package for full details)
}
```
**Features:**
- Automatic switch between single and multipart upload
- Custom part size, buffer size, working file path
- Optional SHA256 checksum
- Callbacks for upload progress, completion, and abort
- S3 object copy support (server-side, multipart)
- Thread-safe and context-aware
**Advanced Example:**
```go
cfg := &pusher.Config{
FuncGetClientS3: myS3ClientFunc,
FuncCallOnUpload: func(upd pusher.UploadInfo, obj pusher.ObjectInfo, e error) {
fmt.Printf("Uploaded part %d, etag=%s, error=%v\n", upd.PartNumber, upd.Etag, e)
},
FuncCallOnComplete: func(obj pusher.ObjectInfo, e error) {
fmt.Printf("Upload complete: %+v, error=%v\n", obj, e)
},
WorkingPath: "/var/tmp",
PartSize: 50 * 1024 * 1024,
BufferSize: 256 * 1024,
CheckSum: true,
ObjectS3Options: pusher.ConfigObjectOptions{
Bucket: aws.String("my-bucket"),
Key: aws.String("my-object"),
Metadata: map[string]string{"x-amz-meta-custom": "value"},
ContentType: aws.String("application/octet-stream"),
},
}
```
---
### Multipart (Deprecated)
**Warning:** The `multipart` package is deprecated.
Use `pusher` for all new developments.
Legacy API for multipart uploads. Maintained for backward compatibility.
---
## Error Handling
All packages return Go `error` values.
Common error variables are exported (e.g., `ErrInvalidInstance`, `ErrInvalidClient`, `ErrInvalidResponse`, `ErrInvalidUploadID`, `ErrEmptyContents`, `ErrInvalidChecksum` in `pusher`).
Always check returned errors and handle them appropriately.
**Example:**
```go
if err := p.Complete(); err != nil {
if errors.Is(err, pusher.ErrInvalidChecksum) {
// handle checksum error
} else {
// handle other errors
}
}
```
---
## License
MIT © Nicolas JUHEL
Generated With © Github Copilot.

14
aws/bucket/bucket.go
View File

@@ -106,6 +106,12 @@ func (cli *client) List() ([]sdkstp.Bucket, error) {
}
func (cli *client) Walk(f WalkFunc) error {
if f == nil {
f = func(bucket sdkstp.Bucket) bool {
return false
}
}
out, err := cli.s3.ListBuckets(cli.GetContext(), nil)
if err != nil {
@@ -114,15 +120,15 @@ func (cli *client) Walk(f WalkFunc) error {
return libhlp.ErrorAwsEmpty.Error(nil)
}
var e error
for _, b := range out.Buckets {
if b.Name == nil || b.CreationDate == nil || len(*b.Name) < 3 || b.CreationDate.IsZero() {
continue
}
if f != nil {
e = f(e, b)
if !f(b) {
return nil
}
}
return e
return nil
}

2
aws/bucket/interface.go
View File

@@ -40,7 +40,7 @@ type client struct {
s3 *sdksss.Client
}
type WalkFunc func(err error, bucket sdkstp.Bucket) error
type WalkFunc func(bucket sdkstp.Bucket) bool
type Bucket interface {
Check() error

9
aws/group/interface.go
View File

@@ -40,22 +40,27 @@ type client struct {
s3 *sdksss.Client
}
type PoliciesWalkFunc func(err error, pol sdktps.AttachedPolicy) error
type PoliciesWalkFunc func(pol sdktps.AttachedPolicy) bool
type FuncWalkGroupForUser func(grp sdktps.Group) bool
type GroupFunc func(group sdktps.Group) bool
type Group interface {
WalkGroupForUser(username string, fct FuncWalkGroupForUser) error
UserList(username string) ([]string, error)
DetachGroups(prefix string) ([]string, error)
UserCheck(username, groupName string) (error, bool)
UserAdd(username, groupName string) error
UserRemove(username, groupName string) error
Walk(prefix string, fct GroupFunc) error
List() (map[string]string, error)
Add(groupName string) error
Remove(groupName string) error
PolicyList(groupName string) (map[string]string, error)
PolicyAttach(groupName, polArn string) error
PolicyDetach(groupName, polArn string) error
PolicyAttachedList(groupName, marker string) ([]sdktps.AttachedPolicy, string, error)
PolicyAttachedList(groupName string) ([]sdktps.AttachedPolicy, error)
PolicyAttachedWalk(groupName string, fct PoliciesWalkFunc) error
}

66
aws/group/policy.go
View File

@@ -32,19 +32,24 @@ import (
)
func (cli *client) PolicyList(groupName string) (map[string]string, error) {
out, _, err := cli.PolicyAttachedList(groupName, "")
if err != nil {
return nil, err
var (
err error
res = make(map[string]string, 0)
fct = func(pol sdktps.AttachedPolicy) bool {
if pol.PolicyArn == nil || len(*pol.PolicyArn) == 0 {
return true
} else if pol.PolicyName == nil || len(*pol.PolicyName) == 0 {
return true
} else {
var res = make(map[string]string)
for _, p := range out {
res[*p.PolicyName] = *p.PolicyArn
res[*pol.PolicyName] = *pol.PolicyArn
}
return res, nil
return true
}
)
err = cli.PolicyAttachedWalk(groupName, fct)
return res, err
}
func (cli *client) PolicyAttach(groupName, polArn string) error {
@@ -65,27 +70,19 @@ func (cli *client) PolicyDetach(groupName, polArn string) error {
return cli.GetError(err)
}
func (cli *client) PolicyAttachedList(groupName, marker string) ([]sdktps.AttachedPolicy, string, error) {
in := &sdkiam.ListAttachedGroupPoliciesInput{
GroupName: sdkaws.String(groupName),
MaxItems: sdkaws.Int32(1000),
func (cli *client) PolicyAttachedList(groupName string) ([]sdktps.AttachedPolicy, error) {
var (
err error
res = make([]sdktps.AttachedPolicy, 0)
fct = func(pol sdktps.AttachedPolicy) bool {
res = append(res, pol)
return true
}
)
if marker != "" {
in.Marker = sdkaws.String(marker)
}
err = cli.PolicyAttachedWalk(groupName, fct)
lst, err := cli.iam.ListAttachedGroupPolicies(cli.GetContext(), in)
if err != nil {
return nil, "", cli.GetError(err)
} else if lst == nil || lst.AttachedPolicies == nil {
return nil, "", nil
} else if lst.IsTruncated && lst.Marker != nil {
return lst.AttachedPolicies, *lst.Marker, nil
} else {
return lst.AttachedPolicies, "", nil
}
return res, err
}
func (cli *client) PolicyAttachedWalk(groupName string, fct PoliciesWalkFunc) error {
@@ -96,6 +93,12 @@ func (cli *client) PolicyAttachedWalk(groupName string, fct PoliciesWalkFunc) er
MaxItems: sdkaws.Int32(1000),
}
if fct == nil {
fct = func(pol sdktps.AttachedPolicy) bool {
return false
}
}
for {
if m != nil {
in.Marker = m
@@ -111,13 +114,10 @@ func (cli *client) PolicyAttachedWalk(groupName string, fct PoliciesWalkFunc) er
return nil
}
var e error
for _, p := range lst.AttachedPolicies {
e = fct(e, p)
for i := range lst.AttachedPolicies {
if !fct(lst.AttachedPolicies[i]) {
return nil
}
if e != nil {
return e
}
if lst.IsTruncated && lst.Marker != nil {

39
aws/group/user.go
View File

@@ -28,6 +28,8 @@ package group
import (
sdkaws "github.com/aws/aws-sdk-go-v2/aws"
sdkiam "github.com/aws/aws-sdk-go-v2/service/iam"
"github.com/aws/aws-sdk-go-v2/service/iam/types"
libhlp "github.com/nabbar/golib/aws/helper"
)
func (cli *client) UserCheck(username, groupName string) (error, bool) {
@@ -49,21 +51,44 @@ func (cli *client) UserCheck(username, groupName string) (error, bool) {
}
func (cli *client) UserList(username string) ([]string, error) {
var (
res = make([]string, 0)
fct = func(grp types.Group) bool {
if grp.GroupName != nil && len(*grp.GroupName) > 3 {
res = append(res, *grp.GroupName)
}
return true
}
)
err := cli.WalkGroupForUser(username, fct)
return res, err
}
func (cli *client) WalkGroupForUser(username string, fct FuncWalkGroupForUser) error {
out, err := cli.iam.ListGroupsForUser(cli.GetContext(), &sdkiam.ListGroupsForUserInput{
UserName: sdkaws.String(username),
})
if fct == nil {
fct = func(grp types.Group) bool {
return false
}
}
if err != nil {
return nil, cli.GetError(err)
return cli.GetError(err)
} else if out == nil {
return libhlp.ErrorAwsEmpty.Error(nil)
} else {
var res = make([]string, 0)
for _, g := range out.Groups {
res = append(res, *g.GroupName)
for i := range out.Groups {
if !fct(out.Groups[i]) {
return nil
}
}
}
return res, nil
}
return nil
}
func (cli *client) UserAdd(username, groupName string) error {

8
aws/object/interface.go
View File

@@ -50,10 +50,10 @@ type Metadata struct {
DeleteMarkers int
}
type WalkFunc func(err error, obj sdktps.Object) error
type WalkFuncMetadata func(err error, md Metadata) error
type WalkFuncVersion func(err error, obj sdktps.ObjectVersion) error
type WalkFuncDelMak func(err error, del sdktps.DeleteMarkerEntry) error
type WalkFunc func(obj sdktps.Object) bool
type WalkFuncMetadata func(md Metadata) bool
type WalkFuncVersion func(obj sdktps.ObjectVersion) bool
type WalkFuncDelMak func(del sdktps.DeleteMarkerEntry) bool
type Object interface {
Find(regex string) ([]string, error)

34
aws/object/object.go
View File

@@ -87,8 +87,26 @@ func (cli *client) WalkPrefix(prefix string, md WalkFuncMetadata, f WalkFunc) er
var (
e error
t = sdkaws.String("")
om = true
fm = func(md Metadata) bool {
return false
}
ov = true
fo = func(obj sdktps.Object) bool {
return false
}
)
if md != nil {
fm = md
}
if f != nil {
fo = f
}
for {
if len(*t) > 0 {
in.ContinuationToken = t
@@ -100,23 +118,27 @@ func (cli *client) WalkPrefix(prefix string, md WalkFuncMetadata, f WalkFunc) er
return cli.GetError(err)
} else if out == nil {
return libhlp.ErrorResponse.Error()
} else if md != nil {
e = md(e, Metadata{
} else if om {
om = fm(Metadata{
Objects: len(out.Contents),
})
}
if ov {
for _, o := range out.Contents {
if o.Key == nil || len(*o.Key) < 1 {
if !ov {
break
} else if o.Key == nil || len(*o.Key) < 1 {
continue
}
if f != nil {
e = f(e, o)
ov = fo(o)
}
}
if out != nil && out.IsTruncated != nil && *out.IsTruncated {
if !ov && !om {
return e
} else if out != nil && out.IsTruncated != nil && *out.IsTruncated {
t = out.NextContinuationToken
} else {
return e

51
aws/object/version.go
View File

@@ -86,8 +86,35 @@ func (cli *client) VersionWalkPrefix(prefix string, md WalkFuncMetadata, fv Walk
e error
km = sdkaws.String("")
mi = sdkaws.String("")
okm = true
fmd = func(md Metadata) bool {
return false
}
okv = true
fov = func(obj sdktps.ObjectVersion) bool {
return false
}
okd = true
fod = func(del sdktps.DeleteMarkerEntry) bool {
return false
}
)
if md != nil {
fmd = md
}
if fv != nil {
fov = fv
}
if fd != nil {
fod = fd
}
for {
if len(*km) > 0 && len(*mi) > 0 {
in.KeyMarker = km
@@ -100,38 +127,44 @@ func (cli *client) VersionWalkPrefix(prefix string, md WalkFuncMetadata, fv Walk
return cli.GetError(err)
} else if out == nil {
return libhlp.ErrorResponse.Error(nil)
} else if md != nil {
e = md(e, Metadata{
} else if okm {
okm = fmd(Metadata{
Versions: len(out.Versions),
DeleteMarkers: len(out.DeleteMarkers),
})
}
if okv {
for _, o := range out.Versions {
if o.Key == nil || len(*o.Key) < 1 {
if !okv {
break
} else if o.Key == nil || len(*o.Key) < 1 {
continue
} else if o.VersionId == nil || len(*o.VersionId) < 1 {
continue
}
if fv != nil {
e = fv(e, o)
okv = fov(o)
}
}
if okd {
for _, o := range out.DeleteMarkers {
if o.Key == nil || len(*o.Key) < 1 {
if !okd {
break
} else if o.Key == nil || len(*o.Key) < 1 {
continue
} else if o.VersionId == nil || len(*o.VersionId) < 1 {
continue
}
if fd != nil {
e = fd(e, o)
okd = fod(o)
}
}
if out != nil && out.IsTruncated != nil && *out.IsTruncated {
if !okm && !okv && !okd {
return e
} else if out != nil && out.IsTruncated != nil && *out.IsTruncated {
km = out.NextKeyMarker
mi = out.NextVersionIdMarker
} else {

22
aws/policy/interface.go
View File

@@ -28,34 +28,36 @@ package policy
import (
"context"
"github.com/aws/aws-sdk-go-v2/service/iam"
"github.com/aws/aws-sdk-go-v2/service/iam/types"
"github.com/aws/aws-sdk-go-v2/service/s3"
sdkiam "github.com/aws/aws-sdk-go-v2/service/iam"
iamtps "github.com/aws/aws-sdk-go-v2/service/iam/types"
sdksss "github.com/aws/aws-sdk-go-v2/service/s3"
libhlp "github.com/nabbar/golib/aws/helper"
)
type client struct {
libhlp.Helper
iam *iam.Client
s3 *s3.Client
iam *sdkiam.Client
s3 *sdksss.Client
}
type PolicyFunc func(policyArn string) bool
type FuncWalkPolicy func(pol iamtps.Policy) bool
type Policy interface {
List() (map[string]string, error)
Get(arn string) (*types.Policy, error)
Get(arn string) (*iamtps.Policy, error)
Add(name, desc, policy string) (string, error)
Update(polArn, polContents string) error
Delete(polArn string) error
VersionList(arn string, maxItem int32, noDefaultVersion bool) (map[string]string, error)
VersionGet(arn string, vers string) (*types.PolicyVersion, error)
VersionGet(arn string, vers string) (*iamtps.PolicyVersion, error)
VersionAdd(arn string, doc string) error
VersionDel(arn string, vers string) error
CompareUpdate(arn string, doc string) (upd bool, err error)
Walk(prefix string, fct PolicyFunc) error
Walk(prefix string, fct FuncWalkPolicy) error
GetAllPolicies(prefix string) ([]string, error)
}
func New(ctx context.Context, bucket, region string, iam *iam.Client, s3 *s3.Client) Policy {
func New(ctx context.Context, bucket, region string, iam *sdkiam.Client, s3 *sdksss.Client) Policy {
return &client{
Helper: libhlp.New(ctx, bucket, region),
iam: iam,

108
aws/policy/policies.go
View File

@@ -26,39 +26,69 @@
package policy
import (
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/iam"
"github.com/aws/aws-sdk-go-v2/service/iam/types"
sdkaws "github.com/aws/aws-sdk-go-v2/aws"
sdkiam "github.com/aws/aws-sdk-go-v2/service/iam"
iamtps "github.com/aws/aws-sdk-go-v2/service/iam/types"
libhlp "github.com/nabbar/golib/aws/helper"
)
func (cli *client) List() (map[string]string, error) {
out, err := cli.iam.ListPolicies(cli.GetContext(), &iam.ListPoliciesInput{})
if err != nil {
return nil, cli.GetError(err)
} else {
var res = make(map[string]string)
for _, p := range out.Policies {
res[*p.PolicyName] = *p.Arn
var (
err error
res = make(map[string]string)
fct = func(pol iamtps.Policy) bool {
if pol.PolicyName == nil || len(*pol.PolicyName) == 0 {
return true
} else if pol.Arn == nil || len(*pol.Arn) == 0 {
return true
}
return res, nil
res[*pol.PolicyName] = *pol.Arn
return true
}
)
err = cli.Walk("", fct)
return res, err
}
func (cli *client) Walk(prefix string, fct PolicyFunc) error {
func (cli *client) GetAllPolicies(prefix string) ([]string, error) {
var (
err error
res = make([]string, 0)
fct = func(pol iamtps.Policy) bool {
if pol.PolicyName == nil || len(*pol.PolicyName) == 0 {
return true
} else if pol.Arn == nil || len(*pol.Arn) == 0 {
return true
}
res = append(res, *pol.Arn)
return true
}
)
err = cli.Walk(prefix, fct)
return res, err
}
func (cli *client) Walk(prefix string, fct FuncWalkPolicy) error {
var (
mrk *string
trk = true
)
if fct == nil {
fct = func(pol iamtps.Policy) bool {
return false
}
}
for trk {
in := &iam.ListPoliciesInput{}
in := &sdkiam.ListPoliciesInput{}
if len(prefix) > 0 {
in.PathPrefix = aws.String(prefix)
in.PathPrefix = sdkaws.String(prefix)
}
if mrk != nil && len(*mrk) > 0 {
@@ -70,8 +100,8 @@ func (cli *client) Walk(prefix string, fct PolicyFunc) error {
return cli.GetError(err)
}
for _, policy := range out.Policies {
if !fct(*policy.Arn) {
for i := range out.Policies {
if !fct(out.Policies[i]) {
return nil
}
}
@@ -86,23 +116,9 @@ func (cli *client) Walk(prefix string, fct PolicyFunc) error {
return nil
}
func (cli *client) GetAllPolicies(prefix string) ([]string, error) {
var policies []string
err := cli.Walk(prefix, func(policyArn string) bool {
policies = append(policies, policyArn)
return true
})
if err != nil {
return nil, err
}
return policies, nil
}
func (cli *client) Get(arn string) (*types.Policy, error) {
out, err := cli.iam.GetPolicy(cli.GetContext(), &iam.GetPolicyInput{
PolicyArn: aws.String(arn),
func (cli *client) Get(arn string) (*iamtps.Policy, error) {
out, err := cli.iam.GetPolicy(cli.GetContext(), &sdkiam.GetPolicyInput{
PolicyArn: sdkaws.String(arn),
})
if err != nil {
@@ -115,10 +131,10 @@ func (cli *client) Get(arn string) (*types.Policy, error) {
}
func (cli *client) Add(name, desc, policy string) (string, error) {
out, err := cli.iam.CreatePolicy(cli.GetContext(), &iam.CreatePolicyInput{
PolicyName: aws.String(name),
Description: aws.String(desc),
PolicyDocument: aws.String(policy),
out, err := cli.iam.CreatePolicy(cli.GetContext(), &sdkiam.CreatePolicyInput{
PolicyName: sdkaws.String(name),
Description: sdkaws.String(desc),
PolicyDocument: sdkaws.String(policy),
})
if err != nil {
@@ -130,7 +146,7 @@ func (cli *client) Add(name, desc, policy string) (string, error) {
func (cli *client) Update(polArn, polContents string) error {
var (
pol *types.Policy
pol *iamtps.Policy
lst map[string]string
err error
)
@@ -157,8 +173,8 @@ func (cli *client) Update(polArn, polContents string) error {
}
func (cli *client) Delete(polArn string) error {
out, err := cli.iam.ListPolicyVersions(cli.GetContext(), &iam.ListPolicyVersionsInput{
PolicyArn: aws.String(polArn),
out, err := cli.iam.ListPolicyVersions(cli.GetContext(), &sdkiam.ListPolicyVersionsInput{
PolicyArn: sdkaws.String(polArn),
})
if err != nil {
@@ -170,8 +186,8 @@ func (cli *client) Delete(polArn string) error {
}
if !v.IsDefaultVersion {
_, _ = cli.iam.DeletePolicyVersion(cli.GetContext(), &iam.DeletePolicyVersionInput{
PolicyArn: aws.String(polArn),
_, _ = cli.iam.DeletePolicyVersion(cli.GetContext(), &sdkiam.DeletePolicyVersionInput{
PolicyArn: sdkaws.String(polArn),
VersionId: v.VersionId,
})
}
@@ -182,8 +198,8 @@ func (cli *client) Delete(polArn string) error {
return nil
}
_, err = cli.iam.DeletePolicy(cli.GetContext(), &iam.DeletePolicyInput{
PolicyArn: aws.String(polArn),
_, err = cli.iam.DeletePolicy(cli.GetContext(), &sdkiam.DeletePolicyInput{
PolicyArn: sdkaws.String(polArn),
})
return cli.GetError(err)

12
aws/role/interface.go
View File

@@ -40,20 +40,22 @@ type client struct {
s3 *sdksss.Client
}
type PoliciesWalkFunc func(err error, pol sdktps.AttachedPolicy) error
type RoleFunc func(roleName string) bool
type FuncWalkPolicies func(pol sdktps.AttachedPolicy) bool
type FuncWalkRole func(role sdktps.Role) bool
type Role interface {
Walk(prefix string, fct FuncWalkRole) error
List() ([]sdktps.Role, error)
Check(name string) (string, error)
Add(name, role string) (string, error)
Delete(roleName string) error
PolicyAttach(policyARN, roleName string) error
PolicyDetach(policyARN, roleName string) error
PolicyListAttached(roleName string) ([]sdktps.AttachedPolicy, error)
PolicyAttachedList(roleName, marker string) ([]sdktps.AttachedPolicy, string, error)
PolicyAttachedWalk(roleName string, fct PoliciesWalkFunc) error
Walk(prefix string, fct RoleFunc) error
DetachRoles(prefix string) ([]string, error)
PolicyAttachedWalk(roleName string, fct FuncWalkPolicies) error
PolicyDetachRoles(prefix string) ([]string, error)
}
func New(ctx context.Context, bucket, region string, iam *sdkiam.Client, s3 *sdksss.Client) Role {

76
aws/role/policy.go
View File

@@ -85,7 +85,7 @@ func (cli *client) PolicyAttachedList(roleName, marker string) ([]sdktps.Attache
}
}
func (cli *client) PolicyAttachedWalk(roleName string, fct PoliciesWalkFunc) error {
func (cli *client) PolicyAttachedWalk(roleName string, fct FuncWalkPolicies) error {
var m *string
in := &sdkiam.ListAttachedRolePoliciesInput{
@@ -93,6 +93,12 @@ func (cli *client) PolicyAttachedWalk(roleName string, fct PoliciesWalkFunc) err
MaxItems: sdkaws.Int32(1000),
}
if fct == nil {
fct = func(pol sdktps.AttachedPolicy) bool {
return false
}
}
for {
if m != nil {
in.Marker = m
@@ -108,13 +114,10 @@ func (cli *client) PolicyAttachedWalk(roleName string, fct PoliciesWalkFunc) err
return nil
}
var e error
for _, p := range lst.AttachedPolicies {
e = fct(e, p)
for i := range lst.AttachedPolicies {
if !fct(lst.AttachedPolicies[i]) {
return nil
}
if e != nil {
return e
}
if lst.IsTruncated && lst.Marker != nil {
@@ -124,3 +127,62 @@ func (cli *client) PolicyAttachedWalk(roleName string, fct PoliciesWalkFunc) err
}
}
}
func (cli *client) PolicyDetachRoles(prefix string) ([]string, error) {
var (
e error
err error
res = make([]string, 0)
fct = func(role sdktps.Role) bool {
if role.RoleName == nil || len(*role.RoleName) == 0 {
return true
} else if role.Arn == nil || len(*role.Arn) == 0 {
return true
} else if e = cli.detachPolicyDetachRoles(*role.RoleName); e != nil {
return false
}
res = append(res, *role.RoleName)
return true
}
)
err = cli.Walk(prefix, fct)
if err != nil {
return nil, err
} else if e != nil {
return nil, e
} else {
return res, nil
}
}
func (cli *client) detachPolicyDetachRoles(roleName string) error {
out, err := cli.iam.ListAttachedRolePolicies(cli.GetContext(), &sdkiam.ListAttachedRolePoliciesInput{
RoleName: sdkaws.String(roleName),
})
if err != nil {
return cli.GetError(err)
} else if out == nil || out.AttachedPolicies == nil {
return nil
}
for i := range out.AttachedPolicies {
if out.AttachedPolicies[i].PolicyArn == nil || len(*out.AttachedPolicies[i].PolicyArn) == 0 {
continue
}
_, err = cli.iam.DetachRolePolicy(cli.GetContext(), &sdkiam.DetachRolePolicyInput{
RoleName: sdkaws.String(roleName),
PolicyArn: out.AttachedPolicies[i].PolicyArn,
})
if err != nil {
return cli.GetError(err)
}
}
return nil
}

62
aws/role/role.go
View File

@@ -31,16 +31,7 @@ import (
iamtps "github.com/aws/aws-sdk-go-v2/service/iam/types"
)
func (cli *client) List() ([]iamtps.Role, error) {
out, err := cli.iam.ListRoles(cli.GetContext(), &sdkiam.ListRolesInput{})
if err != nil {
return nil, cli.GetError(err)
} else {
return out.Roles, nil
}
}
func (cli *client) Walk(prefix string, fct RoleFunc) error {
func (cli *client) Walk(prefix string, fct FuncWalkRole) error {
var (
err error
out *sdkiam.ListRolesOutput
@@ -48,6 +39,12 @@ func (cli *client) Walk(prefix string, fct RoleFunc) error {
trk = true
)
if fct == nil {
fct = func(role iamtps.Role) bool {
return false
}
}
for trk {
in := &sdkiam.ListRolesInput{}
@@ -64,8 +61,8 @@ func (cli *client) Walk(prefix string, fct RoleFunc) error {
return cli.GetError(err)
}
for _, role := range out.Roles {
if !fct(*role.RoleName) {
for i := range out.Roles {
if !fct(out.Roles[i]) {
return nil
}
}
@@ -80,45 +77,14 @@ func (cli *client) Walk(prefix string, fct RoleFunc) error {
return nil
}
func (cli *client) detachPoliciesFromRole(roleName string) error {
attachedPoliciesOutput, err := cli.iam.ListAttachedRolePolicies(cli.GetContext(),
&sdkiam.ListAttachedRolePoliciesInput{
RoleName: sdkaws.String(roleName),
})
if err != nil {
return cli.GetError(err)
}
for _, policy := range attachedPoliciesOutput.AttachedPolicies {
_, err := cli.iam.DetachRolePolicy(cli.GetContext(),
&sdkiam.DetachRolePolicyInput{
RoleName: sdkaws.String(roleName),
PolicyArn: policy.PolicyArn,
})
if err != nil {
return cli.GetError(err)
}
}
return nil
}
func (cli *client) DetachRoles(prefix string) ([]string, error) {
var detachedRoleNames []string
err := cli.Walk(prefix, func(roleName string) bool {
if err := cli.detachPoliciesFromRole(roleName); err != nil {
return false
}
detachedRoleNames = append(detachedRoleNames, roleName)
return true
})
func (cli *client) List() ([]iamtps.Role, error) {
out, err := cli.iam.ListRoles(cli.GetContext(), &sdkiam.ListRolesInput{})
if err != nil {
return nil, err
return nil, cli.GetError(err)
} else {
return out.Roles, nil
}
return detachedRoleNames, nil
}
func (cli *client) Check(name string) (string, error) {

17
aws/user/interface.go
View File

@@ -40,27 +40,32 @@ type client struct {
s3 *sdksss.Client
}
type PoliciesWalkFunc func(err error, pol sdktps.AttachedPolicy) error
type UserFunc func(user sdktps.User) bool
type FuncWalkPolicies func(pol sdktps.AttachedPolicy) bool
type FuncWalkUsers func(user sdktps.User) bool
type User interface {
List() (map[string]string, error)
Walk(prefix string, fct FuncWalkUsers) error
Get(username string) (*sdktps.User, error)
Create(username string) error
Delete(username string) error
PolicyPut(policyDocument, policyName, username string) error
PolicyAttach(policyARN, username string) error
PolicyDetach(policyARN, username string) error
PolicyAttachedList(username, marker string) ([]sdktps.AttachedPolicy, string, error)
PolicyAttachedWalk(username string, fct PoliciesWalkFunc) error
PolicyAttachedList(username string) ([]sdktps.AttachedPolicy, error)
PolicyAttachedWalk(username string, fct FuncWalkPolicies) error
PolicyDetachUsers(prefix string) ([]string, error)
LoginCheck(username string) error
LoginCreate(username, password string) error
LoginDelete(username string) error
AccessListAll(username string) ([]sdktps.AccessKeyMetadata, error)
AccessList(username string) (map[string]bool, error)
AccessCreate(username string) (string, string, error)
AccessDelete(username, accessKey string) error
Walk(prefix string, fct UserFunc) error
DetachUsers(prefix string) ([]string, error)
}
func New(ctx context.Context, bucket, region string, iam *sdkiam.Client, s3 *sdksss.Client) User {

139
aws/user/policy.go
View File

@@ -41,30 +41,21 @@ func (cli *client) PolicyPut(policyDocument, policyName, username string) error
return cli.GetError(err)
}
func (cli *client) PolicyAttachedList(username, marker string) ([]sdktps.AttachedPolicy, string, error) {
in := &sdkiam.ListAttachedUserPoliciesInput{
UserName: sdkaws.String(username),
MaxItems: sdkaws.Int32(1000),
func (cli *client) PolicyAttachedList(username string) ([]sdktps.AttachedPolicy, error) {
var (
err error
res = make([]sdktps.AttachedPolicy, 0)
fct = func(pol sdktps.AttachedPolicy) bool {
res = append(res, pol)
return true
}
)
err = cli.PolicyAttachedWalk(username, fct)
return res, err
}
if marker != "" {
in.Marker = sdkaws.String(marker)
}
lst, err := cli.iam.ListAttachedUserPolicies(cli.GetContext(), in)
if err != nil {
return nil, "", cli.GetError(err)
} else if lst == nil || lst.AttachedPolicies == nil {
return nil, "", nil
} else if lst.IsTruncated && lst.Marker != nil {
return lst.AttachedPolicies, *lst.Marker, nil
} else {
return lst.AttachedPolicies, "", nil
}
}
func (cli *client) PolicyAttachedWalk(username string, fct PoliciesWalkFunc) error {
func (cli *client) PolicyAttachedWalk(username string, fct FuncWalkPolicies) error {
var m *string
in := &sdkiam.ListAttachedUserPoliciesInput{
@@ -72,6 +63,12 @@ func (cli *client) PolicyAttachedWalk(username string, fct PoliciesWalkFunc) err
MaxItems: sdkaws.Int32(1000),
}
if fct == nil {
fct = func(pol sdktps.AttachedPolicy) bool {
return false
}
}
for {
if m != nil {
in.Marker = m
@@ -87,13 +84,10 @@ func (cli *client) PolicyAttachedWalk(username string, fct PoliciesWalkFunc) err
return nil
}
var e error
for _, p := range lst.AttachedPolicies {
e = fct(e, p)
for i := range lst.AttachedPolicies {
if !fct(lst.AttachedPolicies[i]) {
return nil
}
if e != nil {
return e
}
if lst.IsTruncated && lst.Marker != nil {
@@ -121,3 +115,92 @@ func (cli *client) PolicyDetach(policyARN, username string) error {
return cli.GetError(err)
}
func (cli *client) PolicyDetachUsers(prefix string) ([]string, error) {
var (
e error
err error
res = make([]string, 0)
fct = func(user sdktps.User) bool {
if user.UserName == nil || len(*user.UserName) == 0 {
return true
}
if e = cli.removeGroupsPolicyDetachUsers(*user.UserName); e != nil {
return false
} else if e = cli.detachPolicyDetachUsers(*user.UserName); e != nil {
return false
}
res = append(res, *user.UserName)
return true
}
)
err = cli.Walk(prefix, fct)
if err != nil {
return nil, err
} else if e != nil {
return nil, e
} else {
return res, nil
}
}
func (cli *client) removeGroupsPolicyDetachUsers(username string) error {
out, err := cli.iam.ListGroupsForUser(cli.GetContext(), &sdkiam.ListGroupsForUserInput{
UserName: sdkaws.String(username),
})
if err != nil {
return cli.GetError(err)
} else if out == nil || len(out.Groups) < 1 {
return nil
}
for i := range out.Groups {
if out.Groups[i].GroupName == nil || len(*out.Groups[i].GroupName) == 0 {
continue
}
_, err = cli.iam.RemoveUserFromGroup(cli.GetContext(), &sdkiam.RemoveUserFromGroupInput{
UserName: sdkaws.String(username),
GroupName: out.Groups[i].GroupName,
})
if err != nil {
return cli.GetError(err)
}
}
return nil
}
func (cli *client) detachPolicyDetachUsers(username string) error {
out, err := cli.iam.ListAttachedUserPolicies(cli.GetContext(), &sdkiam.ListAttachedUserPoliciesInput{
UserName: sdkaws.String(username),
})
if err != nil {
return cli.GetError(err)
} else if out == nil || len(out.AttachedPolicies) < 1 {
return nil
}
for i := range out.AttachedPolicies {
if out.AttachedPolicies[i].PolicyArn == nil || len(*out.AttachedPolicies[i].PolicyArn) == 0 {
continue
}
_, err = cli.iam.DetachUserPolicy(cli.GetContext(), &sdkiam.DetachUserPolicyInput{
UserName: sdkaws.String(username),
PolicyArn: out.AttachedPolicies[i].PolicyArn,
})
if err != nil {
return cli.GetError(err)
}
}
return nil
}

99
aws/user/user.go
View File

@@ -26,71 +26,33 @@
package user
import (
"fmt"
sdkaws "github.com/aws/aws-sdk-go-v2/aws"
sdkiam "github.com/aws/aws-sdk-go-v2/service/iam"
"github.com/aws/aws-sdk-go-v2/service/iam/types"
iamtps "github.com/aws/aws-sdk-go-v2/service/iam/types"
awshlp "github.com/nabbar/golib/aws/helper"
)
func (cli *client) List() (map[string]string, error) {
out, err := cli.iam.ListUsers(cli.GetContext(), &sdkiam.ListUsersInput{})
if err != nil {
return nil, cli.GetError(err)
} else if out.Users == nil {
return nil, awshlp.ErrorResponse.Error(nil)
} else {
var res = make(map[string]string)
for _, u := range out.Users {
res[*u.UserId] = *u.UserName
var (
err error
res = make(map[string]string, 0)
fct = func(user iamtps.User) bool {
if user.UserName == nil || len(*user.UserName) == 0 {
return true
} else if user.UserId == nil || len(*user.UserId) == 0 {
return true
}
return res, nil
res[*user.UserName] = *user.UserId
return true
}
}
func (cli *client) detachUserFromGroupsAndPolicies(username string) error {
groups, err := cli.iam.ListGroupsForUser(cli.GetContext(), &sdkiam.ListGroupsForUserInput{
UserName: sdkaws.String(username),
})
if err != nil {
return cli.GetError(err)
)
err = cli.Walk("", fct)
return res, err
}
for _, group := range groups.Groups {
_, err := cli.iam.RemoveUserFromGroup(cli.GetContext(), &sdkiam.RemoveUserFromGroupInput{
UserName: sdkaws.String(username),
GroupName: group.GroupName,
})
if err != nil {
return cli.GetError(err)
}
}
attachedPoliciesOutput, err := cli.iam.ListAttachedUserPolicies(cli.GetContext(), &sdkiam.ListAttachedUserPoliciesInput{
UserName: sdkaws.String(username),
})
if err != nil {
return cli.GetError(err)
}
for _, policy := range attachedPoliciesOutput.AttachedPolicies {
_, err := cli.iam.DetachUserPolicy(cli.GetContext(), &sdkiam.DetachUserPolicyInput{
UserName: sdkaws.String(username),
PolicyArn: policy.PolicyArn,
})
if err != nil {
return cli.GetError(err)
}
}
return nil
}
func (cli *client) Walk(prefix string, fct UserFunc) error {
func (cli *client) Walk(prefix string, fct FuncWalkUsers) error {
var (
err error
out *sdkiam.ListUsersOutput
@@ -98,11 +60,19 @@ func (cli *client) Walk(prefix string, fct UserFunc) error {
trk = true
)
if fct == nil {
fct = func(user iamtps.User) bool {
return false
}
}
for trk {
var in = &sdkiam.ListUsersInput{}
if len(prefix) > 0 {
in.PathPrefix = sdkaws.String(prefix)
}
if mrk != nil && len(*mrk) > 0 {
in.Marker = mrk
}
@@ -118,8 +88,8 @@ func (cli *client) Walk(prefix string, fct UserFunc) error {
mrk = nil
}
for _, u := range out.Users {
if !fct(u) {
for i := range out.Users {
if !fct(out.Users[i]) {
return nil
}
}
@@ -132,25 +102,6 @@ func (cli *client) Walk(prefix string, fct UserFunc) error {
return nil
}
func (cli *client) DetachUsers(prefix string) ([]string, error) {
var detachedUsernames []string
err := cli.Walk(prefix, func(user types.User) bool {
if err := cli.detachUserFromGroupsAndPolicies(*user.UserName); err != nil {
fmt.Println(err)
return false
}
detachedUsernames = append(detachedUsernames, *user.UserName)
return true
})
if err != nil {
return nil, err
}
return detachedUsernames, nil
}
func (cli *client) Get(username string) (*iamtps.User, error) {
out, err := cli.iam.GetUser(cli.GetContext(), &sdkiam.GetUserInput{
UserName: sdkaws.String(username),

795
certificates/README.md
View File

@@ -0,0 +1,795 @@
# Certificates Package
This package provides tools and abstractions to manage TLS/SSL certificates, cipher suites, curves, and related configuration for secure communications in Go applications.
## Features
- **Certificate Management**: Load certificates from files or strings, manage certificate pairs, and support for both file-based and in-memory certificates.
- **Root CA and Client CA**: Add and manage root and client certificate authorities from files or strings.
- **TLS Version Control**: Configure minimum and maximum supported TLS versions.
- **Cipher Suites and Curves**: Select and manage cipher suites and elliptic curves for TLS connections.
- **Client Authentication**: Configure client authentication modes.
- **Dynamic and Session Ticket Options**: Enable or disable dynamic record sizing and session tickets.
- **Configuration Inheritance**: Optionally inherit from a default configuration.
## Installation
```bash
go get github.com/nabbar/golib/certificates
```
## Usage
### Basic Example
```go
import (
"github.com/nabbar/golib/certificates"
)
cfg := certificates.Config{
CurveList: []Curves{...},
CipherList: []Cipher{...},
RootCA: []Cert{...},
ClientCA: []Cert{...},
Certs: []Certif{...},
VersionMin: tlsversion.VersionTLS12,
VersionMax: tlsversion.VersionTLS13,
AuthClient: auth.NoClientCert,
InheritDefault: false,
DynamicSizingDisable: false,
SessionTicketDisable: false,
}
tlsConfig := cfg.New().TLS("your.server.com")
```
### Configuration Fields
- `CurveList`: List of elliptic curves to use (e.g., `["X25519", "P256"]`), see [certificates/curves](#certificatescurves-package) for available options.
- `CipherList`: List of cipher suites (e.g., `["RSA_AES_128_GCM_SHA256"]`), see [certificates/cipher](#certificatescipher-package) for available options.
- `RootCA`: List of root CA certificates, supported formats include PEM strings or file paths. See [certificates/ca](#certificatesca-package) for more details.
- `ClientCA`: List of client CA certificates. Supported formats include PEM strings or file paths. See [certificates/ca](#certificatesca-package) for more details.
- `Certs`: List of certificate pairs (key/cert). Supported formats include PEM strings or file paths. See [certificates/certs](#certificatescerts-package) for more details.
- `VersionMin`: Minimum TLS version (e.g., `"1.2"`). See [certificates/tlsversion](#certificatestlsversion-package) for available options.
- `VersionMax`: Maximum TLS version (e.g., `"1.3"`). See [certificates/tlsversion](#certificatestlsversion-package) for available options.
- `AuthClient`: Client authentication mode (e.g., `"none"`, `"require"`). See [certificates/auth](#certificatesauth-package) for available modes.
- `InheritDefault`: Inherit from the default configuration if set to `true`. Can be used to apply a base configuration across multiple TLS configurations.
- `DynamicSizingDisable`: Disable dynamic record sizing if set to `true`. Used to control how TLS records are sized dynamically based on the payload.
- `SessionTicketDisable`: Disable session tickets if set to `true`. Used to control whether session tickets are used for session resumption.
### Methods
- `New() TLSConfig`: Create a new TLS configuration from the current config.
- `AddRootCAString(string) bool`: Add a root CA from a string.
- `AddRootCAFile(string) error`: Add a root CA from a file.
- `AddClientCAString(string) bool`: Add a client CA from a string.
- `AddClientCAFile(string) error`: Add a client CA from a file.
- `AddCertificatePairString(key, cert string) error`: Add a certificate pair from strings.
- `AddCertificatePairFile(keyFile, certFile string) error`: Add a certificate pair from files.
- `SetVersionMin(Version)`: Set the minimum TLS version.
- `SetVersionMax(Version)`: Set the maximum TLS version.
- `SetCipherList([]Cipher)`: Set the list of cipher suites.
- `SetCurveList([]Curves)`: Set the list of elliptic curves.
- `SetDynamicSizingDisabled(bool)`: Enable/disable dynamic record sizing.
- `SetSessionTicketDisabled(bool)`: Enable/disable session tickets.
- `TLS(serverName string) *tls.Config`: Get a `*tls.Config` for use in servers/clients.
### Testing
Tests are written using [Ginkgo](https://onsi.github.io/ginkgo/) and [Gomega](https://onsi.github.io/gomega/).
```bash
ginkgo -cover .
```
## Liens utiles
- [Documentation Ginkgo](https://onsi.github.io/ginkgo/)
- [crypto/tls (Go)](https://pkg.go.dev/crypto/tls)
- [crypto/x509 (Go)](https://pkg.go.dev/crypto/x509)
## Licence
MIT © Nicolas JUHEL
---
# certificates/auth package
The `certificates/auth` package provides types and helpers to configure TLS client authentication for Go applications. It allows you to select, parse, and serialize the client authentication mode for TLS servers, supporting JSON, YAML, TOML, and text formats.
## Features
- Enumerates all standard TLS client authentication modes
- Parse from string, int, or serialized config (JSON/YAML/TOML)
- Serialize/deserialize for config files and environment variables
- Helper functions for mapping between string, code, and TLS types
- Viper decoder hook for config integration
## Main Types
- `ClientAuth`: Enum type for TLS client authentication (wraps `tls.ClientAuthType`)
### Available Modes
- `NoClientCert` (`"none"`): No client certificate required
- `RequestClientCert` (`"request"`): Request client certificate, but do not require
- `RequireAnyClientCert` (`"require"`): Require any client certificate
- `VerifyClientCertIfGiven` (`"verify"`): Verify client certificate if provided
- `RequireAndVerifyClientCert` (`"strict require verify"`): Require and verify client certificate
## Example: Parse and Use ClientAuth
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/auth"
)
func main() {
// Parse from string
ca := auth.Parse("require")
fmt.Println("ClientAuth:", ca.String()) // Output: require
// Use as tls.ClientAuthType
tlsType := ca.TLS()
fmt.Println("TLS ClientAuthType:", tlsType)
}
```
## Example: Marshal/Unmarshal JSON
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/auth"
)
type MyConfig struct {
Auth auth.ClientAuth `json:"authClient"`
}
func main() {
// Marshal
cfg := MyConfig{Auth: auth.RequireAndVerifyClientCert}
b, _ := json.Marshal(cfg)
fmt.Println(string(b)) // {"authClient":"strict require verify"}
// Unmarshal
var cfg2 MyConfig
_ = json.Unmarshal([]byte(`{"authClient":"verify"}`), &cfg2)
fmt.Println(cfg2.Auth.String()) // verify
}
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/auth"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("authClient", "require")
var cfg struct {
Auth auth.ClientAuth
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = auth.ViperDecoderHook()
})
fmt.Println(cfg.Auth.String()) // require
```
## Options
- **String values:** `"none"`, `"request"`, `"require"`, `"verify"`, `"strict require verify"`
- **Int values:** Use `ParseInt(int)` to convert from integer codes
- **Serialization:** Supports JSON, YAML, TOML, text, CBOR
## Advanced
- Use `auth.List()` to get all available modes
- Use `auth.Parse(s string)` to parse from string
- Use `auth.ParseInt(i int)` to parse from int
- Use `auth.ViperDecoderHook()` for config libraries
## Error Handling
Parsing functions return a default value (`NoClientCert`) if the input is invalid. Serialization methods return errors if the format is not supported.
---
# certificates/ca package
The `certificates/ca` package provides tools to parse, manage, and serialize X.509 certificate chains for use as Root or Client Certificate Authorities (CAs) in Go applications. It supports loading certificates from PEM strings or files, and serializing/deserializing in multiple formats (JSON, YAML, TOML, CBOR, text, binary).
## Features
- Parse and manage X.509 certificate chains
- Load certificates from PEM strings or file paths
- Serialize/deserialize in JSON, YAML, TOML, CBOR, text, and binary
- Append certificates to `x509.CertPool`
- Integrate with Viper for configuration
## Main Types
- `Cert`: Interface for certificate chains (implements marshaling/unmarshaling for all supported formats)
## Example: Parse a PEM Certificate Chain
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/ca"
)
func main() {
pem := `-----BEGIN CERTIFICATE-----
MIIB... (your PEM data)
-----END CERTIFICATE-----`
cert, err := ca.Parse(pem)
if err != nil {
panic(err)
}
fmt.Println("Number of certs:", cert.Len())
}
```
## Example: Load Certificate from File Path
If the PEM block contains a file path, the package will load the certificate from the file.
```go
cert, err := ca.Parse("/etc/ssl/certs/ca-cert.pem")
if err != nil {
panic(err)
}
```
## Example: Marshal/Unmarshal JSON
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/ca"
)
type MyConfig struct {
Root ca.Cert `json:"rootCA"`
}
func main() {
pem := `-----BEGIN CERTIFICATE-----...`
cfg := MyConfig{}
_ = json.Unmarshal([]byte(fmt.Sprintf(`{"rootCA":%q}`, pem)), &cfg)
b, _ := json.Marshal(cfg)
fmt.Println(string(b))
}
```
## Example: Append to CertPool
```go
import (
"crypto/x509"
"github.com/nabbar/golib/certificates/ca"
)
cert, _ := ca.Parse(pemString)
pool := x509.NewCertPool()
cert.AppendPool(pool)
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/ca"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("rootCA", "-----BEGIN CERTIFICATE-----...")
var cfg struct {
Root ca.Cert
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = ca.ViperDecoderHook()
})
```
## Options & Methods
- **Parse(str string) (Cert, error)**: Parse a PEM string or file path to a `Cert`
- **ParseByte([]byte) (Cert, error)**: Parse from bytes
- **Cert.Len() int**: Number of certificates in the chain
- **Cert.AppendPool(*x509.CertPool)**: Add all certs to a pool
- **Cert.AppendBytes([]byte) error**: Append certs from bytes
- **Cert.AppendString(string) error**: Append certs from string
- **Cert.String() string**: PEM-encoded chain as string
- **Cert.Marshal/Unmarshal**: Supports Text, Binary, JSON, YAML, TOML, CBOR
## Error Handling
- Returns Go `error` or custom errors (`ErrInvalidPairCertificate`, `ErrInvalidCertificate`)
- Always check returned errors
---
# certificates/certs package
The `certificates/certs` package provides types and utilities for handling X.509 certificate pairs and chains in Go. It supports loading certificates from PEM strings or files, serializing/deserializing in multiple formats (JSON, YAML, TOML, CBOR, text, binary), and converting to `tls.Certificate` for use in TLS servers/clients.
## Features
- Parse certificate pairs (key + cert) or chains (cert + private key in one PEM)
- Load from PEM strings or file paths
- Serialize/deserialize in JSON, YAML, TOML, CBOR, text, and binary
- Convert to `tls.Certificate`
- Helper methods for extracting PEM, checking type, and file origin
- Viper integration for config loading
## Main Types
- `Cert`: Interface for certificate objects (pair or chain)
- `Certif`: Implementation of `Cert`
- `ConfigPair`: Struct for key/cert pair
- `ConfigChain`: String type for PEM chain
## Example: Parse a Certificate Pair
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/certs"
)
func main() {
cert, err := certs.ParsePair("server.key", "server.crt")
if err != nil {
panic(err)
}
fmt.Println("Is pair:", cert.IsPair())
fmt.Println("Is file:", cert.IsFile())
}
```
## Example: Parse a Certificate Chain
```go
cert, err := certs.Parse("chain.pem")
if err != nil {
panic(err)
}
fmt.Println("Is chain:", cert.IsChain())
```
## Example: Marshal/Unmarshal JSON with Certificates Pair & Chain
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/certs"
)
type MyConfig struct {
Cert []certs.Certif `json:"cert"`
}
func main() {
jsonData := `[{"key":"server1.key","pub":"server1.crt"},{"key":"server2.key","pub":"server2.crt"},"server3.pem","server4.pem"]`
var cfg MyConfig
_ = json.Unmarshal([]byte(jsonData), &cfg)
b, _ := json.Marshal(cfg)
fmt.Println(string(b))
}
```
## Example: Get PEM Strings
```go
pub, key, err := cert.Pair()
if err != nil {
panic(err)
}
fmt.Println("Public cert PEM:", pub)
fmt.Println("Private key PEM:", key)
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/certs"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("cert", "chain.pem")
var cfg struct {
Cert certs.Certif
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = certs.ViperDecoderHook()
})
```
## Options & Methods
- **Parse(chain string) (Cert, error)**: Parse a PEM chain or file path
- **ParsePair(key, pub string) (Cert, error)**: Parse a key/cert pair (PEM or file)
- **Cert.TLS() tls.Certificate**: Get as `tls.Certificate`
- **Cert.IsChain() bool**: Is a chain (cert + key in one PEM)
- **Cert.IsPair() bool**: Is a pair (separate key/cert)
- **Cert.IsFile() bool**: Loaded from file(s)
- **Cert.GetCerts() []string**: Get underlying PEMs or file paths
- **Cert.Pair() (pub, key string, error)**: Get PEM strings for cert and key
- **Cert.Chain() (string, error)**: Get full PEM chain
- **Cert.Marshal/Unmarshal**: Supports Text, Binary, JSON, YAML, TOML, CBOR
## Error Handling
- Returns Go `error` or custom errors (`ErrInvalidPairCertificate`, `ErrInvalidCertificate`, `ErrInvalidPrivateKey`)
- Always check returned errors
---
# certificates/cipher package
The `certificates/cipher` package provides types and utilities for handling TLS cipher suites in Go. It allows you to list, parse, serialize, and use cipher suites for configuring TLS servers and clients, supporting multiple serialization formats (JSON, YAML, TOML, CBOR, text).
## Features
- Enumerates all supported TLS cipher suites (TLS 1.2 & 1.3)
- Parse from string or integer
- Serialize/deserialize for config files (JSON, YAML, TOML, CBOR, text)
- Helper methods for code, string, and TLS value conversion
- Viper decoder hook for config integration
## Main Types
- `Cipher`: Enum type for TLS cipher suites (wraps `uint16`)
## Example: List and Parse Cipher Suites
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/cipher"
)
func main() {
// List all supported ciphers
for _, c := range cipher.List() {
fmt.Println("Cipher:", c.String(), "TLS value:", c.TLS())
}
// Parse from string
c := cipher.Parse("ECDHE_RSA_AES_128_GCM_SHA256")
fmt.Println("Parsed cipher:", c.String())
// Parse from int
c2 := cipher.ParseInt(4865) // Example TLS value
fmt.Println("Parsed from int:", c2.String())
}
```
## Example: Marshal/Unmarshal JSON
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/cipher"
)
type MyConfig struct {
Cipher cipher.Cipher `json:"cipher"`
}
func main() {
// Marshal
cfg := MyConfig{Cipher: cipher.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256}
b, _ := json.Marshal(cfg)
fmt.Println(string(b)) // {"cipher":"ecdhe_rsa_aes_128_gcm_sha256"}
// Unmarshal
var cfg2 MyConfig
_ = json.Unmarshal([]byte(`{"cipher":"aes_128_gcm_sha256"}`), &cfg2)
fmt.Println(cfg2.Cipher.String()) // aes_128_gcm_sha256
}
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/cipher"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("cipher", "chacha20_poly1305_sha256")
var cfg struct {
Cipher cipher.Cipher
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = cipher.ViperDecoderHook()
})
fmt.Println(cfg.Cipher.String()) // chacha20_poly1305_sha256
```
## Options & Methods
- **List() []Cipher**: List all supported cipher suites
- **ListString() []string**: List all supported cipher suite names as strings
- **Parse(s string) Cipher**: Parse from string (case-insensitive, flexible format)
- **ParseInt(i int) Cipher**: Parse from integer TLS value
- **Cipher.String() string**: Get string representation
- **Cipher.TLS() uint16**: Get TLS value
- **Cipher.Check() bool**: Check if cipher is supported
- **ViperDecoderHook()**: Viper integration for config loading
## Supported Cipher Suites
- `ecdhe_rsa_aes_128_gcm_sha256`
- `ecdhe_ecdsa_aes_128_gcm_sha256`
- `ecdhe_rsa_aes_256_gcm_sha384`
- `ecdhe_ecdsa_aes_256_gcm_sha384`
- `ecdhe_rsa_chacha20_poly1305_sha256`
- `ecdhe_ecdsa_chacha20_poly1305_sha256`
- `aes_128_gcm_sha256`
- `aes_256_gcm_sha384`
- `chacha20_poly1305_sha256`
- ...and retro-compatible aliases
## Error Handling
- Parsing functions return `Unknown` if the input is invalid.
- Serialization methods return errors if the format is not supported.
---
# certificates/curves package
The `certificates/curves` package provides types and utilities for handling elliptic curves for TLS in Go. It allows you to list, parse, serialize, and use elliptic curves for configuring TLS servers and clients, supporting multiple serialization formats (JSON, YAML, TOML, CBOR, text).
## Features
- Enumerates all supported TLS elliptic curves (X25519, P256, P384, P521)
- Parse from string or integer
- Serialize/deserialize for config files (JSON, YAML, TOML, CBOR, text)
- Helper methods for code, string, and TLS value conversion
- Viper decoder hook for config integration
## Main Types
- `Curves`: Enum type for TLS elliptic curves (wraps `uint16`)
## Example: List and Parse Curves
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/curves"
)
func main() {
// List all supported curves
for _, c := range curves.List() {
fmt.Println("Curve:", c.String(), "TLS value:", c.TLS())
}
// Parse from string
c := curves.Parse("P256")
fmt.Println("Parsed curve:", c.String())
// Parse from int
c2 := curves.ParseInt(29) // Example TLS value for X25519
fmt.Println("Parsed from int:", c2.String())
}
```
## Example: Marshal/Unmarshal JSON
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/curves"
)
type MyConfig struct {
Curve curves.Curves `json:"curve"`
}
func main() {
// Marshal
cfg := MyConfig{Curve: curves.P256}
b, _ := json.Marshal(cfg)
fmt.Println(string(b)) // {"curve":"P256"}
// Unmarshal
var cfg2 MyConfig
_ = json.Unmarshal([]byte(`{"curve":"X25519"}`), &cfg2)
fmt.Println(cfg2.Curve.String()) // X25519
}
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/curves"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("curve", "P384")
var cfg struct {
Curve curves.Curves
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = curves.ViperDecoderHook()
})
fmt.Println(cfg.Curve.String()) // P384
```
## Options & Methods
- **List() []Curves**: List all supported curves
- **ListString() []string**: List all supported curve names as strings
- **Parse(s string) Curves**: Parse from string (case-insensitive, flexible format)
- **ParseInt(i int) Curves**: Parse from integer TLS value
- **Curves.String() string**: Get string representation
- **Curves.TLS() tls.CurveID**: Get TLS value
- **Curves.Check() bool**: Check if curve is supported
- **ViperDecoderHook()**: Viper integration for config loading
## Supported Curves
- `X25519`
- `P256`
- `P384`
- `P521`
## Error Handling
- Parsing functions return `Unknown` if the input is invalid.
- Serialization methods return errors if the format is not supported.
---
# certificates/tlsversion package
The `certificates/tlsversion` package provides types and utilities for handling TLS protocol versions in Go. It allows you to list, parse, serialize, and use TLS versions for configuring secure servers and clients, supporting multiple serialization formats (JSON, YAML, TOML, CBOR, text).
## Features
- Enumerates all supported TLS protocol versions (TLS 1.0, 1.1, 1.2, 1.3)
- Parse from string or integer
- Serialize/deserialize for config files (JSON, YAML, TOML, CBOR, text)
- Helper methods for code, string, and TLS value conversion
- Viper decoder hook for config integration
## Main Types
- `Version`: Enum type for TLS protocol versions (wraps `int`)
## Supported Versions
- `TLS 1.0`
- `TLS 1.1`
- `TLS 1.2`
- `TLS 1.3`
## Example: List and Parse TLS Versions
```go
package main
import (
"fmt"
"github.com/nabbar/golib/certificates/tlsversion"
)
func main() {
// List all supported TLS versions
for _, v := range tlsversion.List() {
fmt.Println("Version:", v.String(), "TLS value:", v.TLS())
}
// Parse from string
v := tlsversion.Parse("1.2")
fmt.Println("Parsed version:", v.String())
// Parse from int
v2 := tlsversion.ParseInt(0x0304) // Example TLS value for 1.3
fmt.Println("Parsed from int:", v2.String())
}
```
## Example: Marshal/Unmarshal JSON
```go
import (
"encoding/json"
"github.com/nabbar/golib/certificates/tlsversion"
)
type MyConfig struct {
Version tlsversion.Version `json:"version"`
}
func main() {
// Marshal
cfg := MyConfig{Version: tlsversion.VersionTLS12}
b, _ := json.Marshal(cfg)
fmt.Println(string(b)) // {"version":"TLS 1.2"}
// Unmarshal
var cfg2 MyConfig
_ = json.Unmarshal([]byte(`{"version":"TLS 1.3"}`), &cfg2)
fmt.Println(cfg2.Version.String()) // TLS 1.3
}
```
## Example: Use with Viper
```go
import (
"github.com/spf13/viper"
"github.com/nabbar/golib/certificates/tlsversion"
"github.com/go-viper/mapstructure/v2"
)
v := viper.New()
v.Set("version", "1.2")
var cfg struct {
Version tlsversion.Version
}
v.Unmarshal(&cfg, func(dc *mapstructure.DecoderConfig) {
dc.DecodeHook = tlsversion.ViperDecoderHook()
})
fmt.Println(cfg.Version.String()) // TLS 1.2
```
## Options & Methods
- **List() []Version**: List all supported TLS versions
- **ListHigh() []Version**: List only high/secure TLS versions (1.2, 1.3)
- **Parse(s string) Version**: Parse from string (case-insensitive, flexible format)
- **ParseInt(i int) Version**: Parse from integer TLS value
- **Version.String() string**: Get string representation (e.g. "TLS 1.2")
- **Version.Code() string**: Get code representation (e.g. "tls_1_2")
- **Version.TLS() uint16**: Get TLS value
- **Version.Check() bool**: Check if version is supported
- **ViperDecoderHook()**: Viper integration for config loading
## Serialization
- Supports JSON, YAML, TOML, CBOR, and text formats for marshaling/unmarshaling.
## Error Handling
- Parsing functions return `VersionUnknown` if the input is invalid.
- Serialization methods return errors if the format is not supported.
---
This documentation covers all main features, options, and usage examples for the `certificates` package and sub packages.

327
cobra/README.md
View File

@@ -1,242 +1,107 @@
# Cobra pakcage
Help build integration of spf13/Cobra lib.
This package will simplify call and use of flags / command for a CLI Tools.
## `cobra` Package
## Exmaple of implement
Make some folder / file like this:
```
/api
|- root.go
|- one_command.go
/pkg
|- common.go
/main.go
```
The `cobra` package provides a wrapper and utility layer around the [spf13/cobra](https://github.com/spf13/cobra) library, simplifying the creation of CLI applications in Go. It adds helpers for configuration management, command completion, error code printing, and flag handling, as well as integration with logging and versioning.
### Main init :
### Features
This `commong.go` file will make the init of version, logger, cobra, viper and config packages.
- Simplified CLI application setup using Cobra
- Automatic version and description management
- Built-in commands for shell completion, configuration file generation, and error code listing
- Extensive helpers for adding flags of various types (string, int, bool, slices, maps, etc.)
- Integration with custom logger and version modules
- Support for persistent and local flags
---
### Main Types
- **Cobra**: Main interface for building CLI applications
- **FuncInit, FuncLogger, FuncViper, FuncPrintErrorCode**: Function types for custom initialization, logging, configuration, and error printing
---
### Quick Start
#### Create a New CLI Application
```go
import (
"strings"
libcbr "github.com/nabbar/golib/cobra"
libcfg "github.com/nabbar/golib/config"
liblog "github.com/nabbar/golib/logger"
libver "github.com/nabbar/golib/version"
libvpr "github.com/nabbar/golib/viper"
"github.com/nabbar/golib/cobra"
"github.com/nabbar/golib/version"
)
```
Some variables :
```go
var(
// Golib Version Package
vrs libver.Version
// Main config / context Golib package
cfg libcfg.Config
// Root logger before config
log liblog.Logger
// Main Cobra / Viper resource
cbr libcbr.Cobra
vpr libvpr.Viper
)
```
This function will init the libs if not set and return the lib resource :
```go
func GetViper() libvpr.Viper {
if vpr == nil {
vpr = libvpr.New()
vpr.SetHomeBaseName(strings.ToLower(GetVersion().GetPackage()))
vpr.SetEnvVarsPrefix(GetVersion().GetPrefix())
vpr.SetDefaultConfig(GetConfig().DefaultConfig)
}
return vpr
}
func GetCobra() libcbr.Cobra {
if cbr == nil {
cbr = libcbr.New()
cbr.SetVersion(GetVersion())
cbr.SetLogger(GetLogger)
cbr.SetViper(GetViper)
}
return cbr
}
func GetConfig() libcfg.Config {
if cfg == nil {
cfg = libcfg.New()
cfg.RegisterFuncViper(GetViper)
}
return cfg
}
func GetVersion() libver.Version {
if vrs == nil {
//vrs = libver.NewVersion(...)
}
return vrs
}
func GetLogger() liblog.Logger {
if log == nil {
log = liblog.New(cfg.Context())
_ = log.SetOptions(&liblog.Options{
DisableStandard: false,
DisableStack: false,
DisableTimestamp: false,
EnableTrace: true,
TraceFilter: GetVersion().GetRootPackagePath(),
DisableColor: false,
LogFile: nil, // TODO
LogSyslog: nil, // TODO
})
log.SetLevel(liblog.InfoLevel)
log.SetSPF13Level(liblog.InfoLevel, nil)
}
return log
}
```
### The root file in command folder
This file will be the global file for cobra. It will config and init the cobra for yours command.
This file will import your `pkg/common.go` file :
```go
import(
"fmt"
"path"
libcns "github.com/nabbar/golib/console"
liberr "github.com/nabbar/golib/errors"
liblog "github.com/nabbar/golib/logger"
compkg "../pkg"
apipkg "../api"
)
```
And will define some flag vars :
```go
var (
// var for config file path to be load by viper
cfgFile string
// var for log verbose
flgVerbose int
// flag for init cobra has error
iniErr liberr.Error
)
```
The `InitCommand` function will initialize the cobra / viper package and will be call by the main init function into you `main/init()` function:
```go
func InitCommand() {
//the initFunc is a function call on init cobra before calling command but after parsing flag / command ...
compkg.GetCobra().SetFuncInit(initConfig)
// must init after the SetFuncinit function
compkg.GetCobra().Init()
// add the Config file flag
compkg.GetLogger().CheckError(liblog.FatalLevel, liblog.DebugLevel, "Add Flag Config File", compkg.GetCobra().SetFlagConfig(true, &cfgFile))
// add the verbose flas
compkg.GetCobra().SetFlagVerbose(true, &flgVerbose)
// Add some generic command
compkg.GetCobra().AddCommandCompletion()
compkg.GetCobra().AddCommandConfigure("", compkg.GetConfig().DefaultConfig)
compkg.GetCobra().AddCommandPrintErrorCode(cmdPrintError)
// Add one custom command. The best is to isolate each command into a specific file
// Each command file, will having a main function to create the cobra command.
// Here we will only call this main function to add each command into the main cobra command like this
compkg.GetCobra().AddCommand(initCustomCommand1())
// Carefully with the `init` function, because you will not manage the order of running each init function.
// To prevent it, the best is to not having init function, but custom init call in the awaiting order into the `main/init` function
}
// this function will be use in the PrintError command to print first ErrorCode => Package
func cmdPrintError(item, value string) {
println(fmt.Sprintf("%s : %s", libcns.PadLeft(item, 15, " "), path.Dir(value)))
}
// initConfig reads in config file and ENV variables if set.
func initConfig() {
// Redefine the root logger level with the verbose flag
switch flgVerbose {
case 0:
compkg.GetLogger().SetLevel(liblog.ErrorLevel)
case 1:
compkg.GetLogger().SetLevel(liblog.WarnLevel)
case 2:
compkg.GetLogger().SetLevel(liblog.InfoLevel)
default:
compkg.GetLogger().SetLevel(liblog.DebugLevel)
}
// Define the config file into viper to load it if is set
iniErr = compkg.GetViper().SetConfigFile(cfgFile)
compkg.GetLogger().CheckError(liblog.FatalLevel, liblog.DebugLevel, "define config file", iniErr)
// try to init viper with config (local file or remote)
if iniErr = compkg.GetViper().Config(liblog.NilLevel, liblog.NilLevel); iniErr == nil {
// register the reload config function with the watch FS function
compkg.GetViper().SetRemoteReloadFunc(func() {
_ = compkg.GetLogger().CheckError(liblog.ErrorLevel, liblog.DebugLevel, "config reload", compkg.GetConfig().Reload())
})
compkg.GetViper().WatchFS(liblog.InfoLevel)
}
// Make the config for this app (cf config package)
apipkg.SetConfig(compkg.GetLogger().GetLevel(), apirtr.GetHandlers())
}
// This is called by main.main() to parse and run the app.
func Execute() {
compkg.GetLogger().CheckError(liblog.FatalLevel, liblog.DebugLevel, "RootCmd Executed", compkg.GetCobra().Execute())
}
```
### The main file of your app
The main file of your app, wil implement the `main/init()` function and the `main/main()` function.
```go
import(
liberr "github.com/nabbar/golib/errors"
liblog "github.com/nabbar/golib/logger"
compkg "./pkg"
cmdpkg "./cmd"
)
func init() {
// Configure the golib error package
liberr.SetModeReturnError(liberr.ErrorReturnStringErrorFull)
// Check the go runtime use to build
compkg.GetLogger().CheckError(liblog.FatalLevel, liblog.DebugLevel, "Checking go version", compkg.GetVersion().CheckGo("1.16", ">="))
// Call the `cmd/InitCommand` function
cmdpkg.InitCommand()
}
func main() {
// run the command Execute function
cmdpkg.Execute()
app := cobra.New()
app.SetVersion(version.New(...)) // Set your version info
app.Init()
// Add commands, flags, etc.
app.Execute()
}
```
#### Add Built-in Commands
- **Completion**: Generates shell completion scripts (bash, zsh, fish, PowerShell)
- **Configure**: Generates a configuration file in JSON, YAML, or TOML
- **Error**: Prints error codes for the application
```go
app.AddCommandCompletion()
app.AddCommandConfigure("conf", "myapp", defaultConfigFunc)
app.AddCommandPrintErrorCode(func(code, desc string) {
fmt.Printf("%s: %s\n", code, desc)
})
```
#### Add Flags
```go
var configPath string
app.SetFlagConfig(true, &configPath)
var verbose int
app.SetFlagVerbose(true, &verbose)
var myString string
app.AddFlagString(false, &myString, "name", "n", "default", "Description of the flag")
```
#### Add Custom Commands
```go
cmd := app.NewCommand("hello", "Say hello", "Prints hello world", "", "")
cmd.Run = func(cmd *cobra.Command, args []string) {
fmt.Println("Hello, world!")
}
app.AddCommand(cmd)
```
---
### API Overview
- **SetVersion(v Version)**: Set version information
- **SetFuncInit(fct FuncInit)**: Set custom initialization function
- **SetLogger(fct FuncLogger)**: Set custom logger
- **SetViper(fct FuncViper)**: Set custom viper config handler
- **SetFlagConfig(persistent, \*string)**: Add a config file flag
- **SetFlagVerbose(persistent, \*int)**: Add a verbose flag
- **AddFlagXXX(...)**: Add various types of flags (see code for full list)
- **AddCommandCompletion()**: Add shell completion command
- **AddCommandConfigure(alias, basename, defaultConfigFunc)**: Add config file generation command
- **AddCommandPrintErrorCode(fct)**: Add error code listing command
- **Execute()**: Run the CLI application
---
### Error Handling
All commands and helpers return standard Go `error` values. Always check and handle errors when using the API.
---
### Notes
- The package is designed to be extensible and integrates with custom logger and version modules from `golib`.
- Configuration file generation supports JSON, YAML, and TOML formats.
- Shell completion scripts can be generated for bash, zsh, fish, and PowerShell.

327
config/README.md
View File

@@ -1,119 +1,246 @@
# Config pakcage
This package will make a global config management package for an app.
This package work wil component who's will make the real intelligence of the config.
For example, the main config file is not existing into this config, because is generated by the sum of all component config part.
# golib/config
## Exmaple of implement
You can follow the initialization define into the [`cobra` package](../cobra/README.md)
Here we will define the intialization of the config package.
This package provides a modular configuration management system for Go applications, supporting dynamic components, context management, lifecycle events, logging, monitoring, and shell integration.
### Main init of config :
## Features
- **Centralized configuration management** using Viper.
- **Dynamic components**: add, remove, start, stop, reload, and manage dependencies.
- **Context management**: cancellation, custom hooks.
- **Lifecycle events**: hooks before/after start, reload, stop.
- **Configurable logging**.
- **Component monitoring**.
- **Integrated shell commands** for component control.
## Installation
Add to your `go.mod`:
```
require github.com/nabbar/golib/config vX.Y.Z
```
## Quick Start
Make a package's file to be called by yours application command or routers
Into this file we will use some import
```go
import (
"fmt"
"net/http"
liberr "github.com/nabbar/golib/errors"
libsts "github.com/nabbar/golib/status"
liblog "github.com/nabbar/golib/logger"
librtr "github.com/nabbar/golib/router"
cmphea "github.com/nabbar/golib/config/components/head"
cmphtp "github.com/nabbar/golib/config/components/http"
cmplog "github.com/nabbar/golib/config/components/log"
cmptls "github.com/nabbar/golib/config/components/tls"
compkg "../pkg"
"github.com/nabbar/golib/config"
"github.com/nabbar/golib/version"
)
func main() {
vrs := version.New("1.0.0")
cfg := config.New(vrs)
// Register your components here
// cfg.ComponentSet("myComponent", myComponentInstance)
// Register hooks, logger, etc.
// cfg.RegisterFuncStartBefore(func() error { ...; return nil })
if err := cfg.Start(); err != nil {
panic(err)
}
// ...
cfg.Shutdown(0)
}
```
Some constant to prevents error on copy/paste :
```go
const (
ConfigKeyHead = "headers"
ConfigKeyHttp = "servers"
ConfigKeyLog = "log"
ConfigKeyTls = "tls"
)
```
## Main Interfaces
This function will init the libs and return the lib resource.
This is an example, but you can create your own component to add it into you config.
The config can have multi instance of same component type but with different config key.
As That, you can for example define a TLS config for HTTP server, a TLS config for a client, ...
- **Config**: main interface, see `config/interface.go`.
- **Component**: to implement for each component, see `config/types/component.go`.
## Lifecycle Methods
- `Start() error`: starts all registered components respecting dependencies.
- `Reload() error`: reloads configuration and components.
- `Stop()`: stops all components cleanly.
- `Shutdown(code int)`: stops everything and exits the process.
## Component Management
- `ComponentSet(key, cpt)`: register a component.
- `ComponentGet(key)`: retrieve a component.
- `ComponentDel(key)`: remove a component.
- `ComponentList()`: get all components.
- `ComponentStart()`, `ComponentStop()`, `ComponentReload()`: global actions.
## Event Hooks
Register functions to be called before/after each lifecycle step:
```go
func SetConfig(lvl liblog.Level, handler map[string]http.Handler) {
compkg.GetConfig().ComponentSet(ConfigKeyHead, cmphea.New())
compkg.GetConfig().ComponentSet(ConfigKeyLog, cmplog.New(lvl, compkg.GetLogger))
compkg.GetConfig().ComponentSet(ConfigKeyTls, cmptls.New())
compkg.GetConfig().ComponentSet(ConfigKeyHttp, cmphtp.New(_ConfigKeyTls, _ConfigKeyLog, handler))
}
// This function will return the router header based of the component header stored into the config
func GetHeader() librtr.Headers {
if cmp := cmphea.Load(compkg.GetConfig().ComponentGet, _ConfigKeyHead); cmp != nil {
return cmp.GetHeaders()
} else {
GetLog().Fatal("component '%s' is not initialized", _ConfigKeyHead)
return nil
}
}
// This function will return the logger for the application.
// If the component is not started, this function will return the root logger
func GetLog() liblog.Logger {
if !compkg.GetConfig().ComponentIsStarted() {
return nil
}
if cmp := cmplog.Load(compkg.GetConfig().ComponentGet, _ConfigKeyLog); cmp != nil {
return cmp.Log()
} else {
compkg.GetLogger().Error("component '%s' is not initialized", _ConfigKeyLog)
return compkg.GetLogger()
}
}
cfg.RegisterFuncStartBefore(func() error { /* ... */ return nil })
cfg.RegisterFuncStopAfter(func() error { /* ... */ return nil })
cfg.RegisterFuncReloadBefore(func() error { /* ... */ return nil })
cfg.RegisterFuncReloadAfter(func() error { /* ... */ return nil })
```
This function will connect the `status` package of golib with the `httpserver` package stored into the config :
## Context and Cancellation
- `Context()`: returns the config context instance.
- `CancelAdd(func())`: register custom functions to call on context cancel.
- `CancelClean()`: clear all registered cancel functions.
## Shell Commands
Expose commands to list, start, stop, and restart components:
```go
// This function can be call by the implementation of Status Package to expose the status of the pool web server
// This function will update the status after each reload the pool web server.
func SetRouteStatus(sts libsts.RouteStatus) {
compkg.GetConfig().RegisterFuncReloadAfter(func() liberr.Error {
var cmp cmphtp.ComponentHttp
// If component has not been started, skill the function
if !compkg.GetConfig().ComponentIsStarted() {
return nil
}
// if cannot access to the Http component, generate an error
if cmp = cmphtp.Load(compkg.GetConfig().ComponentGet, ConfigKeyHttp); cmp == nil {
return ErrorComponentNotInitialized.ErrorParent(fmt.Errorf("component '%s'", ConfigKeyHttp))
}
// Get the Pool Server
pool := cmp.GetPool()
pool.StatusRoute(_ConfigKeyHttp, GetRouteStatusMessage, sts)
// Don't forget to update the component pool
cmp.SetPool(pool)
return nil
})
}
cmds := cfg.GetShellCommand()
// Integrate these into your CLI
```
To generate the config example, just call this function :
Available commands:
- `list`: list all components
- `start`: start components (all or by name)
- `stop`: stop components (all or by name)
- `restart`: restart components (all or by name)
## Signal Handling
Handles system signals (SIGINT, SIGTERM, SIGQUIT) for graceful shutdown via `WaitNotify()`.
## Default Configuration Generation
Generate a default configuration (JSON) for all registered components:
```go
compkg.GetConfig().DefaultConfig()
r := cfg.DefaultConfig()
// r is an io.Reader containing the default config JSON
```
This function will associate the config Key with the component default config into a main buffer to return it.
As that, config can be extended easily with many component and the config is automatically managed.
## Component Interface
To create a component, implement the following interface (see `config/types/component.go`):
```go
type Component interface {
Type() string
Init(key string, ctx libctx.FuncContext, get FuncCptGet, vpr libvpr.FuncViper, vrs libver.Version, log liblog.FuncLog)
DefaultConfig(indent string) []byte
Dependencies() []string
SetDependencies(d []string) error
RegisterFlag(Command *spfcbr.Command) error
RegisterMonitorPool(p montps.FuncPool)
RegisterFuncStart(before, after FuncCptEvent)
RegisterFuncReload(before, after FuncCptEvent)
IsStarted() bool
IsRunning() bool
Start() error
Reload() error
Stop()
}
```
---
## Creating a Config Component
This guide explains how to implement a new config component for the `golib/config` system. Components are modular units that can be registered, started, stopped, reloaded, and monitored within the configuration framework.
### 1. Component Structure
A component should implement the `Component` interface, which defines lifecycle methods, dependency management, configuration, and monitoring hooks.
**Example structure:**
```go
type componentMyType struct {
x libctx.Config[uint8] // Context and config storage
// Add your own fields here (atomic values, state, etc.)
}
```
### 2. Implementing the Interface
Implement the following methods:
- `Type() string`: Returns the component type name.
- `Init(key, ctx, get, vpr, vrs, log)`: Initializes the component with its key, context, dependency getter, viper, version, and logger.
- `DefaultConfig(indent string) []byte`: Returns the default configuration as JSON (or other format).
- `Dependencies() []string`: Lists required component dependencies.
- `SetDependencies(d []string) error`: Sets dependencies.
- `RegisterFlag(cmd *cobra.Command) error`: Registers CLI flags.
- `RegisterMonitorPool(fct montps.FuncPool)`: Registers a monitor pool for health/metrics.
- `RegisterFuncStart(before, after FuncCptEvent)`: Registers hooks for start events.
- `RegisterFuncReload(before, after FuncCptEvent)`: Registers hooks for reload events.
- `IsStarted() bool`: Returns true if started.
- `IsRunning() bool`: Returns true if running.
- `Start() error`: Starts the component.
- `Reload() error`: Reloads the component.
- `Stop()`: Stops the component.
### 3. Configuration Handling
- Use Viper for configuration loading.
- Implement a method to unmarshal and validate the config section for your component.
- Provide a static default config as a JSON/YAML byte slice.
### 4. Lifecycle Management
- Use atomic values or context for thread-safe state.
- Implement logic for `Start`, `Reload`, and `Stop` to manage resources and state.
- Use hooks (`RegisterFuncStart`, `RegisterFuncReload`) to allow custom logic before/after lifecycle events.
### 5. Dependency Management
- Use `Dependencies()` and `SetDependencies()` to declare and manage required components (e.g., TLS, logger).
- Retrieve dependencies using the provided `FuncCptGet` function.
### 6. Monitoring Integration
- Implement `RegisterMonitorPool` to support health and metrics monitoring.
- Use the monitor pool to register and manage monitor instances for your component.
### 7. Example Skeleton
```go
type componentMyType struct {
x libctx.Config[uint8]
// Add fields as needed
}
func (o *componentMyType) Type() string { return "mytype" }
func (o *componentMyType) Init(key string, ctx libctx.FuncContext, get cfgtps.FuncCptGet, vpr libvpr.FuncViper, vrs libver.Version, log liblog.FuncLog) {
o.x.Store( /* ... */ )
}
func (o *componentMyType) DefaultConfig(indent string) []byte { /* ... */ }
func (o *componentMyType) Dependencies() []string { /* ... */ }
func (o *componentMyType) SetDependencies(d []string) error { /* ... */ }
func (o *componentMyType) RegisterFlag(cmd *cobra.Command) error { /* ... */ }
func (o *componentMyType) RegisterMonitorPool(fct montps.FuncPool) { /* ... */ }
func (o *componentMyType) RegisterFuncStart(before, after cfgtps.FuncCptEvent) { /* ... */ }
func (o *componentMyType) RegisterFuncReload(before, after cfgtps.FuncCptEvent) { /* ... */ }
func (o *componentMyType) IsStarted() bool { /* ... */ }
func (o *componentMyType) IsRunning() bool { /* ... */ }
func (o *componentMyType) Start() error { /* ... */ }
func (o *componentMyType) Reload() error { /* ... */ }
func (o *componentMyType) Stop() { /* ... */ }
```
### 8. Registration
Register your component with the config system:
```go
cfg.ComponentSet("myComponentKey", myComponentInstance)
```
### 9. Error Handling
- Define error codes and messages for your component.
- Return errors using the `liberr.Error` type for consistency.
---
**Note:**
- Use atomic values for thread safety.
- Always validate configuration before starting the component.
- Integrate logging and monitoring as needed.
- Follow the modular and decoupled design to ensure maintainability and testability.
- Consider using context for cancellation and timeouts in long-running operations.
- Ensure proper cleanup in the `Stop` method to release resources gracefully.

68
console/README.md
View File

@@ -0,0 +1,68 @@
## `console` Package
The `console` package provides utilities for enhanced console input/output in Go applications. It offers colored output, user prompts, and string formatting helpers to improve CLI user experience.
### Features
- Colored printing and formatting using [fatih/color](https://github.com/fatih/color)
- Customizable color types for standard and prompt outputs
- User input prompts for strings, integers, booleans, URLs, and passwords (with hidden input)
- String padding and tabulated printing helpers
- Error handling with custom error codes
---
### Main Types & Functions
#### Color Output
- **SetColor(col colorType, value ...int)**: Set color attributes for a color type.
- **ColorPrint / ColorPrompt**: Predefined color types for standard and prompt outputs.
- **Print, Println, Printf, Sprintf**: Print text with or without color.
- **BuffPrintf**: Print formatted text to an `io.Writer` with color support.
#### User Prompts
- **PromptString(text string) (string, error)**: Prompt user for a string.
- **PromptInt(text string) (int64, error)**: Prompt user for an integer.
- **PromptBool(text string) (bool, error)**: Prompt user for a boolean.
- **PromptUrl(text string) (\*url.URL, error)**: Prompt user for a URL.
- **PromptPassword(text string) (string, error)**: Prompt user for a password (input hidden).
#### String Formatting Helpers
- **PadLeft/PadRight/PadCenter(str string, len int, pad string) string**: Pad a string to a given length.
- **PrintTabf(tablLevel int, format string, args ...interface{})**: Print formatted text with indentation.
#### Error Handling
- Custom error codes for parameter validation and I/O errors.
- Errors are wrapped using the `github.com/nabbar/golib/errors` package.
---
### Example Usage
```go
import "github.com/nabbar/golib/console"
func main() {
// Set prompt color to green
console.SetColor(console.ColorPrompt, 32)
name, _ := console.PromptString("Enter your name")
console.ColorPrint.Printf("Hello, %s!\n", name)
// Print padded and colored output
padded := console.PadCenter("Welcome", 20, "-")
console.ColorPrint.Println(padded)
}
```
---
### Notes
- Colors can be customized using ANSI attributes.
- Password prompts use hidden input (no echo).
- All prompt functions return errors; always check them in production code.
- Deprecated functions are marked and will be removed in future versions.

91
context/README.md Normal file
View File

@@ -0,0 +1,91 @@
## `context` Package
The `context` package extends Go's standard `context.Context` with advanced configuration and management features. It provides generic, thread-safe context storage, context cloning, merging, and key-value management, making it easier to handle complex application state and configuration.
### Features
- Generic context configuration with type-safe keys
- Thread-safe key-value storage and retrieval
- Context cloning and merging
- Walk and filter stored values
- Integration with Go's `context.Context` interface
### Main Types & Functions
- **Config\[T comparable\]**: Generic interface for context configuration and key-value management.
- **MapManage\[T\]**: Interface for map operations (load, store, delete, clean).
- **FuncContext**: Function type returning a `context.Context`.
- **NewConfig**: Create a new generic context configuration.
#### Example Usage
```go
import "github.com/nabbar/golib/context"
type MyKey string
cfg := context.NewConfig[MyKey](nil)
cfg.Store("myKey", "myValue")
val, ok := cfg.Load("myKey")
```
#### Key Methods
- `SetContext(ctx FuncContext)`
- `Clone(ctx context.Context) Config[T]`
- `Merge(cfg Config[T]) bool`
- `Walk(fct FuncWalk[T]) bool`
- `LoadOrStore(key T, cfg interface{}) (val interface{}, loaded bool)`
- `Clean()`
#### IsolateParent
Creates a new context with cancelation, isolated from the parent.
```go
ctx := context.IsolateParent(parentCtx)
```
---
## `context/gin` Subpackage
The `context/gin` subpackage provides a bridge between Go's context and the [Gin](https://github.com/gin-gonic/gin) web framework. It wraps Gin's `Context` to add context management, signal handling, and metadata utilities for HTTP request handling.
### Features
- Wraps Gin's `Context` with Go's `context.Context`
- Signal-based cancellation (e.g., on OS signals)
- Key-value storage and retrieval for request-scoped data
- Type-safe getters for common types (string, int, bool, etc.)
- Integrated logging support
### Main Types & Functions
- **GinTonic**: Interface combining `context.Context` and Gin's `Context` with extra helpers.
- **New**: Create a new `GinTonic` context from a Gin `Context` and logger.
#### Example Usage
```go
import (
"github.com/nabbar/golib/context/gin"
"github.com/gin-gonic/gin"
"github.com/nabbar/golib/logger"
)
func handler(c *gin.Context) {
ctx := gin.New(c, logger.New(nil))
ctx.Set("userID", 123)
id := ctx.GetInt("userID")
// ...
}
```
#### Key Methods
- `Set(key string, value interface{})`
- `Get(key string) (value interface{}, exists bool)`
- `GetString(key string) string`
- `CancelOnSignal(s ...os.Signal)`
- `SetLogger(log liblog.FuncLog)`

35
context/gin/interface.go
View File

@@ -31,7 +31,7 @@ import (
"os"
"time"
"github.com/gin-gonic/gin"
ginsdk "github.com/gin-gonic/gin"
liblog "github.com/nabbar/golib/logger"
)
@@ -39,7 +39,7 @@ type GinTonic interface {
context.Context
//generic
GinContext() *gin.Context
GinContext() *ginsdk.Context
CancelOnSignal(s ...os.Signal)
//gin context metadata
@@ -60,3 +60,34 @@ type GinTonic interface {
SetLogger(log liblog.FuncLog)
}
func New(c *ginsdk.Context, log liblog.FuncLog) GinTonic {
if c == nil {
c = &ginsdk.Context{
Request: nil,
Writer: nil,
Params: make(ginsdk.Params, 0),
Keys: make(map[string]interface{}),
Errors: make([]*ginsdk.Error, 0),
Accepted: make([]string, 0),
}
}
var (
x context.Context
l context.CancelFunc
)
if c.Request != nil && c.Request.Context() != nil {
x, l = context.WithCancel(c.Request.Context())
} else {
x, l = context.WithCancel(c)
}
return &ctxGinTonic{
l: log,
g: c,
x: x,
c: l,
}
}

46
context/gin/model.go
View File

@@ -32,55 +32,23 @@ import (
"os/signal"
"time"
"github.com/nabbar/golib/logger/level"
"github.com/gin-gonic/gin"
ginsdk "github.com/gin-gonic/gin"
liblog "github.com/nabbar/golib/logger"
loglvl "github.com/nabbar/golib/logger/level"
)
type ctxGinTonic struct {
l liblog.FuncLog
g *gin.Context
g *ginsdk.Context
x context.Context
c context.CancelFunc
}
func NewGinTonic(c *gin.Context, log liblog.FuncLog) GinTonic {
if c == nil {
c = &gin.Context{
Request: nil,
Writer: nil,
Params: make(gin.Params, 0),
Keys: make(map[string]interface{}),
Errors: make([]*gin.Error, 0),
Accepted: make([]string, 0),
}
}
var (
x context.Context
l context.CancelFunc
)
if c.Request != nil && c.Request.Context() != nil {
x, l = context.WithCancel(c.Request.Context())
} else {
x, l = context.WithCancel(c)
}
return &ctxGinTonic{
l: log,
g: c,
x: x,
c: l,
}
}
func (c *ctxGinTonic) SetLogger(fct liblog.FuncLog) {
c.l = fct
}
func (c *ctxGinTonic) log(lvl level.Level, msg string, args ...interface{}) {
func (c *ctxGinTonic) log(lvl loglvl.Level, msg string, args ...interface{}) {
if c.l != nil {
c.l().Entry(lvl, msg, args...).Log()
} else {
@@ -99,11 +67,11 @@ func (c *ctxGinTonic) CancelOnSignal(s ...os.Signal) {
select {
case <-sc:
c.log(level.InfoLevel, "OS Signal received, calling context cancel !")
c.log(loglvl.InfoLevel, "OS Signal received, calling context cancel !")
c.c()
return
case <-c.Done():
c.log(level.InfoLevel, "Context has been closed !")
c.log(loglvl.InfoLevel, "Context has been closed !")
return
}
}()
@@ -125,7 +93,7 @@ func (c *ctxGinTonic) Value(key interface{}) interface{} {
return c.g.Value(key)
}
func (c *ctxGinTonic) GinContext() *gin.Context {
func (c *ctxGinTonic) GinContext() *ginsdk.Context {
return c.g
}

550
database/README.md Normal file
View File

@@ -0,0 +1,550 @@
# golib database
This package provides helpers for database management, including a GORM helper and a generic Key-Value (KV) database abstraction. It is designed to simplify database operations and provide a unified interface for both relational and KV stores.
## Sub-packages Overview
### 1. `gorm`
- **Purpose:** Helper for [GORM](https://gorm.io/) ORM.
- **Features:**
- Configuration struct for GORM options (transactions, connection pool, logger, etc.).
- Database initialization and health check.
- Context and logger registration.
- Monitoring integration.
- **Usage:**
Use the `Config` struct to configure your GORM connection, then use the `New` function to instantiate a database connection.
Example:
```go
import "github.com/nabbar/golib/database/gorm"
cfg := &gorm.Config{ /* ... */ }
db, err := gorm.New(cfg)
```
- See [gorm Subpackage](#gorm-subpackage) for more details.
### 2. `kvdriver`
- **Purpose:** Generic driver interface for Key-Value databases.
- **Features:**
- Define comparison functions for keys (equality, contains, empty).
- Abstracts the basic operations: Get, Set, Delete, List, Search, Walk.
- Allows custom driver implementations by providing function pointers.
- **Usage:**
Implement the required functions and use `kvdriver.New` to create a driver instance.
- See [kvdriver Subpackage](#kvdriver-subpackage) for more details.
### 3. `kvmap`
- **Purpose:** Implements a KV driver using a `map[comparable]any` transformation.
- **Features:**
- Serializes/deserializes models to/from `map[comparable]any` using JSON.
- Useful for stores where data is naturally represented as maps.
- **Usage:**
Use `kvmap.New` to create a driver, providing serialization logic and storage functions.
- See [kvmap Subpackage](#kvmap-subpackage) for more details.
### 4. `kvtable`
- **Purpose:** Table abstraction for KV stores, providing higher-level operations.
- **Features:**
- Operations: Get, Delete, List, Search, Walk.
- Each table is backed by a KV driver.
- **Usage:**
Create a table with `kvtable.New(driver)` and use the table interface for record management.
- See [kvtable Subpackage](#kvtable-subpackage) for more details.
### 5. `kvitem`
- **Purpose:** Represents a single record/item in a KV table.
- **Features:**
- Load, store, remove, and clean a record.
- Change detection between loaded and stored state.
- **Usage:**
Use `kvitem.New(driver, key)` to create an item, then use its methods to manipulate the record.
- See [kvitem Subpackage](#kvitem-subpackage) for more details.
### 6. `kvtypes`
- **Purpose:** Defines generic types and interfaces for KV drivers, items, and tables.
- **Features:**
- `KVDriver`: Interface for drivers.
- `KVItem`: Interface for single records.
- `KVTable`: Interface for table operations.
- Type aliases for walk functions.
---
## Example Usage
```go
import (
"github.com/nabbar/golib/database/kvdriver"
"github.com/nabbar/golib/database/kvtable"
"github.com/nabbar/golib/database/kvitem"
)
// Define your key and model types
type Key string
type Model struct { /* ... */ }
// Implement required functions for your storage backend
// ...
// Create a driver
driver := kvdriver.New(compare, newFunc, getFunc, setFunc, delFunc, listFunc, searchFunc, walkFunc)
// Create a table
table := kvtable.New(driver)
// Get an item
item, err := table.Get("myKey")
if err == nil {
model := item.Get()
// ...
}
```
---
## `kvdriver` Subpackage
The `kvdriver` package provides a generic, extensible driver interface for Key-Value (KV) databases. It allows you to define custom drivers by supplying function pointers for all core operations, and supports advanced key comparison logic.
### Features
- Generic driver interface for any key (`comparable`) and model type
- Customizable comparison functions for keys (equality, contains, empty)
- Abstracts basic KV operations: Get, Set, Delete, List, Search, Walk
- Error codes and messages integrated with the `liberr` package
- Optional support for advanced search and walk operations
---
### Main Types & Functions
#### Comparison Functions
Define how keys are compared and matched:
- `CompareEqual[K]`: func(ref, part K) bool
- `CompareContains[K]`: func(ref, part K) bool
- `CompareEmpty[K]`: func(part K) bool
Create a comparison instance:
```go
cmp := kvdriver.NewCompare(eqFunc, containsFunc, emptyFunc)
```
#### Driver Functions
Implement the following function types for your backend:
- `FuncNew[K, M]`: Create a new driver instance
- `FuncGet[K, M]`: Get a model by key
- `FuncSet[K, M]`: Set a model by key
- `FuncDel[K]`: Delete a model by key
- `FuncList[K]`: List all keys
- `FuncSearch[K]`: (Optional) Search keys by pattern
- `FuncWalk[K, M]`: (Optional) Walk through all records
#### Creating a Driver
Instantiate a driver by providing all required functions:
```go
driver := kvdriver.New(
cmp, // Compare[K]
newFunc, // FuncNew[K, M]
getFunc, // FuncGet[K, M]
setFunc, // FuncSet[K, M]
delFunc, // FuncDel[K]
listFunc, // FuncList[K]
searchFunc, // FuncSearch[K] (optional)
walkFunc, // FuncWalk[K, M] (optional)
)
```
The returned driver implements the `KVDriver[K, M]` interface from `kvtypes`.
---
### Example
```go
import (
"github.com/nabbar/golib/database/kvdriver"
"github.com/nabbar/golib/database/kvtypes"
)
type Key string
type Model struct { /* ... */ }
// Define comparison functions
eq := func(a, b Key) bool { return a == b }
contains := func(a, b Key) bool { return strings.Contains(string(a), string(b)) }
empty := func(a Key) bool { return a == "" }
cmp := kvdriver.NewCompare(eq, contains, empty)
// Implement storage functions (get, set, etc.)
// ...
driver := kvdriver.New(cmp, newFunc, getFunc, setFunc, delFunc, listFunc, nil, nil)
```
---
### Error Handling
All errors are returned as `liberr.Error` with specific codes (e.g., `ErrorBadInstance`, `ErrorGetFunction`). Always check errors after each operation.
---
### Notes
- The package is fully generic and requires Go 1.18+.
- If `FuncSearch` or `FuncWalk` are not provided, default implementations are used (based on `List` and `Get`).
- Integrates with the `kvtypes` package for type definitions.
---
Voici une documentation en anglais pour le package `github.com/nabbar/golib/database/kvmap`, à inclure dans votre `README.md` ou documentation technique.
---
## `kvmap` Subpackage
The `kvmap` package provides a generic Key-Value (KV) driver implementation using a `map[comparable]any` as the underlying storage. It serializes and deserializes models to and from maps using JSON, making it suitable for stores where data is naturally represented as maps.
### Features
- Generic driver for any key (`comparable`) and model type
- Serializes/deserializes models to `map[comparable]any` via JSON
- Customizable storage, retrieval, and management functions
- Supports advanced operations: Get, Set, Delete, List, Search, Walk
- Integrates with the `kvtypes` and `kvdriver` packages
---
### Main Types & Functions
#### Function Types
- `FuncNew[K, M]`: Creates a new driver instance
- `FuncGet[K, MK]`: Retrieves a map for a given key
- `FuncSet[K, MK]`: Stores a map for a given key
- `FuncDel[K]`: Deletes a key
- `FuncList[K]`: Lists all keys
- `FuncSearch[K]`: (Optional) Searches keys by prefix/pattern
- `FuncWalk[K, M]`: (Optional) Walks through all records
#### Creating a Driver
Instantiate a driver by providing all required functions:
```go
import "github.com/nabbar/golib/database/kvmap"
driver := kvmap.New(
cmp, // Compare[K]
newFunc, // FuncNew[K, M]
getFunc, // FuncGet[K, MK]
setFunc, // FuncSet[K, MK]
delFunc, // FuncDel[K]
listFunc, // FuncList[K]
searchFunc, // FuncSearch[K] (optional)
walkFunc, // FuncWalk[K, M] (optional)
)
```
The returned driver implements the `KVDriver[K, M]` interface.
---
### Example
```go
import (
"github.com/nabbar/golib/database/kvmap"
"github.com/nabbar/golib/database/kvdriver"
)
type Key string
type Model struct { /* ... */ }
// Define comparison functions
cmp := kvdriver.NewCompare(eqFunc, containsFunc, emptyFunc)
// Implement storage functions (get, set, etc.)
// ...
driver := kvmap.New(cmp, newFunc, getFunc, setFunc, delFunc, listFunc, nil, nil)
```
---
### Error Handling
All errors are returned as `liberr.Error` with specific codes (e.g., `ErrorBadInstance`, `ErrorGetFunction`). Always check errors after each operation.
---
### Notes
- The package is fully generic and requires Go 1.18+.
- If `FuncSearch` or `FuncWalk` are not provided, default implementations are used (based on `List` and `Get`).
- Serialization and deserialization use JSON under the hood.
- Integrates with the `kvtypes` and `kvdriver` packages for type definitions and comparison logic.
---
Voici une documentation en anglais pour le package `github.com/nabbar/golib/database/kvtable`, à inclure dans votre `README.md` ou documentation technique.
---
## `kvtable` Subpackage
The `kvtable` package provides a high-level table abstraction for Key-Value (KV) stores, built on top of a generic KV driver. It simplifies record management by exposing table-like operations and returning item wrappers for each record.
### Features
- Table abstraction for any key (`comparable`) and model type
- Operations: Get, Delete, List, Search, Walk
- Each table is backed by a pluggable KV driver
- Returns `KVItem` wrappers for each record, enabling further manipulation
- Integrates with the `kvtypes` and `kvitem` packages
---
### Main Types & Functions
#### Creating a Table
Instantiate a table by providing a KV driver:
```go
import (
"github.com/nabbar/golib/database/kvtable"
"github.com/nabbar/golib/database/kvtypes"
)
driver := /* your KVDriver[K, M] implementation */
table := kvtable.New(driver)
```
#### Table Operations
- `Get(key K) (KVItem[K, M], error)`: Retrieve an item by key.
- `Del(key K) error`: Delete an item by key.
- `List() ([]KVItem[K, M], error)`: List all items in the table.
- `Search(pattern K) ([]KVItem[K, M], error)`: Search items by pattern.
- `Walk(fct FuncWalk[K, M]) error`: Walk through all items, applying a function.
Example usage:
```go
item, err := table.Get("myKey")
if err == nil {
model := item.Get()
// ...
}
```
---
### Error Handling
All operations return errors as `liberr.Error` with specific codes (e.g., `ErrorBadDriver`). Always check errors after each operation.
---
### Notes
- The package is fully generic and requires Go 1.18+.
- Each table instance is backed by a driver implementing the `KVDriver` interface.
- Returned `KVItem` wrappers allow further operations like load, store, and change detection.
---
Voici une documentation en anglais pour le package `github.com/nabbar/golib/database/kvitem`, à inclure dans votre `README.md` ou documentation technique.
---
## `kvitem` Subpackage
The `kvitem` package provides a generic wrapper for managing a single record (item) in a Key-Value (KV) store. It is designed to work with any key and model type, and relies on a pluggable KV driver for storage operations. The package offers change detection, atomic state management, and error handling.
### Features
- Generic item wrapper for any key (`comparable`) and model type
- Load, store, remove, and clean a record
- Change detection between loaded and stored state
- Atomic operations for thread safety
- Integrates with the `kvtypes` package for driver abstraction
- Custom error codes and messages
---
### Main Types & Functions
#### Creating an Item
Instantiate a new item by providing a KV driver and a key:
```go
import (
"github.com/nabbar/golib/database/kvitem"
"github.com/nabbar/golib/database/kvtypes"
)
driver := /* your KVDriver[K, M] implementation */
item := kvitem.New(driver, key)
```
#### Item Operations
- `Set(model M)`: Set the model value to be stored.
- `Get() M`: Get the current model (from store or last loaded).
- `Load() error`: Load the model from the backend.
- `Store(force bool) error`: Store the model to the backend. If `force` is true, always writes even if unchanged.
- `Remove() error`: Remove the item from the backend.
- `Clean()`: Reset the loaded and stored model to zero value.
- `HasChange() bool`: Returns true if the stored model differs from the loaded model.
- `Key() K`: Returns the item's key.
#### Example Usage
```go
item := kvitem.New(driver, "user:123")
err := item.Load()
if err == nil {
model := item.Get()
// modify model...
item.Set(model)
if item.HasChange() {
_ = item.Store(false)
}
}
```
---
### Error Handling
All operations return errors as `liberr.Error` with specific codes (e.g., `ErrorLoadFunction`, `ErrorStoreFunction`). Always check errors after each operation.
---
### Notes
- The package is fully generic and requires Go 1.18+.
- Uses atomic values for thread-safe state management.
- Integrates with the `kvtypes` package for driver and item interfaces.
- Change detection is based on deep equality of loaded and stored models.
---
Voici une documentation en anglais pour le package `github.com/nabbar/golib/database/gorm`, à inclure dans votre `README.md` ou documentation technique.
---
## `gorm` Subpackage
The `gorm` package provides helpers for configuring, initializing, and managing [GORM](https://gorm.io/) database connections in Go applications. It offers advanced configuration options, context and logger integration, connection pooling, and monitoring support.
### Features
- Configuration struct for all GORM options (driver, DSN, transactions, pooling, etc.)
- Database initialization and health checking
- Context and logger registration
- Monitoring integration for health and metrics
- Error codes and messages for robust error handling
- Support for multiple database drivers (MySQL, PostgreSQL, SQLite, SQL Server, ClickHouse)
---
### Main Types & Functions
#### `Config` Struct
Defines all configuration options for a GORM database connection:
- `Driver`: Database driver (`mysql`, `psql`, `sqlite`, `sqlserver`, `clickhouse`)
- `Name`: Instance name for status/monitoring
- `DSN`: Data Source Name (connection string)
- Transaction, pooling, and migration options
- Monitoring configuration
#### Example Usage
```go
import (
"github.com/nabbar/golib/database/gorm"
)
cfg := &gorm.Config{
Driver: gorm.DriverMysql,
Name: "main-db",
DSN: "user:pass@tcp(localhost:3306)/dbname",
EnableConnectionPool: true,
PoolMaxIdleConns: 5,
PoolMaxOpenConns: 10,
// ... other options
}
db, err := gorm.New(cfg)
if err != nil {
// handle error
}
defer db.Close()
```
#### Registering Logger and Context
```go
import (
"github.com/nabbar/golib/logger"
"github.com/nabbar/golib/context"
)
cfg.RegisterLogger(func() logger.Logger { /* ... */ }, false, 200*time.Millisecond)
cfg.RegisterContext(func() context.Context { /* ... */ })
```
#### Monitoring
You can enable monitoring and health checks for your database instance:
```go
mon, err := db.Monitor(version)
if err != nil {
// handle error
}
```
#### Error Handling
All errors are returned as `liberr.Error` with specific codes (e.g., `ErrorDatabaseOpen`, `ErrorValidatorError`). Always check errors after each operation.
---
### Supported Drivers
- MySQL
- PostgreSQL
- SQLite
- SQL Server
- ClickHouse
Select the driver using the `Driver` field in the config.
---
### Notes
- The package is thread-safe and uses atomic values for state management.
- Integrates with the `liberr`, `liblog`, and `libctx` packages for error, logging, and context management.
- Monitoring is based on the `monitor` subpackage and can be customized via the config.
---

49
database/kvdriver/compare.go
View File

@@ -1,49 +0,0 @@
package kvdriver
type CompareEqual[K comparable] func(ref, part K) bool
type CompareContains[K comparable] func(ref, part K) bool
type CompareEmpty[K comparable] func(part K) bool
type Compare[K comparable] interface {
IsEqual(ref, part K) bool
IsContains(ref, part K) bool
IsEmpty(part K) bool
}
func NewCompare[K comparable](eq CompareEqual[K], cn CompareContains[K], em CompareEmpty[K]) Compare[K] {
return &cmp[K]{
feq: eq,
fcn: cn,
fem: em,
}
}
type cmp[K comparable] struct {
feq CompareEqual[K]
fcn CompareContains[K]
fem CompareEmpty[K]
}
func (o *cmp[K]) IsEqual(ref, part K) bool {
if o == nil || o.feq == nil {
return false
}
return o.feq(ref, part)
}
func (o *cmp[K]) IsContains(ref, part K) bool {
if o == nil || o.fcn == nil {
return false
}
return o.fcn(ref, part)
}
func (o *cmp[K]) IsEmpty(part K) bool {
if o == nil || o.fem == nil {
return false
}
return o.fem(part)
}

4
database/kvdriver/interface.go
View File

@@ -39,7 +39,7 @@ type FuncSearch[K comparable] func(prefix K) ([]K, error)
type FuncWalk[K comparable, M any] func(fct libkvt.FctWalk[K, M]) error
type drv[K comparable, M any] struct {
cmp Compare[K]
cmp libkvt.Compare[K]
fctNew FuncNew[K, M]
fctGet FuncGet[K, M]
@@ -51,7 +51,7 @@ type drv[K comparable, M any] struct {
fctWlk FuncWalk[K, M] // optional
}
func New[K comparable, M any](cmp Compare[K], fn FuncNew[K, M], fg FuncGet[K, M], fs FuncSet[K, M], fd FuncDel[K], fl FuncList[K], fh FuncSearch[K], fw FuncWalk[K, M]) libkvt.KVDriver[K, M] {
func New[K comparable, M any](cmp libkvt.Compare[K], fn FuncNew[K, M], fg FuncGet[K, M], fs FuncSet[K, M], fd FuncDel[K], fl FuncList[K], fh FuncSearch[K], fw FuncWalk[K, M]) libkvt.KVDriver[K, M] {
return &drv[K, M]{
cmp: cmp, // compare instance

29
database/kvmap/interface.go
View File

@@ -34,22 +34,21 @@ type FuncNew[K comparable, M any] func() libkvt.KVDriver[K, M]
type FuncGet[K comparable, MK comparable] func(key K) (map[MK]any, error)
type FuncSet[K comparable, MK comparable] func(key K, model map[MK]any) error
type FuncDel[K comparable] func(key K) error
type FuncList[K comparable, MK comparable] func() ([]K, error)
type FuncList[K comparable] func() ([]K, error)
type FuncSearch[K comparable] func(prefix K) ([]K, error)
type FuncWalk[K comparable, M any] func(fct libkvt.FctWalk[K, M]) error
type drv[K comparable, MK comparable, M any] struct {
FctNew FuncNew[K, M]
FctGet FuncGet[K, MK]
FctSet FuncSet[K, MK]
FctDel FuncDel[K]
FctList FuncList[K, MK]
}
func New[K comparable, MK comparable, M any](fn FuncNew[K, M], fg FuncGet[K, MK], fs FuncSet[K, MK], fd FuncDel[K], fl FuncList[K, MK]) libkvt.KVDriver[K, M] {
func New[K comparable, MK comparable, M any](cmp libkvt.Compare[K], fn FuncNew[K, M], fg FuncGet[K, MK], fs FuncSet[K, MK], fd FuncDel[K], fl FuncList[K], fh FuncSearch[K], fw FuncWalk[K, M]) libkvt.KVDriver[K, M] {
return &drv[K, MK, M]{
FctNew: fn,
FctGet: fg,
FctSet: fs,
FctDel: fd,
FctList: fl,
cmp: cmp, // compare instance
fctNew: fn,
fctGet: fg,
fctSet: fs,
fctDel: fd,
fctLst: fl,
fctSch: fh, // optional
fctWlk: fw, // optional
}
}

77
database/kvmap/model.go
View File

@@ -32,6 +32,19 @@ import (
libkvt "github.com/nabbar/golib/database/kvtypes"
)
type drv[K comparable, MK comparable, M any] struct {
cmp libkvt.Compare[K]
fctNew FuncNew[K, M]
fctGet FuncGet[K, MK]
fctSet FuncSet[K, MK]
fctDel FuncDel[K]
fctLst FuncList[K]
fctSch FuncSearch[K] // optional
fctWlk FuncWalk[K, M] // optional
}
func (o *drv[K, MK, M]) serialize(model *M, modelMap *map[MK]any) error {
if p, e := json.Marshal(model); e != nil {
return e
@@ -50,19 +63,21 @@ func (o *drv[K, MK, M]) unSerialize(modelMap *map[MK]any, model *M) error {
func (o *drv[K, MK, M]) New() libkvt.KVDriver[K, M] {
return &drv[K, MK, M]{
FctGet: o.FctGet,
FctSet: o.FctSet,
FctDel: o.FctDel,
FctList: o.FctList,
fctGet: o.fctGet,
fctSet: o.fctSet,
fctDel: o.fctDel,
fctLst: o.fctLst,
fctSch: o.fctSch,
fctWlk: o.fctWlk,
}
}
func (o *drv[K, MK, M]) Get(key K, model *M) error {
if o == nil {
return ErrorBadInstance.Error(nil)
} else if o.FctGet == nil {
} else if o.fctGet == nil {
return ErrorGetFunction.Error(nil)
} else if m, e := o.FctGet(key); e != nil {
} else if m, e := o.fctGet(key); e != nil {
return e
} else {
return o.unSerialize(&m, model)
@@ -74,32 +89,46 @@ func (o *drv[K, MK, M]) Set(key K, model M) error {
if o == nil {
return ErrorBadInstance.Error(nil)
} else if o.FctSet == nil {
} else if o.fctSet == nil {
return ErrorSetFunction.Error(nil)
} else if e := o.serialize(&model, &m); e != nil {
return e
} else {
return o.FctSet(key, m)
return o.fctSet(key, m)
}
}
func (o *drv[K, MK, M]) Del(key K) error {
if o == nil {
return ErrorBadInstance.Error(nil)
} else if o.FctDel == nil {
} else if o.fctDel == nil {
return ErrorDelFunction.Error(nil)
} else {
return o.FctDel(key)
return o.fctDel(key)
}
}
func (o *drv[K, MK, M]) List() ([]K, error) {
if o == nil {
return nil, ErrorBadInstance.Error(nil)
} else if o.FctList == nil {
} else if o.fctLst == nil {
return nil, ErrorListFunction.Error(nil)
} else {
return o.FctList()
return o.fctLst()
}
}
func (o *drv[K, MK, M]) Search(pattern K) ([]K, error) {
if o == nil {
return nil, ErrorBadInstance.Error(nil)
} else if o.cmp == nil {
return nil, ErrorBadInstance.Error(nil)
} else if o.cmp.IsEmpty(pattern) {
return o.List()
} else if o.fctSch != nil {
return o.fctSch(pattern)
} else {
return o.fakeSrch(pattern)
}
}
@@ -108,7 +137,29 @@ func (o *drv[K, MK, M]) Walk(fct libkvt.FctWalk[K, M]) error {
return ErrorBadInstance.Error(nil)
} else if fct == nil {
return ErrorFunctionParams.Error(nil)
} else if l, e := o.List(); e != nil {
} else if o.fctWlk == nil {
return o.fakeWalk(fct)
} else {
return o.fctWlk(fct)
}
}
func (o *drv[K, MK, M]) fakeSrch(pattern K) ([]K, error) {
if l, e := o.List(); e != nil {
return nil, e
} else {
var res = make([]K, 0)
for _, k := range l {
if o.cmp.IsContains(k, pattern) {
res = append(res, k)
}
}
return res, nil
}
}
func (o *drv[K, MK, M]) fakeWalk(fct libkvt.FctWalk[K, M]) error {
if l, e := o.List(); e != nil {
return e
} else {
for _, k := range l {

60
ioutils/iowrapper.go → database/kvtypes/compare.go
View File

@@ -1,7 +1,7 @@
/*
* MIT License
*
* Copyright (c) 2019 Nicolas JUHEL
* Copyright (c) 2023 Nicolas JUHEL
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
@@ -21,53 +21,55 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
*
*/
package ioutils
package kvtypes
import "io"
type CompareEqual[K comparable] func(ref, part K) bool
type CompareContains[K comparable] func(ref, part K) bool
type CompareEmpty[K comparable] func(part K) bool
type IOWrapper struct {
iow interface{}
read func(p []byte) []byte
write func(p []byte) []byte
type Compare[K comparable] interface {
IsEqual(ref, part K) bool
IsContains(ref, part K) bool
IsEmpty(part K) bool
}
func NewIOWrapper(ioInput interface{}) *IOWrapper {
return &IOWrapper{
iow: ioInput,
func NewCompare[K comparable](eq CompareEqual[K], cn CompareContains[K], em CompareEmpty[K]) Compare[K] {
return &cmp[K]{
feq: eq,
fcn: cn,
fem: em,
}
}
func (w *IOWrapper) SetWrapper(read func(p []byte) []byte, write func(p []byte) []byte) {
if read != nil {
w.read = read
}
if write != nil {
w.write = write
}
type cmp[K comparable] struct {
feq CompareEqual[K]
fcn CompareContains[K]
fem CompareEmpty[K]
}
func (w IOWrapper) Read(p []byte) (n int, err error) {
if w.read != nil {
return w.iow.(io.Reader).Read(w.read(p))
func (o *cmp[K]) IsEqual(ref, part K) bool {
if o == nil || o.feq == nil {
return false
}
return w.iow.(io.Reader).Read(p)
return o.feq(ref, part)
}
func (w IOWrapper) Write(p []byte) (n int, err error) {
if w.write != nil {
return w.iow.(io.Writer).Write(w.write(p))
func (o *cmp[K]) IsContains(ref, part K) bool {
if o == nil || o.fcn == nil {
return false
}
return w.iow.(io.Writer).Write(p)
return o.fcn(ref, part)
}
func (w IOWrapper) Seek(offset int64, whence int) (int64, error) {
return w.iow.(io.Seeker).Seek(offset, whence)
func (o *cmp[K]) IsEmpty(part K) bool {
if o == nil || o.fem == nil {
return false
}
func (w IOWrapper) Close() error {
return w.iow.(io.Closer).Close()
return o.fem(part)
}

204
duration/README.md Normal file
View File

@@ -0,0 +1,204 @@
# `duration` Package
The `duration` package provides an extended and user-friendly wrapper around Go's `time.Duration`, supporting parsing, formatting, encoding/decoding, and arithmetic operations with additional units (including days). It is designed for easy integration with JSON, YAML, TOML, CBOR, and Viper, and offers helper functions for common duration manipulations.
A `big` subpackage is also available for handling very large durations that exceed the limits of `time.Duration`.<br />See the [`big` subpackage documentation](#big-subpackage) for more details.
## Features
- Extended duration type supporting days (`d`), hours, minutes, seconds, milliseconds, microseconds, nanoseconds
- Parse and format durations as strings (e.g., `5d23h15m13s`)
- Marshal/unmarshal support for JSON, YAML, TOML, CBOR, and text
- Viper decoder hook for configuration loading
- Helper functions for creating durations from days, hours, minutes, etc.
- Truncation helpers (to days, hours, minutes, etc.)
- Range generation using PID controller logic (for smooth transitions)
- Thread-safe and compatible with Go generics
---
## Main Types & Functions
### `Duration` Type
A custom type based on `time.Duration`:
```go
type Duration time.Duration
```
### Creating Durations
```go
import "github.com/nabbar/golib/duration"
d1 := duration.Days(2)
d2 := duration.Hours(5)
d3 := duration.ParseDuration(time.Hour * 3)
d4, err := duration.Parse("1d2h30m")
```
### Parsing and Formatting
- `Parse(s string) (Duration, error)`: Parse a duration string (supports `d`, `h`, `m`, `s`, etc.)
- `ParseByte([]byte) (Duration, error)`: Parse from byte slice
- `String() string`: Format as string (e.g., `2d5h`)
- `Time() time.Duration`: Convert to standard Go duration
### Encoding/Decoding
Supports JSON, YAML, TOML, CBOR, and text:
```go
type Example struct {
Value duration.Duration `json:"value" yaml:"value" toml:"value"`
}
// Marshal to JSON
b, _ := json.Marshal(Example{Value: duration.Days(1) + duration.Hours(2)})
// Unmarshal from YAML, TOML, etc.
```
### Viper Integration
Use the provided decoder hook for Viper:
```go
import "github.com/nabbar/golib/duration"
cfg := viper.New()
cfg.Set("timeout", "2d3h")
var d duration.Duration
cfg.UnmarshalKey("timeout", &d, viper.DecodeHook(duration.ViperDecoderHook()))
```
### Truncation Helpers
- `TruncateDays()`, `TruncateHours()`, `TruncateMinutes()`, etc.
### Range Generation
Generate a range of durations between two values using PID controller logic:
```go
r := d1.RangeTo(d2, rateP, rateI, rateD)
rDef := d1.RangeDefTo(d2) // uses default PID rates
```
---
## Error Handling
All parsing and conversion functions return standard Go `error` values. Always check errors when parsing or decoding durations.
---
## Notes
- Duration strings support days (`d`), which is not available in Go's standard library.
- The package is compatible with Go 1.18+ and supports generics.
- Integrates with `github.com/go-viper/mapstructure/v2` for configuration decoding.
---
## `big` Subpackage
The `big` subpackage provides an extended duration type supporting very large time intervals, including days, and offers advanced parsing, formatting, encoding/decoding, arithmetic, and range generation. It is designed for scenarios where Go's standard `time.Duration` is insufficient due to its limited range.
### Features
- Extended duration type (`Duration`) supporting days (`d`), hours, minutes, seconds
- Parse and format durations as strings (e.g., `5d23h15m13s`)
- Marshal/unmarshal support for JSON, YAML, TOML, CBOR, and text
- Viper decoder hook for configuration loading
- Helper functions for creating durations from days, hours, minutes, seconds
- Truncation and rounding helpers (to days, hours, minutes)
- Range generation using PID controller logic (for smooth transitions)
- Thread-safe and compatible with Go generics
---
### Main Types & Functions
#### `Duration` Type
A custom type based on `int64`, supporting very large values:
```go
type Duration int64
```
#### Creating Durations
```go
import "github.com/nabbar/golib/duration/big"
d1 := big.Days(2)
d2 := big.Hours(5)
d3 := big.ParseDuration(time.Hour * 3)
d4, err := big.Parse("1d2h30m")
```
#### Parsing and Formatting
- `Parse(s string) (Duration, error)`: Parse a duration string (supports `d`, `h`, `m`, `s`)
- `ParseByte([]byte) (Duration, error)`: Parse from byte slice
- `String() string`: Format as string (e.g., `2d5h`)
- `Time() (time.Duration, error)`: Convert to standard Go duration (with overflow check)
#### Encoding/Decoding
Supports JSON, YAML, TOML, CBOR, and text:
```go
type Example struct {
Value big.Duration `json:"value" yaml:"value" toml:"value"`
}
// Marshal to JSON
b, _ := json.Marshal(Example{Value: big.Days(1) + big.Hours(2)})
// Unmarshal from YAML, TOML, etc.
```
#### Viper Integration
Use the provided decoder hook for Viper:
```go
import "github.com/nabbar/golib/duration/big"
cfg := viper.New()
cfg.Set("timeout", "2d3h")
var d big.Duration
cfg.UnmarshalKey("timeout", &d, viper.DecodeHook(big.ViperDecoderHook()))
```
#### Truncation & Rounding Helpers
- `TruncateDays()`, `TruncateHours()`, `TruncateMinutes()`
- `Round(unit Duration) Duration`
#### Range Generation
Generate a range of durations between two values using PID controller logic:
```go
r := d1.RangeTo(d2, rateP, rateI, rateD)
rDef := d1.RangeDefTo(d2) // uses default PID rates
```
---
### Error Handling
All parsing and conversion functions return standard Go `error` values. Always check errors when parsing or decoding durations.
---
### Notes
- Duration strings support days (`d`), which is not available in Go's standard library.
- The package is compatible with Go 1.18+ and supports generics.
- Integrates with `github.com/go-viper/mapstructure/v2` for configuration decoding.
- Maximum supported value: `106,751,991,167,300d15h30m7s`.

0
duration/big_suite_test.go → duration/duration_suite_test.go
View File

0
duration/big_test.go → duration/duration_test.go
View File

493
encoding/README.md
View File

@@ -0,0 +1,493 @@
## `encoding` Package
The `encoding` package provides a unified interface and common utilities for various encoding and cryptographic operations. It serves as the entry point for several specialized subpackages, each implementing a specific encoding or cryptographic algorithm.
### Overview
This package defines the `Coder` interface, which standardizes encoding and decoding operations across different algorithms. The subpackages implement this interface for specific use cases, such as encryption, hashing, hexadecimal encoding, and random data generation.
### Main Features
- Common `Coder` interface for encoding/decoding bytes and streams
- Support for encoding and decoding via both byte slices and `io.Reader`/`io.Writer`
- Memory management with a `Reset` method
- Extensible design for adding new encoding algorithms
### Subpackages
The following subpackages provide concrete implementations of the `Coder` interface and related utilities:
- **aes**: Symmetric encryption and decryption using the AES algorithm. See the [`aes` subpackage](#aes-subpackage) for details.
- **hexa**: Hexadecimal encoding and decoding. See the [`hexa` subpackage](#hexa-subpackage) for details.
- **mux**: Multiplexed encoding, allowing composition of multiple encoders/decoders. See the [`mux` subpackage](#mux-subpackage) for details.
- **randRead**: Secure random byte generation for cryptographic use. See the [`randRead` subpackage](#randread-subpackage) for details.
- **sha256**: SHA-256 hashing and verification.
Refer to each subpackage's documentation for detailed usage, configuration, and examples.
### The `Coder` Interface
All subpackages implement the following interface:
```go
type Coder interface {
Encode(p []byte) []byte
Decode(p []byte) ([]byte, error)
EncodeReader(r io.Reader) io.ReadCloser
DecodeReader(r io.Reader) io.ReadCloser
EncodeWriter(w io.Writer) io.WriteCloser
DecodeWriter(w io.Writer) io.WriteCloser
Reset()
}
```
This interface allows you to:
- Encode or decode data in memory (`[]byte`)
- Encode or decode data streams (`io.Reader`/`io.Writer`)
- Release resources with `Reset()`
### Usage Example
To use an encoding algorithm, import the relevant subpackage and instantiate its coder:
```go
import (
"github.com/nabbar/golib/encoding/aes"
)
coder := aes.NewCoder(key)
encoded := coder.Encode([]byte("my data"))
decoded, err := coder.Decode(encoded)
```
### Notes
- Each subpackage provides its own constructor and configuration options.
- Always check for errors when decoding or working with streams.
- Use the `Reset()` method to free resources when done.
---
## `aes` Subpackage
The `aes` subpackage provides symmetric encryption and decryption using the AES-GCM algorithm. It implements the common `Coder` interface from the parent `encoding` package, allowing easy integration for secure data encoding/decoding in memory or via streams.
### Features
- AES-GCM encryption and decryption (256-bit key, 12-byte nonce)
- Secure random key and nonce generation
- Hexadecimal encoding/decoding for keys and nonces
- Implements the `Coder` interface for byte slices and `io.Reader`/`io.Writer`
- Thread-safe and stateless design
- Resource cleanup with `Reset()`
---
### Main Types & Functions
#### Key and Nonce Management
- `GenKey() ([32]byte, error)`: Generate a secure random 256-bit AES key.
- `GenNonce() ([12]byte, error)`: Generate a secure random 12-byte nonce.
- `GetHexKey(s string) ([32]byte, error)`: Decode a hex string to a 256-bit key.
- `GetHexNonce(s string) ([12]byte, error)`: Decode a hex string to a 12-byte nonce.
#### Creating a Coder
- `New(key [32]byte, nonce [12]byte) (encoding.Coder, error)`: Create a new AES-GCM coder instance with the given key and nonce.
#### Example Usage
```go
import (
"github.com/nabbar/golib/encoding/aes"
)
key, _ := aes.GenKey()
nonce, _ := aes.GenNonce()
coder, err := aes.New(key, nonce)
if err != nil {
// handle error
}
defer coder.Reset()
// Encrypt data
ciphertext := coder.Encode([]byte("my secret data"))
// Decrypt data
plaintext, err := coder.Decode(ciphertext)
```
#### Stream Encoding/Decoding
- `EncodeReader(r io.Reader) io.ReadCloser`: Returns a reader that encrypts data from `r`.
- `DecodeReader(r io.Reader) io.ReadCloser`: Returns a reader that decrypts data from `r`.
- `EncodeWriter(w io.Writer) io.WriteCloser`: Returns a writer that encrypts data to `w`.
- `DecodeWriter(w io.Writer) io.WriteCloser`: Returns a writer that decrypts data to `w`.
---
### Error Handling
- All decoding and stream operations may return errors (e.g., invalid buffer size, decryption failure).
- Always check errors when decoding or using stream interfaces.
---
### Notes
- The key must be 32 bytes (256 bits) and the nonce 12 bytes, as required by AES-GCM.
- Use `Reset()` to clear sensitive data from memory when done.
- For security, never reuse the same key/nonce pair for different data.
---
## `hexa` Subpackage
The `hexa` subpackage provides hexadecimal encoding and decoding utilities, implementing the common `Coder` interface from the parent `encoding` package. It allows you to encode and decode data as hexadecimal strings, both in memory and via streaming interfaces.
### Features
- Hexadecimal encoding and decoding for byte slices
- Stream encoding/decoding via `io.Reader` and `io.Writer`
- Implements the `Coder` interface for easy integration
- Error handling for invalid buffer sizes and decoding errors
- Stateless and thread-safe design
---
### Main Types & Functions
#### Creating a Coder
Instantiate a new hexadecimal coder:
```go
import (
"github.com/nabbar/golib/encoding/hexa"
)
coder := hexa.New()
```
#### Encoding and Decoding
- `Encode(p []byte) []byte`: Encodes a byte slice to its hexadecimal representation.
- `Decode(p []byte) ([]byte, error)`: Decodes a hexadecimal byte slice back to its original bytes.
#### Stream Interfaces
- `EncodeReader(r io.Reader) io.ReadCloser`: Returns a reader that encodes data from `r` to hexadecimal.
- `DecodeReader(r io.Reader) io.ReadCloser`: Returns a reader that decodes hexadecimal data from `r`.
- `EncodeWriter(w io.Writer) io.WriteCloser`: Returns a writer that encodes data to hexadecimal and writes to `w`.
- `DecodeWriter(w io.Writer) io.WriteCloser`: Returns a writer that decodes hexadecimal data and writes to `w`.
#### Example Usage
```go
coder := hexa.New()
// Encode bytes
encoded := coder.Encode([]byte("Hello World"))
// Decode bytes
decoded, err := coder.Decode(encoded)
// Stream encoding
r := coder.EncodeReader(myReader)
defer r.Close()
// Stream decoding
w := coder.DecodeWriter(myWriter)
defer w.Close()
```
---
### Error Handling
- Decoding returns an error if the input is not valid hexadecimal.
- Stream operations may return errors for invalid buffer sizes or I/O issues.
- Use the `Reset()` method to release any resources (no-op for this stateless implementation).
---
### Notes
- The package is stateless and safe for concurrent use.
- Buffer sizes must be sufficient for encoding/decoding operations; otherwise, an error is returned.
- Always check errors when decoding or using stream interfaces.
---
## `mux` Subpackage
The `mux` subpackage provides multiplexing and demultiplexing utilities for encoding and decoding data streams over a single `io.Writer` or `io.Reader`. It allows you to send and receive data on multiple logical channels, identified by a key, through a single stream. This is useful for scenarios where you need to transmit different types of data or messages over the same connection.
### Features
- Multiplex multiple logical channels into a single stream
- Demultiplex a stream into multiple channels based on a key
- Channel identification using a `rune` key
- CBOR serialization and hexadecimal encoding for data blocks
- Thread-safe and efficient design
- Error handling for invalid channels and stream issues
---
### Main Types & Functions
#### Multiplexer
The `Multiplexer` interface allows you to create logical channels for writing data:
```go
type Multiplexer interface {
NewChannel(key rune) io.Writer
}
```
- `NewChannel(key rune) io.Writer`: Returns an `io.Writer` for the given channel key. Data written to this writer is multiplexed into the main stream.
**Example:**
```go
import (
"github.com/nabbar/golib/encoding/mux"
)
muxer := mux.NewMultiplexer(myWriter, '\n')
chA := muxer.NewChannel('a')
chB := muxer.NewChannel('b')
chA.Write([]byte("data for channel A"))
chB.Write([]byte("data for channel B"))
```
#### Demultiplexer
The `DeMultiplexer` interface allows you to register output channels and read data from the main stream:
```go
type DeMultiplexer interface {
io.Reader
Copy() error
NewChannel(key rune, w io.Writer)
}
```
- `NewChannel(key rune, w io.Writer)`: Registers an `io.Writer` for a given channel key. Data for this key will be written to the provided writer.
- `Copy() error`: Continuously reads from the main stream and dispatches data to the correct channel writers. Intended to be run in a goroutine.
**Example:**
```go
dmx := mux.NewDeMultiplexer(myReader, '\n', 0)
bufA := &bytes.Buffer{}
bufB := &bytes.Buffer{}
dmx.NewChannel('a', bufA)
dmx.NewChannel('b', bufB)
go dmx.Copy()
// bufA and bufB will receive their respective data
```
#### Construction
- `NewMultiplexer(w io.Writer, delim byte) Multiplexer`: Creates a new multiplexer with the given writer and delimiter.
- `NewDeMultiplexer(r io.Reader, delim byte, size int) DeMultiplexer`: Creates a new demultiplexer with the given reader, delimiter, and buffer size.
---
### Data Format
Each data block is serialized using CBOR and includes:
- `K`: The channel key (`rune`)
- `D`: The data payload (hexadecimal encoded)
A delimiter byte is appended to each block to separate messages.
---
### Error Handling
- Returns errors for invalid instances or unknown channel keys.
- `Copy()` returns any error encountered during reading or writing, except for `io.EOF` which is ignored.
---
### Notes
- The package is suitable for use with network sockets, files, or any stream-based transport.
- Always register channels before calling `Copy()` on the demultiplexer.
- The delimiter should not appear in the encoded data.
---
## `randRead` Subpackage
The `randRead` subpackage provides a utility for creating a random byte stream reader from a remote or dynamic source. It is designed to wrap any function that returns an `io.ReadCloser` (such as an HTTP request or a cryptographic random source) and expose it as a buffered, reusable `io.ReadCloser` interface.
### Features
- Wraps any remote or dynamic byte stream as an `io.ReadCloser`
- Buffers data for efficient reading
- Automatically refreshes the underlying source when needed
- Thread-safe using atomic values
- Simple integration with any function returning `io.ReadCloser`
---
### Main Types & Functions
#### `FuncRemote` Type
A function type that returns an `io.ReadCloser` and an error:
```go
type FuncRemote func() (io.ReadCloser, error)
```
#### Creating a Random Reader
Use the `New` function to create a new random reader from a remote source:
```go
import "github.com/nabbar/golib/encoding/randRead"
reader := randRead.New(func() (io.ReadCloser, error) {
// Return your io.ReadCloser here (e.g., HTTP response body, random source, etc.)
})
```
- The provided function will be called whenever the reader needs to fetch new data.
#### Example Usage
```go
import (
"github.com/nabbar/golib/encoding/randRead"
"crypto/rand"
"io"
)
reader := randRead.New(func() (io.ReadCloser, error) {
// Wrap crypto/rand.Reader as an io.ReadCloser
return io.NopCloser(rand.Reader), nil
})
buf := make([]byte, 16)
n, err := reader.Read(buf)
// Use buf[0:n] as random data
_ = reader.Close()
```
---
### How It Works
- The random reader buffers data from the remote source using a `bufio.Reader`.
- If the buffer is empty or the underlying source is exhausted, it automatically calls the provided function to obtain a new `io.ReadCloser`.
- The reader is thread-safe and can be used concurrently.
---
### Error Handling
- If the remote function returns an error, the reader will propagate it on `Read`.
- Always check errors when reading or closing the reader.
---
### Notes
- The package is suitable for scenarios where you need a continuous or on-demand random byte stream from a remote or dynamic source.
- The underlying source is closed and replaced automatically as needed.
- Use `Close()` to release resources when done.
---
## `sha256` Subpackage
The `sha256` subpackage provides a simple and unified interface for computing SHA-256 hashes, implementing the common `Coder` interface from the parent `encoding` package. It allows you to hash data in memory or via streaming interfaces, making it easy to integrate SHA-256 hashing into your applications.
### Features
- SHA-256 hashing for byte slices and data streams
- Implements the `Coder` interface for compatibility with other encoding packages
- Stateless and thread-safe design
- Resource cleanup with `Reset()`
- Stream support for `io.Reader` and `io.Writer` (encoding only)
---
### Main Types & Functions
#### Creating a Coder
Instantiate a new SHA-256 coder:
```go
import (
"github.com/nabbar/golib/encoding/sha256"
)
coder := sha256.New()
```
#### Hashing Data
- `Encode(p []byte) []byte`: Computes the SHA-256 hash of the input byte slice and returns the hash as a byte slice.
- `Decode(p []byte) ([]byte, error)`: Not supported; always returns an error (SHA-256 is not reversible).
#### Stream Interfaces
- `EncodeReader(r io.Reader) io.ReadCloser`: Returns a reader that passes data through and updates the hash state. Use `Encode(nil)` after reading to get the hash.
- `EncodeWriter(w io.Writer) io.WriteCloser`: Returns a writer that writes data to `w` and updates the hash state. Use `Encode(nil)` after writing to get the hash.
- `DecodeReader(r io.Reader) io.ReadCloser`: Not supported; always returns `nil`.
- `DecodeWriter(w io.Writer) io.WriteCloser`: Not supported; always returns `nil`.
#### Example Usage
```go
coder := sha256.New()
// Hash a byte slice
hash := coder.Encode([]byte("Hello World"))
// Use with io.Writer
buf := &bytes.Buffer{}
w := coder.EncodeWriter(buf)
w.Write([]byte("Hello World"))
w.Close()
hash = coder.Encode(nil) // Get the hash after writing
// Use with io.Reader
r := coder.EncodeReader(bytes.NewReader([]byte("Hello World")))
io.ReadAll(r)
r.Close()
hash = coder.Encode(nil) // Get the hash after reading
```
#### Resetting State
- `Reset()`: Resets the internal hash state, allowing reuse of the coder for new data.
---
### Error Handling
- `Decode` and stream decoding methods are not supported and will return an error or `nil`.
- Always check for errors when using unsupported methods.
---
### Notes
- SHA-256 is a one-way hash function; decoding is not possible.
- The package is stateless and safe for concurrent use.
- Use `Reset()` to clear the internal state before reusing the coder.

247
errors/README.md
View File

@@ -1,60 +1,221 @@
# Error package
This package allows extending go error interface to add parent tree, code of error, trace, ...
This package still compatible with error interface and can be customized to define what information is return for standard go error implementation.
# `errors` Package
The main mind of this package is to be simple use, quickly implement and having more capabilities than go error interface
The `errors` package provides a comprehensive framework for error handling in Go, supporting error codes, messages, parent/child error relationships, stack traces, and integration with web frameworks like Gin. It is designed to facilitate structured, traceable, and user-friendly error management in complex applications.
## Example of implement
You will find many uses of this package in the `golib` repos.
## Features
- Custom error type with code, message, and parent errors
- Error code registration and message mapping
- Stack trace capture and filtering
- Error wrapping and unwrapping (compatible with Go's standard `errors` package)
- Flexible error formatting modes (code, message, trace, etc.)
- Integration with Gin for HTTP error responses
- Utilities for error lists, code checks, and string search
---
## Main Types & Interfaces
### `Error` Interface
Represents a single error with code, message, trace, and parent errors. Key methods:
- `IsCode(code CodeError) bool`: Checks if the error has the given code.
- `HasCode(code CodeError) bool`: Checks if the error or any parent has the code.
- `GetCode() CodeError`: Returns the error code.
- `GetParentCode() []CodeError`: Returns all codes in the error chain.
- `Is(e error) bool`: Checks if the error matches another error (compatible with `errors.Is`).
- `HasError(err error) bool`: Checks if the error or any parent matches the given error.
- `HasParent() bool`: Returns true if there are parent errors.
- `GetParent(withMainError bool) []error`: Returns all parent errors.
- `Map(fct FuncMap) bool`: Applies a function to the error and all parents.
- `ContainsString(s string) bool`: Checks if the message contains a substring.
- `Add(parent ...error)`: Adds parent errors.
- `SetParent(parent ...error)`: Replaces parent errors.
- `Code() uint16`: Returns the code as uint16.
- `CodeSlice() []uint16`: Returns all codes in the chain.
- `CodeError(pattern string) string`: Formats the error with code and message.
- `CodeErrorTrace(pattern string) string`: Formats with code, message, and trace.
- `Error() string`: Returns the error string (format depends on mode).
- `StringError() string`: Returns the error message.
- `GetError() error`: Returns a standard error.
- `Unwrap() []error`: Returns parent errors for Go error unwrapping.
- `GetTrace() string`: Returns the stack trace.
- `Return(r Return)`: Fills a `Return` struct for API responses.
### `Errors` Interface
- `ErrorsLast() error`: Returns the last error.
- `ErrorsList() []error`: Returns all errors.
### `Return` and `ReturnGin` Interfaces
For API error responses, including Gin integration.
---
## Error Creation & Usage
### Creating Errors
We will create an `error.go` file into any package, and we will construct it like this :
```go
import errors "github.com/nabbar/golib/errors"
import "github.com/nabbar/golib/errors"
err := errors.New(1001, "Something went wrong")
err2 := errors.New(1002, "Another error", err)
```
### Wrapping and Checking Errors
```go
if errors.IsCode(err, 1001) {
// handle specific error code
}
if errors.ContainsString(err, "wrong") {
// handle error containing substring
}
```
### Error Formatting Modes
Set the global error formatting mode:
```go
errors.SetModeReturnError(errors.ErrorReturnCodeErrorTrace)
```
Modes include: code only, code+message, code+message+trace, message only, etc.
### Gin Integration
```go
import "github.com/gin-gonic/gin"
var r errors.DefaultReturn
err.Return(&r)
r.GinTonicAbort(ctx, 500)
```
---
## Error Codes & Messages
- Register custom error codes and messages using `RegisterIdFctMessage`.
- Retrieve code locations with `GetCodePackages`.
---
## Stack Traces
- Each error captures the file, line, and function where it was created.
- Traces are filtered to remove vendor and runtime paths.
---
## Notes
- Fully compatible with Go's `errors.Is` and `errors.As`.
- Supports error chaining and parent/child relationships.
- Designed for use in both CLI and web applications.
---
Voici une section de documentation en anglais pour aider les développeurs à définir et enregistrer des codes d'erreur personnalisés avec le package `github.com/nabbar/golib/errors` :
---
## Defining and Registering Custom Error Codes
To create your own error codes and messages with the `errors` package, follow these steps:
### 1. Define Error Code Constants
Define your error codes as constants of type `liberr.CodeError`. Use an offset (e.g., your package's minimum code) to avoid collisions with other packages:
```go
import liberr "github.com/nabbar/golib/errors"
const (
// create const for code as type CodeError to use features func
// we init the iota number to an minimal free code
// golib/errors expose the minimal available iota with const : errors.MIN_AVAILABLE
// with this method, all code uint will be predictable
EMPTY_PARAMS errors.CodeError = iota + errors.MIN_AVAILABLE
MY_ERROR
// add here all const you will use as code to identify error with code & message
MyErrorInvalidParam liberr.CodeError = iota + liberr.MinPkgMyFeature
MyErrorFileNotFound
MyErrorProcessingFailed
)
```
### 2. Register Message Retrieval Function
Implement a function that returns a message for each error code, and register it using `liberr.RegisterIdFctMessage` in an `init()` function. Before registering, check for code collisions with `liberr.ExistInMapMessage`:
```go
func init() {
if liberr.ExistInMapMessage(MyErrorInvalidParam) {
panic(fmt.Errorf("error code collision with package myfeature"))
}
liberr.RegisterIdFctMessage(MyErrorInvalidParam, getMyFeatureErrorMessage)
}
func getMyFeatureErrorMessage(code liberr.CodeError) string {
switch code {
case MyErrorInvalidParam:
return "invalid parameter provided"
case MyErrorFileNotFound:
return "file not found"
case MyErrorProcessingFailed:
return "processing failed"
}
return liberr.NullMessage
}
```
### 3. Usage Example
Create and use your custom errors as follows:
```go
err := MyErrorInvalidParam.Error(fmt.Errorf("additional context: %s", "details"))
```
And check for specific error codes or messages:
```go
import (
"fmt"
liberr "github.com/nabbar/golib/errors"
)
func init() {
// register your function getMessage
errors.RegisterFctMessage(getMessage)
if liberr.IsCode(err, MyErrorInvalidParam) {
// handle specific error
}
// This function will return the message of one error code
func getMessage(code errors.CodeError) (message string) {
switch code {
case EMPTY_PARAMS:
return "given parameters is empty"
case My_ERROR:
return "example of message for code MY_ERROR"
if e := Get(err); e != nil && e.HasParent() {
for _, parent := range err.GetParent(true) {
fmt.Println("Parent error:", parent)
}
}
// CAREFUL : the default return if code is not found must be en empty string !
return ""
if ContainsString(err, "missing") {
// handle error containing substring
}
if e := Get(err); e != nil {
fmt.Println("Error code:", e.Code())
fmt.Println("Error message:", e.StringError())
fmt.Println("Error trace:", e.GetTrace())
for _, oneErr := range e.GetErrorSlice() {
fmt.Println("Error code in slice:", oneErr.Code())
fmt.Println("Error message:", oneErr.StringError())
fmt.Println("Error trace:", oneErr.GetTrace())
}
}
```
In go source file, we can implement the call to package `golib/errors` like this :
```go
// will test if err is not nil
// if is nil, will return nil, otherwise will return a Error interface from the MY_ERROR code with a parent err
_ = MY_ERROR.Iferror(err)
```
---
```go
// return an Error interface from the MY_ERROR code and if err is not nil, add a parent err
_ = MY_ERROR.ErrorParent(err)
```
```go
// error can be cascaded add like this
_ = MY_ERROR.IfError(EMPTY_PARAMS.IfError(err))
return MY_ERROR.Error(EMPTY_PARAMS.ErrorParent(err))
```
**Notes:**
- Always check for code collisions before registering new error codes.
- Use a unique offset for your package to avoid overlapping with other packages.
- Register a message function for each error code group to provide meaningful error messages.

2
errors/interface.go
View File

@@ -40,6 +40,8 @@ type FuncMap func(e error) bool
type ReturnError func(code int, msg string, file string, line int)
type Error interface {
error
//IsCode check if the given error code is matching with the current Error
IsCode(code CodeError) bool
//HasCode check if current error or parent has the given error code

325
file/README.md Normal file
View File

@@ -0,0 +1,325 @@
# `file` Package
The `file` package provides a set of utilities and abstractions for file management, including bandwidth throttling, permission handling, and progress tracking.
<br/>It is organized into several subpackages, each focusing on a specific aspect of file operations.
## Subpackages
- **[bandwidth subpackage](#bandwidth-subpackage)**
<br/>Bandwidth throttling and rate limiting for file operations.
<br/>This subpackage is ideal for applications that need to control file transfer rates, such as backup tools,
<br/>file servers, or any scenario where bandwidth usage must be limited.
<br />It integrates seamlessly with the progress tracking system.
<br />It is designed for applications that require precise control over data transfer rates.
<br /><br />
- **[perm subpackage](#perm-subpackage)**
<br/>File permission management and utilities.
<br/>This subpackage is ideal for applications needing robust, portable, and easily configurable file permission management.
<br />It supports parsing, formatting, and encoding/decoding of file permissions across various formats.
<br />It is designed for applications that require consistent permission handling across different platforms and configurations.
<br /><br />
- **[progress subpackage](#progress-subpackage)**
<br/>Progress tracking and reporting for file operations.
<br/>This subpackage is ideal for applications needing detailed file operation tracking, custom progress reporting, or advanced file management.
<br />It provides a unified interface for file I/O with integrated progress tracking, buffer management, and event hooks.
<br />It is designed for applications that require real-time feedback on file operations, such as file transfer applications, backup tools,
<br />or any application that needs to monitor file I/O progress.
<br /><br />
---
## `bandwidth` Subpackage
The `bandwidth` subpackage provides utilities for bandwidth throttling and rate limiting during file operations. It is designed to integrate seamlessly with the progress tracking system, allowing you to control the data transfer rate (in bytes per second) for file reads and writes.
### Overview
- Allows you to set a bandwidth limit for file operations.
- Integrates with the progress tracking system to monitor and control data flow.
- Provides hooks for increment and reset events to enforce bandwidth constraints.
- Thread-safe implementation using atomic operations.
---
### Main Types & Interfaces
#### `BandWidth` Interface
Defines the main methods for bandwidth control:
```go
type BandWidth interface {
RegisterIncrement(fpg libfpg.Progress, fi libfpg.FctIncrement)
RegisterReset(fpg libfpg.Progress, fr libfpg.FctReset)
}
```
- `RegisterIncrement`: Registers a callback to be called on each progress increment (e.g., bytes read/written). The bandwidth limiter is applied here.
- `RegisterReset`: Registers a callback to be called when the progress is reset.
#### Constructor
Create a new bandwidth limiter by specifying the maximum bytes per second:
```go
import (
"github.com/nabbar/golib/file/bandwidth"
"github.com/nabbar/golib/size"
)
bw := bandwidth.New(size.Size(1024 * 1024)) // 1 MB/s limit
```
---
### Usage Example
```go
import (
"github.com/nabbar/golib/file/bandwidth"
"github.com/nabbar/golib/file/progress"
"github.com/nabbar/golib/size"
)
bw := bandwidth.New(size.Size(512 * 1024)) // 512 KB/s
f, err := progress.Open("example.txt")
if err != nil {
// handle error
}
defer f.Close()
bw.RegisterIncrement(f, nil)
bw.RegisterReset(f, nil)
// Now, all read/write operations on f will be bandwidth-limited
```
---
### Error Handling
- The subpackage does not return errors directly; it relies on the progress and file operation layers for error reporting.
- Always check errors from file and progress operations.
---
### Notes
- The bandwidth limiter works by measuring the time since the last operation and introducing a sleep if the data rate exceeds the configured limit.
- The implementation is thread-safe and can be used concurrently.
- Designed to be used in conjunction with the `progress` subpackage for seamless integration.
---
## `perm` Subpackage
The `perm` subpackage provides utilities for handling file permissions in a portable and user-friendly way. It offers parsing, formatting, conversion, and encoding/decoding of file permissions, making it easy to work with permissions across different formats and configuration systems.
### Overview
- Defines a `Perm` type based on `os.FileMode` for representing file permissions.
- Supports parsing from strings, integers, and byte slices.
- Provides conversion methods to various integer types and string representations.
- Implements encoding and decoding for JSON, YAML, TOML, CBOR, and text.
- Integrates with Viper for configuration loading via a decoder hook.
---
### Main Types & Functions
#### `Perm` Type
A type alias for `os.FileMode`:
```go
type Perm os.FileMode
```
#### Parsing Functions
- `Parse(s string) (Perm, error)`: Parse a permission string (octal, e.g., "0644").
- `ParseInt(i int) (Perm, error)`: Parse from an integer (interpreted as octal).
- `ParseInt64(i int64) (Perm, error)`: Parse from an int64 (octal).
- `ParseByte(p []byte) (Perm, error)`: Parse from a byte slice.
#### Conversion Methods
- `FileMode() os.FileMode`: Convert to `os.FileMode`.
- `String() string`: Return octal string representation (e.g., "0644").
- `Int64() int64`, `Int32() int32`, `Int() int`: Convert to signed integers.
- `Uint64() uint64`, `Uint32() uint32`, `Uint() uint`: Convert to unsigned integers.
#### Encoding/Decoding
Implements marshaling and unmarshaling for:
- JSON (`MarshalJSON`, `UnmarshalJSON`)
- YAML (`MarshalYAML`, `UnmarshalYAML`)
- TOML (`MarshalTOML`, `UnmarshalTOML`)
- CBOR (`MarshalCBOR`, `UnmarshalCBOR`)
- Text (`MarshalText`, `UnmarshalText`)
This allows seamless integration with configuration files and serialization formats.
#### Viper Integration
- `ViperDecoderHook()`: Returns a Viper decode hook for automatic permission parsing from configuration files.
---
### Usage Example
```go
import (
"github.com/nabbar/golib/file/perm"
"os"
)
p, err := perm.Parse("0755")
if err != nil {
// handle error
}
file, err := os.OpenFile("example.txt", os.O_CREATE, p.FileMode())
if err != nil {
// handle error
}
defer file.Close()
```
#### With Viper
```go
import (
"github.com/nabbar/golib/file/perm"
"github.com/spf13/viper"
)
v := viper.New()
v.Set("file_perm", "0644")
type Config struct {
FilePerm perm.Perm `mapstructure:"file_perm"`
}
var cfg Config
v.Unmarshal(&cfg, viper.DecodeHook(perm.ViperDecoderHook()))
```
---
### Error Handling
- Parsing functions return standard Go `error` values.
- Invalid or out-of-range permissions result in descriptive errors.
- Always check errors when parsing or decoding permissions.
---
### Notes
- Permission strings must be in octal format (e.g., "0644", "0755").
- Handles overflow and invalid values gracefully.
- Designed for use with Go 1.18+ and compatible with common configuration systems.
---
## `progress` Subpackage
The `progress` subpackage provides advanced file I/O utilities with integrated progress tracking, buffer management, and event hooks. It wraps standard file operations and exposes interfaces for monitoring and controlling file read/write progress, making it ideal for applications that need to report or limit file operation progress.
### Overview
- Wraps standard file I/O with progress tracking.
- Supports registering callbacks for increment, reset, and EOF events.
- Allows buffer size customization for optimized I/O.
- Provides file management utilities (open, create, temp, unique, truncate, sync, stat, etc.).
- Implements the full set of `io.Reader`, `io.Writer`, `io.Seeker`, and related interfaces.
---
### Main Types & Interfaces
#### Interfaces
- **Progress**: Main interface combining file operations and progress tracking.
- **File**: File management operations (stat, truncate, sync, etc.).
- **GenericIO**: Embeds all standard Go I/O interfaces.
#### Key Functions
- `New(name string, flags int, perm os.FileMode) (Progress, error)`: Open or create a file with progress tracking.
- `Open(name string) (Progress, error)`: Open an existing file.
- `Create(name string) (Progress, error)`: Create a new file.
- `Temp(pattern string) (Progress, error)`: Create a temporary file.
- `Unique(basePath, pattern string) (Progress, error)`: Create a unique temporary file.
#### Progress Event Registration
- `RegisterFctIncrement(fct FctIncrement)`: Register a callback for each progress increment (e.g., bytes read/written).
- `RegisterFctReset(fct FctReset)`: Register a callback for progress reset events.
- `RegisterFctEOF(fct FctEOF)`: Register a callback for EOF events.
- `SetBufferSize(size int32)`: Set the buffer size for I/O operations.
- `SetRegisterProgress(f Progress)`: Copy registered callbacks to another Progress instance.
#### File Operations
- `Path() string`: Get the file path.
- `Stat() (os.FileInfo, error)`: Get file info.
- `SizeBOF() (int64, error)`: Get current offset.
- `SizeEOF() (int64, error)`: Get size from current offset to EOF.
- `Truncate(size int64) error`: Truncate the file.
- `Sync() error`: Sync file to disk.
- `Close() error`: Close the file.
- `CloseDelete() error`: Close and delete the file.
#### I/O Operations
Implements all standard I/O methods:
- `Read`, `ReadAt`, `ReadFrom`, `Write`, `WriteAt`, `WriteTo`, `WriteString`, `ReadByte`, `WriteByte`, `Seek`, etc.
---
### Usage Example
```go
import (
"github.com/nabbar/golib/file/progress"
"os"
)
f, err := progress.New("example.txt", os.O_RDWR|os.O_CREATE, 0644)
if err != nil {
// handle error
}
defer f.Close()
f.RegisterFctIncrement(func(size int64) {
// Called on each read/write increment
})
f.RegisterFctEOF(func() {
// Called on EOF
})
buf := make([]byte, 1024)
n, err := f.Read(buf)
// ... use n, err
```
---
### Error Handling
- All errors are wrapped with custom error codes for precise diagnostics (e.g., `ErrorNilPointer`, `ErrorIOFileStat`, etc.).
- Always check returned errors from file and I/O operations.
- Error messages are descriptive and help identify the source of the problem.
---
### Notes
- The buffer size can be tuned for performance using `SetBufferSize`.
- Progress callbacks allow integration with UI, logging, or bandwidth throttling.
- Temporary and unique file creation is supported for safe file operations.
- Implements all standard file and I/O interfaces for drop-in replacement.

123
ftpclient/README.md Normal file
View File

@@ -0,0 +1,123 @@
# `ftpclient` Package
The `ftpclient` package provides a high-level, thread-safe FTP client abstraction for Go, built on top of [jlaffaye/ftp](https://github.com/jlaffaye/ftp). It simplifies FTP operations, connection management, error handling, and configuration, including TLS support and advanced options.
## Features
- Thread-safe FTP client with connection pooling
- Rich configuration struct with validation (hostname, login, password, timeouts, TLS, etc.)
- Support for explicit/implicit TLS, time zones, and FTP protocol options (UTF8, EPSV, MLSD, MDTM)
- Full set of FTP commands: file/directory listing, upload, download, rename, delete, recursive removal, and more
- Custom error codes for precise diagnostics
- Context and TLS configuration registration
- Automatic connection checking and recovery
---
## Main Types & Functions
### `Config` Struct
Defines all connection and protocol options:
- `Hostname`: FTP server address (required, RFC1123)
- `Login` / `Password`: Credentials for authentication
- `ConnTimeout`: Timeout for all operations
- `TimeZone`: Force a specific time zone for the connection
- `DisableUTF8`, `DisableEPSV`, `DisableMLSD`, `EnableMDTM`: Protocol feature toggles
- `ForceTLS`: Require TLS for the connection
- `TLS`: TLS configuration (see `github.com/nabbar/golib/certificates`)
- Methods to register context and default TLS providers
#### Example
```go
import (
"github.com/nabbar/golib/ftpclient"
"time"
)
cfg := &ftpclient.Config{
Hostname: "ftp.example.com:21",
Login: "user",
Password: "pass",
ConnTimeout: 10 * time.Second,
ForceTLS: true,
// ... other options
}
if err := cfg.Validate(); err != nil {
// handle config error
}
```
---
### `FTPClient` Interface
Main interface for FTP operations:
- `Connect() error`: Establish connection
- `Check() error`: Check and validate connection (NOOP)
- `Close()`: Close connection (QUIT)
- `NameList(path string) ([]string, error)`: List file names (NLST)
- `List(path string) ([]*ftp.Entry, error)`: List directory entries (LIST/MLSD)
- `ChangeDir(path string) error`: Change working directory (CWD)
- `CurrentDir() (string, error)`: Get current directory (PWD)
- `FileSize(path string) (int64, error)`: Get file size (SIZE)
- `GetTime(path string) (time.Time, error)`: Get file modification time (MDTM)
- `SetTime(path string, t time.Time) error`: Set file modification time (MFMT/MDTM)
- `Retr(path string) (*ftp.Response, error)`: Download file (RETR)
- `RetrFrom(path string, offset uint64) (*ftp.Response, error)`: Download from offset
- `Stor(path string, r io.Reader) error`: Upload file (STOR)
- `StorFrom(path string, r io.Reader, offset uint64) error`: Upload from offset
- `Append(path string, r io.Reader) error`: Append to file (APPE)
- `Rename(from, to string) error`: Rename file (RNFR/RNTO)
- `Delete(path string) error`: Delete file (DELE)
- `RemoveDirRecur(path string) error`: Recursively delete directory
- `MakeDir(path string) error`: Create directory (MKD)
- `RemoveDir(path string) error`: Remove directory (RMD)
- `Walk(root string) (*ftp.Walker, error)`: Walk directory tree
#### Example
```go
cli, err := ftpclient.New(cfg)
if err != nil {
// handle error
}
defer cli.Close()
files, err := cli.List("/")
if err != nil {
// handle error
}
for _, entry := range files {
// process entry
}
```
---
### Error Handling
All errors are wrapped with custom codes (see `errors.go`):
- `ErrorParamsEmpty`
- `ErrorValidatorError`
- `ErrorEndpointParser`
- `ErrorNotInitialized`
- `ErrorFTPConnection`
- `ErrorFTPConnectionCheck`
- `ErrorFTPLogin`
- `ErrorFTPCommand`
Use `err.Error()` for user-friendly messages and check error codes for diagnostics.
---
### Notes
- The client is thread-safe and manages connection state automatically.
- TLS and context can be customized via registration methods.
- All FTP commands are wrapped and errors are contextualized.
- Designed for Go 1.18+.

199
httpcli/README.md Normal file
View File

@@ -0,0 +1,199 @@
## `httpcli` Package
The `httpcli` package provides helpers and abstractions for creating, configuring, and managing HTTP clients in Go. It is designed to simplify HTTP client instantiation, support advanced options (TLS, proxy, timeouts, DNS mapping), and enable easy integration with custom DNS resolvers for testing or advanced routing.
### Features
- Easy creation of HTTP clients with sensible defaults
- Support for custom DNS mapping (mock/fake DNS) via the `dns-mapper` subpackage
- TLS configuration and proxy support
- Configurable timeouts and connection options
- Thread-safe management of default DNS mappers and clients
- Error handling with custom error codes
---
### Main Types & Functions
#### Getting a Default HTTP Client
```go
import "github.com/nabbar/golib/httpcli"
client := httpcli.GetClient()
// Use client for HTTP requests
```
#### Custom DNS Mapper
You can set a custom DNS mapper to control how hostnames are resolved (useful for testing or routing):
```go
import (
"github.com/nabbar/golib/httpcli"
htcdns "github.com/nabbar/golib/httpcli/dns-mapper"
)
dns := htcdns.New(context.Background(), &htcdns.Config{/* ... */}, nil, nil)
httpcli.SetDefaultDNSMapper(dns)
client := httpcli.GetClient()
```
For more details on the `dns-mapper`, see the [dns-mapper subpackage](#subpackage-dns-mapper).
#### Options Structure
The `Options` struct allows you to configure timeouts, keep-alive, compression, HTTP/2, TLS, forced IP, and proxy settings:
```go
import "github.com/nabbar/golib/httpcli"
opt := httpcli.Options{
Timeout: 10 * time.Second,
DisableKeepAlive: false,
TLS: httpcli.OptionTLS{Enable: true, Config: /* ... */},
// ... other options
}
```
#### Validation
Validate your options before using them:
```go
err := opt.Validate()
if err != nil {
// handle validation error
}
```
#### Error Handling
All errors are returned as `liberr.Error` with specific codes (e.g., `ErrorParamEmpty`, `ErrorValidatorError`). Always check errors after each operation.
---
### Example Usage
```go
client := httpcli.GetClient()
resp, err := client.Get("https://example.com")
if err != nil {
// handle error
}
defer resp.Body.Close()
// process response
```
---
### Subpackage: `dns-mapper`
The `dns-mapper` subpackage provides a flexible and thread-safe DNS mapping and mock resolver for Go HTTP clients.
<br />It allows you to map specific hostnames (with or without ports, including wildcards) to custom destinations, making it ideal for testing, local development, or advanced routing scenarios.
---
#### Features
- Map hostnames (optionally with port and wildcards) to custom IP:port destinations.
- Transparent integration with HTTP clients and transports.
- Dynamic add, remove, and lookup of DNS mappings at runtime.
- Caching for efficient repeated lookups.
- Customizable cleaning interval for idle connections.
- Full configuration struct for transport and TLS options.
- Thread-safe and context-aware.
---
#### Main Types & Functions
##### `Config` Struct
Defines DNS mapping and transport options:
- `DNSMapper`: `map[string]string` — source to destination mappings (e.g., `"test.example.com:8080": "127.0.0.1:8081"`).
- `TimerClean`: cleaning interval for idle connections.
- `Transport`: HTTP transport configuration (timeouts, proxy, TLS, etc.).
- `TLSConfig`: optional TLS configuration.
##### `DNSMapper` Interface
Main interface for DNS mapping and HTTP client integration:
- `Add(from, to string)`: Add a mapping.
- `Get(from string) string`: Get the mapped destination for a source.
- `Del(from string)`: Remove a mapping.
- `Len() int`: Number of mappings.
- `Walk(func(from, to string) bool)`: Iterate over mappings.
- `Search(endpoint string) (string, error)`: Find the mapped destination for an endpoint.
- `SearchWithCache(endpoint string) (string, error)`: Same as `Search`, with caching.
- `DialContext(ctx, network, address string) (net.Conn, error)`: Custom dialer for HTTP transport.
- `Transport(cfg TransportConfig) *http.Transport`: Create a custom HTTP transport.
- `Client(cfg TransportConfig) *http.Client`: Create an HTTP client using the DNS mapper.
- `DefaultTransport() *http.Transport`: Get the default transport.
- `DefaultClient() *http.Client`: Get the default client.
- `Close() error`: Clean up resources.
---
#### Example Usage
```go
import (
"context"
"github.com/nabbar/golib/httpcli/dns-mapper"
"time"
)
cfg := dns_mapper.Config{
DNSMapper: map[string]string{
"test.example.com:8080": "127.0.0.1:8081",
"*.dev.local:80": "127.0.0.2:8080",
},
TimerClean: dns_mapper.ParseDuration(5 * time.Minute),
// Transport and TLSConfig can be set as needed
}
dns := dns_mapper.New(context.Background(), &cfg, nil, nil)
defer dns.Close()
// Add or remove mappings dynamically
dns.Add("api.local:443", "10.0.0.1:8443")
dns.Del("test.example.com:8080")
// Use with HTTP client
client := dns.DefaultClient()
resp, err := client.Get("http://api.local:443/health")
if err != nil {
// handle error
}
defer resp.Body.Close()
// process response
```
---
#### Wildcard and Port Matching
- Hostnames can include wildcards (e.g., `*.example.com` or `*.*.dev.local`).
- Ports can be specified or wildcarded (e.g., `*:8080`).
- The mapping logic matches the most specific rule.
---
#### Error Handling
All errors are wrapped with custom codes for diagnostics. Use `err.Error()` for user-friendly messages.
---
#### Notes
- The DNS mapper is thread-safe and suitable for concurrent use.
- Integrates seamlessly with Go's `http.Transport` and `http.Client`.
- Designed for Go 1.18+.
---
For more details, refer to the GoDoc or the source code in the `dns-mapper` package.

291
httpserver/README.md Normal file
View File

@@ -0,0 +1,291 @@
# `httpserver` Package
## `httpserver/pool` Subpackage
The `httpserver/pool` package provides a high-level abstraction for managing a pool of [`httpserver` Package](#package-httpserver).
<br />It allows you to configure, start, stop, and monitor multiple HTTP server instances as a unified group, with advanced filtering, merging, and handler management capabilities.
### Features
- Manage multiple HTTP server instances as a pool
- Add, remove, and retrieve servers by bind address
- Start, stop, and restart all servers in the pool with aggregated error handling
- Filter and list servers by name, bind address, or expose address (with pattern or regex)
- Merge pools and clone pool state
- Register global handler functions for all servers
- Monitor all servers and retrieve monitoring data
- Thread-safe operations
---
### Main Types & Interfaces
#### `Pool` Interface
The main interface for managing a pool of HTTP servers. Key methods include:
- `Start(ctx context.Context) error`: Start all servers in the pool.
- `Stop(ctx context.Context) error`: Stop all servers in the pool.
- `Restart(ctx context.Context) error`: Restart all servers in the pool.
- `IsRunning() bool`: Check if any server in the pool is running.
- `Uptime() time.Duration`: Get the maximum uptime among all servers.
- `Handler(fct FuncHandler)`: Register a global handler function for all servers.
- `Monitor(vrs Version) ([]Monitor, error)`: Retrieve monitoring data for all servers.
- `Clone(ctx context.Context) Pool`: Clone the pool (optionally with a new context).
- `Merge(p Pool, defLog FuncLog) error`: Merge another pool into this one.
- `Manage` and `Filter` interfaces for advanced management and filtering.
#### `Manage` Interface
- `Walk(fct FuncWalk) bool`: Iterate over all servers.
- `StoreNew(cfg Config, defLog FuncLog) error`: Add a new server from config.
- `Load(bindAddress string) Server`: Retrieve a server by bind address.
- `Delete(bindAddress string)`: Remove a server by bind address.
- `MonitorNames() []string`: List all monitor names.
#### `Filter` Interface
- `Has(bindAddress string) bool`: Check if a server exists.
- `Len() int`: Number of servers in the pool.
- `List(fieldFilter, fieldReturn FieldType, pattern, regex string) []string`: List server fields matching criteria.
- `Filter(field FieldType, pattern, regex string) Pool`: Filter servers by field and pattern/regex.
#### `Config` Type
A slice of server configuration objects. Provides helper methods to:
- Set global handler, TLS, and context functions for all configs
- Validate all configs
- Instantiate a pool from the configs
---
### Example Usage
```go
import (
"github.com/nabbar/golib/httpserver/pool"
"github.com/nabbar/golib/httpserver"
"github.com/nabbar/golib/logger"
"context"
)
cfgs := pool.Config{
/* ... fill with httpserver.Config objects ... */
}
err := cfgs.Validate()
if err != nil {
// handle config validation error
}
p, err := cfgs.Pool(nil, nil, logger.Default)
if err != nil {
// handle pool creation error
}
// Start all servers
if err := p.Start(context.Background()); err != nil {
// handle start error
}
// Stop all servers
if err := p.Stop(context.Background()); err != nil {
// handle stop error
}
```
---
### Error Handling
All errors are wrapped with custom codes for diagnostics, such as:
- `ErrorParamEmpty`
- `ErrorPoolAdd`
- `ErrorPoolValidate`
- `ErrorPoolStart`
- `ErrorPoolStop`
- `ErrorPoolRestart`
- `ErrorPoolMonitor`
Use `err.Error()` for user-friendly messages and check error codes for diagnostics.
---
### Filtering and Listing
You can filter or list servers in the pool by name, bind address, or expose address, using exact match or regular expressions.
```go
// List all bind addresses matching a pattern
binds := p.List(FieldBind, FieldBind, "127.0.0.1:8080", "")
// Filter pool by expose address regex
filtered := p.Filter(FieldExpose, "", "^/api")
```
---
### Monitoring
Retrieve monitoring data for all servers in the pool:
```go
monitors, err := p.Monitor(version)
if err != nil {
// handle monitoring error
}
```
---
### Notes
- The pool is thread-safe and suitable for concurrent use.
- All operations are designed for Go 1.18+.
- Integrates with the `logger`, `context`, and `monitor` packages for advanced features.
---
## Package `httpserver`
The `httpserver` package provides advanced abstractions for configuring, running, and monitoring HTTP servers in Go. It is designed for robust, concurrent, and production-grade server management, supporting TLS, custom handlers, logging, and health monitoring.
---
### Key Features
- **Configurable HTTP/HTTPS servers** with extensive options (timeouts, keep-alive, HTTP/2, TLS, etc.)
- **Handler registration** for flexible routing and API management
- **Integrated logging** and monitoring support
- **Thread-safe** and suitable for concurrent use
- **Custom error codes** for diagnostics and troubleshooting
---
### Main Types
#### `Config`
Represents the configuration for a single HTTP server instance.
Key fields include:
- `Name`: Unique server name (required)
- `Listen`: Bind address (host:port or unix socket, required)
- `Expose`: Public/external address (URL, required)
- `HandlerKey`: Key to associate with a specific handler
- `Disabled`: Enable/disable the server without removing its config
- `Monitor`: Monitoring configuration
- `TLSMandatory`: Require valid TLS configuration to start
- `TLS`: TLS settings (can inherit defaults)
- HTTP/2 and HTTP options: timeouts, max handlers, keep-alive, etc.
- `Logger`: Logger configuration
**Helper methods:**
- `Clone()`: Deep copy of the config
- `RegisterHandlerFunc(f)`: Register a handler function
- `SetDefaultTLS(f)`: Set default TLS provider
- `SetContext(f)`: Set parent context provider
- `Validate()`: Validate config fields and constraints
- `Server(defLog)`: Instantiate a server from the config
---
#### `Server` Interface
Represents a running HTTP server instance.
- `Start(ctx) error`: Start the server
- `Stop(ctx) error`: Stop the server gracefully
- `Restart(ctx) error`: Restart the server
- `IsRunning() bool`: Check if the server is running
- `GetConfig() *Config`: Get the current config
- `SetConfig(cfg, defLog) error`: Update the config
- `Handler(f)`: Register handler function
- `Merge(srv, defLog) error`: Merge another server's config
- `Monitor(version)`: Get monitoring data
- `MonitorName() string`: Get monitor name
- `GetName() string`: Get server name
- `GetBindable() string`: Get bind address
- `GetExpose() string`: Get expose address
- `IsDisable() bool`: Check if server is disabled
- `IsTLS() bool`: Check if TLS is enabled
---
#### Handler Management
Handlers are registered via a function returning a map of handler keys to `http.Handler` instances.
You can associate a server with a specific handler using the `HandlerKey` field.
---
#### Monitoring
Integrated with the monitoring system, each server can expose runtime, build, and health information.
Use `Monitor(version)` to retrieve monitoring data.
---
#### Error Handling
All errors are wrapped with custom codes for diagnostics, such as:
- `ErrorParamEmpty`
- `ErrorHTTP2Configure`
- `ErrorServerValidate`
- `ErrorServerStart`
- `ErrorPortUse`
Use `err.Error()` for user-friendly messages and check error codes for diagnostics.
---
### Example Usage
```go
import (
"github.com/nabbar/golib/httpserver"
"github.com/nabbar/golib/logger"
"context"
)
cfg := httpserver.Config{
Name: "api-server",
Listen: "127.0.0.1:8080",
Expose: "http://api.example.com:8080",
// ... other fields
}
cfg.RegisterHandlerFunc(myHandlerFunc)
cfg.SetDefaultTLS(myTLSProvider)
cfg.SetContext(myContextProvider)
if err := cfg.Validate(); err != nil {
// handle config error
}
srv, err := httpserver.New(cfg, logger.Default)
if err != nil {
// handle server creation error
}
if err := srv.Start(context.Background()); err != nil {
// handle start error
}
// ... later
if err := srv.Stop(context.Background()); err != nil {
// handle stop error
}
```
---
### Notes
- The package is designed for Go 1.18+.
- All operations are thread-safe.
- Integrates with `logger`, `context`, and `monitor` packages for advanced features.
- For advanced management of multiple servers, see the `httpserver/pool` package.

775
ioutils/README.md Normal file
View File

@@ -0,0 +1,775 @@
# `ioutils` Package Documentation
The `ioutils` package provides utility functions and abstractions for I/O operations in Go.
<br />It includes helpers for file and directory management, as well as interfaces and wrappers to simplify and extend standard I/O patterns.
---
## Features
- File and directory existence checks and creation with permissions
- I/O wrapper interface for custom or dynamic I/O implementations
- Extensible design for advanced I/O utilities
---
## Main Types & Functions
### PathCheckCreate
Checks if a file or directory exists at the given path, creates it if necessary, and sets the appropriate permissions.
**Signature:**
```go
func PathCheckCreate(isFile bool, path string, permFile os.FileMode, permDir os.FileMode) error
```
- `isFile`: `true` to check/create a file, `false` for a directory
- `path`: target path
- `permFile`: permissions for files
- `permDir`: permissions for directories
---
## Subpackages
This package includes several subpackages, each providing specific utilities for I/O operations:
- `bufferReadCloser`: Buffered I/O utilities with close support. See [`bufferReadCloser` documentation](#bufferreadcloser-subpackage-documentation) for details.
- `fileDescriptor`: Utilities for working with file descriptors and low-level file operations. See [`fileDescriptor` documentation](#filedescriptor-subpackage-documentation) for details.
- `ioprogress`: Progress tracking and reporting for I/O operations. See [`ioprogress` documentation](#ioprogress-subpackage-documentation) for details
- `iowrapper`: Provides an interface for wrapping I/O operations with custom logic. See [`iowrapper` documentation](#iowrapper-subpackage-documentation) for details.
- `mapCloser`: Manages multiple `io.Closer` instances with mapping and batch close support. See [`mapCloser` documentation](#mapcloser-subpackage-documentation) for details.
- `maxstdio`: Handles system limits and management for maximum open standard I/O descriptors. See [`maxstdio` documentation](#maxstdio-subpackage-documentation) for details.
- `multiplexer`: Multiplexes I/O streams for advanced routing or splitting of data. See [`multiplexer` documentation](#multiplexer-subpackage-documentation) for details.
- `nopwritecloser`: Implements a no-operation `io.WriteCloser` for testing or stubbing. See [`nopwritecloser` documentation](#nopwritecloser-subpackage-documentation) for details.
---
### `bufferReadCloser` Subpackage Documentation
The `bufferReadCloser` subpackage provides buffered I/O utilities that combine standard Go buffer types
<br />with the `io.Closer` interface. It offers convenient wrappers for `bytes.Buffer`, `bufio.Reader`,
<br />`bufio.Writer`, and `bufio.ReadWriter`, allowing for easy resource management and integration with custom close logic.
---
#### Features
- Buffered read, write, and read/write utilities with close support
- Compatible with standard Go I/O interfaces
- Optional custom close function for resource cleanup
- Reset and flush logic integrated with close operations
---
#### Main Types & Constructors
##### Buffer
A buffered read/write/close utility based on `bytes.Buffer`.
**Implements:**
- `io.Reader`, `io.ReaderFrom`, `io.ByteReader`, `io.RuneReader`
- `io.Writer`, `io.WriterTo`, `io.ByteWriter`, `io.StringWriter`
- `io.Closer`
**Constructor:**
```go
func NewBuffer(b *bytes.Buffer, fct FuncClose) Buffer
```
- `b`: underlying buffer
- `fct`: optional close function
---
##### Reader
A buffered reader with close support, based on `bufio.Reader`.
**Implements:**
- `io.Reader`, `io.WriterTo`, `io.Closer`
**Constructor:**
```go
func NewReader(b *bufio.Reader, fct FuncClose) Reader
```
---
##### Writer
A buffered writer with close support, based on `bufio.Writer`.
**Implements:**
- `io.Writer`, `io.StringWriter`, `io.ReaderFrom`, `io.Closer`
**Constructor:**
```go
func NewWriter(b *bufio.Writer, fct FuncClose) Writer
```
---
##### ReadWriter
A buffered read/write utility with close support, based on `bufio.ReadWriter`.
**Implements:**
- All methods of `Reader` and `Writer`
**Constructor:**
```go
func NewReadWriter(b *bufio.ReadWriter, fct FuncClose) ReadWriter
```
---
#### Example Usage
```go
import (
"bytes"
"github.com/nabbar/golib/ioutils/bufferReadCloser"
)
buf := bytes.NewBuffer(nil)
brc := bufferReadCloser.NewBuffer(buf, nil)
_, _ = brc.Write([]byte("hello"))
_ = brc.Close() // resets buffer and calls optional close function
```
---
#### Notes
- The `Close()` method resets or flushes the buffer and then calls the optional close function if provided.
- These types are useful for managing in-memory buffers with explicit resource cleanup, especially in complex I/O pipelines.
- All types are compatible with standard Go I/O interfaces for seamless integration.
---
### `fileDescriptor` Subpackage Documentation
The `fileDescriptor` subpackage provides utilities to query and manage the system's file descriptor limits for the current process.
<br />It is useful for applications that need to handle many open files or network connections and want to ensure the process is configured with appropriate resource limits.
---
#### Features
- Query the current and maximum file descriptor limits for the process.
- Increase the current file descriptor limit up to the system's maximum (platform-dependent).
- Cross-platform support (Linux/Unix and Windows).
---
#### Main Function
##### SystemFileDescriptor
Returns the current and maximum file descriptor limits, or sets a new limit if requested.
**Signature:**
```go
func SystemFileDescriptor(newValue int) (current int, max int, err error)
```
- `newValue`:
- If `0`, only queries the current and maximum limits.
- If greater than the current limit, attempts to increase the limit (up to the system maximum).
- Returns:
- `current`: the current file descriptor limit for the process.
- `max`: the maximum file descriptor limit allowed by the system.
- `err`: error if the operation fails.
---
#### Example Usage
```go
import "github.com/nabbar/golib/ioutils/fileDescriptor"
cur, max, err := fileDescriptor.SystemFileDescriptor(0)
if err != nil {
// handle error
}
// Try to increase the limit to 4096
cur, max, err = fileDescriptor.SystemFileDescriptor(4096)
if err != nil {
// handle error
}
```
---
#### Platform Notes
- **Linux/Unix:** Uses `syscall.Getrlimit` and `syscall.Setrlimit` to manage `RLIMIT_NOFILE`.
- **Windows:** Uses system calls to manage the maximum number of open files (`maxstdio`), with hard limits defined by the OS.
---
#### Use Cases
- Ensuring your application can open enough files or sockets for high concurrency.
- Dynamically adjusting resource limits at startup based on workload requirements.
---
#### Notes
- Changing file descriptor limits may require appropriate system permissions.
- Always check the returned `err` to ensure the operation succeeded.
- Designed for Go 1.18+ and cross-platform compatibility.
---
### `ioprogress` Subpackage Documentation
The `ioprogress` subpackage provides utilities for tracking and reporting progress during I/O operations.
<br />It wraps standard `io.ReadCloser` and `io.WriteCloser` interfaces, allowing developers to monitor
<br />the amount of data read or written, and to register custom callbacks for progress, reset, and end-of-file events.
---
#### Features
- Progress tracking for reading and writing operations
- Register custom callbacks for increment, reset, and EOF events
- Thread-safe progress counters using atomic operations
- Simple integration with existing I/O streams
---
#### Main Interfaces
##### Progress
Defines methods to register callbacks and reset progress.
```go
type Progress interface {
RegisterFctIncrement(fct FctIncrement)
RegisterFctReset(fct FctReset)
RegisterFctEOF(fct FctEOF)
Reset(max int64)
}
```
##### Reader
Combines `io.ReadCloser` and `Progress` for read operations with progress tracking.
```go
type Reader interface {
io.ReadCloser
Progress
}
```
##### Writer
Combines `io.WriteCloser` and `Progress` for write operations with progress tracking.
```go
type Writer interface {
io.WriteCloser
Progress
}
```
---
#### Constructors
##### NewReadCloser
Wraps an `io.ReadCloser` to provide progress tracking.
```go
func NewReadCloser(r io.ReadCloser) Reader
```
##### NewWriteCloser
Wraps an `io.WriteCloser` to provide progress tracking.
```go
func NewWriteCloser(w io.WriteCloser) Writer
```
---
#### Example Usage
```go
import (
"os"
"github.com/nabbar/golib/ioutils/ioprogress"
)
file, _ := os.Open("data.txt")
reader := ioprogress.NewReadCloser(file)
reader.RegisterFctIncrement(func(size int64) {
// Called after each read with the number of bytes read
})
reader.RegisterFctEOF(func() {
// Called when EOF is reached
})
buf := make([]byte, 1024)
for {
n, err := reader.Read(buf)
if err != nil {
break
}
// process buf[:n]
}
_ = reader.Close()
```
---
#### Notes
- Callbacks for increment, reset, and EOF can be registered at any time.
- The `Reset` method allows resetting the progress counter and optionally triggering a reset callback.
- Designed for Go 1.18+ and thread-safe usage.
- Useful for monitoring file transfers, network streams, or any I/O operation where progress feedback is needed.
---
### `iowrapper` Subpackage Documentation
The `iowrapper` subpackage provides a flexible interface to wrap and extend standard Go I/O operations (`io.Reader`, `io.Writer`, `io.Seeker`, `io.Closer`).
<br />It allows developers to inject custom logic for reading, writing, seeking, and closing, making it easy to adapt or mock I/O behaviors.
---
#### Features
- Wraps any I/O-compatible object with custom read, write, seek, and close functions
- Implements standard Go I/O interfaces for seamless integration
- Dynamic assignment of custom handlers at runtime
- Useful for testing, instrumentation, or adapting legacy I/O
---
#### Main Types & Functions
##### IOWrapper Interface
Defines a wrapper for I/O operations with methods to set custom logic.
```go
type IOWrapper interface {
io.Reader
io.Writer
io.Seeker
io.Closer
SetRead(read FuncRead)
SetWrite(write FuncWrite)
SetSeek(seek FuncSeek)
SetClose(close FuncClose)
}
```
##### Function Types
- `FuncRead`: `func(p []byte) []byte`
- `FuncWrite`: `func(p []byte) []byte`
- `FuncSeek`: `func(offset int64, whence int) (int64, error)`
- `FuncClose`: `func() error`
---
##### Constructor
Creates a new IOWrapper for any I/O-compatible object.
```go
func New(in any) IOWrapper
```
- `in`: any object implementing one or more standard I/O interfaces
---
#### Example Usage
```go
import (
"os"
"github.com/nabbar/golib/ioutils/iowrapper"
)
file, _ := os.Open("data.txt")
w := iowrapper.New(file)
// Set a custom read function (e.g., for logging or transformation)
w.SetRead(func(p []byte) []byte {
// custom logic here
return p
})
buf := make([]byte, 128)
_, _ = w.Read(buf)
_ = w.Close()
```
---
#### Notes
- If no custom function is set, the wrapper delegates to the underlying object's standard methods.
- Setting a function to `nil` restores the default behavior.
- Useful for testing, instrumentation, or adapting I/O flows without modifying the original implementation.
---
### `mapCloser` Subpackage Documentation
The `mapCloser` subpackage provides a utility to manage multiple `io.Closer` instances as a group, allowing for batch addition, retrieval, cloning, and closing of resources.
<br />It is designed for robust resource management in concurrent or context-driven applications.
---
##### Features
- Add and manage multiple `io.Closer` objects
- Retrieve all managed closers
- Clean and reset the internal state
- Clone the current set of closers
- Batch close all resources, collecting errors if any
- Context-aware: automatically closes resources when the context is cancelled
- Thread-safe operations
---
##### Main Interface
```go
type Closer interface {
Add(clo ...io.Closer)
Get() []io.Closer
Len() int
Clean()
Clone() Closer
Close() error
}
```
---
##### Constructor
Creates a new `Closer` instance bound to a context.
```go
func New(ctx context.Context) Closer
```
- `ctx`: The context to monitor for cancellation. When cancelled, all managed closers are closed automatically.
---
##### Example Usage
```go
import (
"context"
"os"
"github.com/nabbar/golib/ioutils/mapCloser"
)
ctx := context.Background()
mc := mapCloser.New(ctx)
file1, _ := os.Open("file1.txt")
file2, _ := os.Open("file2.txt")
mc.Add(file1, file2)
// Retrieve all closers
closers := mc.Get()
// Close all resources
err := mc.Close()
```
---
##### Notes
- The `Close()` method attempts to close all managed resources and returns a combined error if any close operations fail.
- The `Clone()` method creates a copy of the current `Closer` with the same set of managed resources.
- The internal state is thread-safe and suitable for concurrent use.
- Automatically handles context cancellation for safe resource cleanup.
---
##### Use Cases
- Managing multiple files, network connections, or other closable resources in a batch.
- Ensuring all resources are properly closed on application shutdown or context cancellation.
- Simplifying resource management in complex workflows.
---
### `maxstdio` Subpackage Documentation
The `maxstdio` subpackage provides utilities to get and set the maximum number of standard I/O file descriptors (stdio) that a process can open on Windows systems.
<br />It uses cgo to call the underlying C runtime functions for managing this limit.
---
##### Features
- Query the current maximum stdio limit for the process.
- Set a new maximum stdio limit (up to the system hard limit).
- Direct integration with the Windows C runtime via cgo.
---
##### Main Functions
###### GetMaxStdio
Returns the current maximum number of stdio file descriptors allowed for the process.
```go
func GetMaxStdio() int
```
- Returns the current stdio limit as an integer.
---
###### SetMaxStdio
Sets a new maximum number of stdio file descriptors for the process.
```go
func SetMaxStdio(newMax int) int
```
- `newMax`: The desired new maximum value.
- Returns the updated stdio limit as an integer.
---
##### Example Usage
```go
import "github.com/nabbar/golib/ioutils/maxstdio"
cur := maxstdio.GetMaxStdio()
newLimit := 2048
updated := maxstdio.SetMaxStdio(newLimit)
```
---
##### Notes
- These functions are only available on Windows with cgo enabled.
- The actual hard limit is defined by the Windows OS and may not be exceeded.
- Useful for applications that need to handle many open files or sockets simultaneously.
---
##### Use Cases
- Increasing the stdio limit for high-concurrency servers or applications.
- Querying the current stdio limit for diagnostics or configuration validation.
- Ensuring resource limits are sufficient for the application's workload.
---
### `multiplexer` Subpackage Documentation
The `multiplexer` subpackage provides a generic and thread-safe way to multiplex and demultiplex I/O streams,
<br />allowing you to route messages to different logical streams identified by a comparable key.
<br />It is useful for advanced I/O routing, such as handling multiple output streams over a single connection.
---
#### Features
- Multiplexes multiple logical streams over a single I/O channel
- Supports any comparable type as a stream key (e.g., `int`, `string`)
- Thread-safe management of stream handlers
- Easy integration with standard Go `io.Reader` and `io.Writer` interfaces
- Uses CBOR encoding for efficient message framing
---
#### Main Types & Interfaces
##### MixStdOutErr
A generic interface for multiplexed I/O operations.
```go
type MixStdOutErr[T comparable] interface {
io.Reader
Writer(key T) io.Writer
Add(key T, fct FuncWrite)
}
```
- `Writer(key T) io.Writer`: Returns a writer for the given stream key.
- `Add(key T, fct FuncWrite)`: Registers a write handler for a specific stream key.
##### FuncWrite
Function signature for custom write handlers.
```go
type FuncWrite func(p []byte) (n int, err error)
```
##### Message
Represents a multiplexed message with a stream key and payload.
```go
type Message[T comparable] struct {
Stream T
Message []byte
}
```
---
#### Constructor
##### New
Creates a new multiplexer instance.
```go
func New[T comparable](r io.Reader, w io.Writer) MixStdOutErr[T]
```
- `r`: The underlying reader (for demultiplexing incoming messages)
- `w`: The underlying writer (for multiplexing outgoing messages)
---
#### Example Usage
```go
import (
"os"
"github.com/nabbar/golib/ioutils/multiplexer"
)
mux := multiplexer.New[string](os.Stdin, os.Stdout)
// Register a handler for a stream key
mux.Add("stdout", func(p []byte) (int, error) {
// handle output for "stdout"
return len(p), nil
})
// Write to a specific stream
writer := mux.Writer("stdout")
_, _ = writer.Write([]byte("Hello, multiplexed world!"))
// Read and dispatch messages
buf := make([]byte, 1024)
_, err := mux.Read(buf)
```
---
#### Notes
- Each message is encoded using CBOR, containing both the stream key and the message payload.
- The `Add` method allows you to register custom handlers for each logical stream.
- The `Writer` method provides a standard `io.Writer` for sending data to a specific stream.
- Reading from the multiplexer will decode messages and dispatch them to the appropriate handler based on the stream key.
- Suitable for scenarios where you need to route or split data between multiple logical channels over a single physical connection.
---
#### Use Cases
- Multiplexing stdout and stderr over a single network connection
- Routing logs or messages to different consumers based on type or channel
- Building advanced I/O pipelines with dynamic stream management
---
### `nopwritecloser` Subpackage Documentation
The `nopwritecloser` subpackage provides a simple utility that wraps any `io.Writer` to implement the `io.WriteCloser` interface, where the `Close()` method is a no-op.
<br /> This is useful for cases where an `io.WriteCloser` is required but no actual resource needs to be closed, such as in testing or when working with in-memory buffers.
This subpackage is similar to the standard `io.NopCloser`, but specifically designed to wrap `io.Writer` types while providing a no-operation `Close()` method.
---
##### Features
- Wraps any `io.Writer` to provide a no-operation `Close()` method
- Fully compatible with the standard Go `io.WriteCloser` interface
- Useful for testing, stubbing, or adapting APIs that require a closer
---
##### Main Function
###### New
Creates a new `io.WriteCloser` from any `io.Writer`. The `Close()` method does nothing and always returns `nil`.
```go
func New(w io.Writer) io.WriteCloser
```
- `w`: The underlying writer to wrap
---
##### Example Usage
```go
import (
"bytes"
"github.com/nabbar/golib/ioutils/nopwritecloser"
)
buf := &bytes.Buffer{}
wc := nopwritecloser.New(buf)
_, _ = wc.Write([]byte("example"))
_ = wc.Close() // does nothing, always returns nil
```
---
##### Notes
- The wrapped writer is not closed or affected by the `Close()` call.
- This utility is ideal for adapting APIs that expect an `io.WriteCloser` but where closing is unnecessary or undesired.
- Designed for Go 1.18+ and compatible with all standard `io.Writer` implementations.
---
## Notes
- All utilities are designed for Go 1.18+.
- Thread-safe where applicable.
- Integrates with standard Go `io` interfaces for maximum compatibility.
---
For more details, refer to the GoDoc or the source code in the `ioutils` package and its subpackages.

52
ioutils/iowrapper/interface.go Normal file
View File

@@ -0,0 +1,52 @@
/*
* MIT License
*
* Copyright (c) 2020 Nicolas JUHEL
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
*/
package iowrapper
import "io"
type FuncRead func(p []byte) []byte
type FuncWrite func(p []byte) []byte
type FuncSeek func(offset int64, whence int) (int64, error)
type FuncClose func() error
// IOWrapper is an interface that wraps basic I/O operations.
type IOWrapper interface {
io.Reader
io.Writer
io.Seeker
io.Closer
SetRead(read FuncRead)
SetWrite(write FuncWrite)
SetSeek(seek FuncSeek)
SetClose(close FuncClose)
}
func New(in any) IOWrapper {
return &iow{
i: in,
}
}

162
ioutils/iowrapper/model.go Normal file
View File

@@ -0,0 +1,162 @@
/*
* MIT License
*
* Copyright (c) 2020 Nicolas JUHEL
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
*/
package iowrapper
import (
"io"
libatm "github.com/nabbar/golib/atomic"
)
type iow struct {
i any
r libatm.Value[FuncRead]
w libatm.Value[FuncWrite]
s libatm.Value[FuncSeek]
c libatm.Value[FuncClose]
}
func (o *iow) SetRead(read FuncRead) {
if read == nil {
read = o.fakeRead
}
o.r.Store(read)
}
func (o *iow) SetWrite(write FuncWrite) {
if write == nil {
write = o.fakeWrite
}
o.w.Store(write)
}
func (o *iow) SetSeek(seek FuncSeek) {
if seek == nil {
seek = o.fakeSeek
}
o.s.Store(seek)
}
func (o *iow) SetClose(close FuncClose) {
if close == nil {
close = o.fakeClose
}
o.c.Store(close)
}
func (o *iow) Read(p []byte) (n int, err error) {
var r []byte
if o.r != nil {
r = o.r.Load()(p)
} else {
r = o.fakeRead(p)
}
if r == nil {
return 0, io.ErrUnexpectedEOF
}
copy(p, r)
return len(r), nil
}
func (o *iow) Write(p []byte) (n int, err error) {
var r []byte
if o.w != nil {
r = o.w.Load()(p)
} else {
r = o.fakeWrite(p)
}
if r == nil {
return 0, io.ErrUnexpectedEOF
}
copy(p, r)
return len(r), nil
}
func (o *iow) Seek(offset int64, whence int) (int64, error) {
if o.s != nil {
return o.s.Load()(offset, whence)
} else {
return o.fakeSeek(offset, whence)
}
}
func (o *iow) Close() error {
if o.c != nil {
return o.c.Load()()
} else {
return o.fakeClose()
}
}
func (o *iow) fakeRead(p []byte) []byte {
if r, k := o.i.(io.Reader); k {
n, err := r.Read(p)
if err != nil {
return p[:n]
}
return p[:n]
} else {
return nil
}
}
func (o *iow) fakeWrite(p []byte) []byte {
if r, k := o.i.(io.Writer); k {
n, err := r.Write(p)
if err != nil {
return p[:n]
}
return p[:n]
} else {
return nil
}
}
func (o *iow) fakeSeek(offset int64, whence int) (int64, error) {
if r, k := o.i.(io.Seeker); k {
return r.Seek(offset, whence)
} else {
return 0, io.ErrUnexpectedEOF
}
}
func (o *iow) fakeClose() error {
if r, k := o.i.(io.Closer); k {
return r.Close()
} else {
return nil
}
}

32
ioutils/nopwritecloser/interface.go Normal file
View File

@@ -0,0 +1,32 @@
/*
* MIT License
*
* Copyright (c) 2020 Nicolas JUHEL
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
*/
package nopwritecloser
import "io"
func New(w io.Writer) io.WriteCloser {
return &wrp{w: w}
}

6
archive/writecloser.go → ioutils/nopwritecloser/model.go
View File

@@ -23,14 +23,10 @@
*
*/
package archive
package nopwritecloser
import "io"
func NopWriteCloser(w io.Writer) io.WriteCloser {
return &wrp{w: w}
}
type wrp struct {
w io.Writer
}

145
ldap/README.md
View File

@@ -0,0 +1,145 @@
## `ldap` Package Documentation
> **Note:**
> This package uses an older design and would benefit from a refactor to modern Go idioms and best practices.
---
### Overview
The `ldap` package provides helpers for connecting to, authenticating with, and querying LDAP servers in Go. It supports both plain and TLS/StartTLS connections, user and group lookups, and flexible configuration.
---
### Features
- Connect to LDAP servers with or without TLS/StartTLS
- Bind and authenticate users
- Retrieve user and group information
- Check group membership and list group members
- Customizable search filters and attributes
- Integrated error handling with custom codes
- Logging support for debugging and tracing
---
### Main Types
#### `Config`
Represents the LDAP server configuration.
- `Uri`: Server hostname (FQDN, required)
- `PortLdap`: LDAP port (required, integer)
- `Portldaps`: LDAPS port (optional, integer)
- `Basedn`: Base DN for searches
- `FilterGroup`: Pattern for group search (e.g., `(&(objectClass=groupOfNames)(%s=%s))`)
- `FilterUser`: Pattern for user search (e.g., `(%s=%s)`)
**Validation:**
Use `Validate()` to check config correctness.
#### `TLSMode`
Enum for connection mode:
- `TLSModeNone`: No TLS
- `TLSModeTLS`: Strict TLS
- `TLSModeStarttls`: StartTLS
- `_TLSModeInit`: Not defined
#### `HelperLDAP`
Main struct for managing LDAP connections and queries.
- `NewLDAP(ctx, config, attributes)`: Create a new helper
- `SetLogger(fct)`: Set a logger function
- `SetCredentials(user, pass)`: Set bind DN and password
- `ForceTLSMode(mode, tlsConfig)`: Force a specific TLS mode and config
---
### Main Methods
- `Check()`: Test connection (no bind)
- `Connect()`: Connect and bind using credentials
- `AuthUser(username, password)`: Test user bind
- `UserInfo(username)`: Get user attributes as a map
- `UserInfoByField(username, field)`: Get user info by a specific field
- `GroupInfo(groupname)`: Get group attributes as a map
- `GroupInfoByField(groupname, field)`: Get group info by a specific field
- `UserMemberOf(username)`: List groups a user belongs to
- `UserIsInGroup(username, groupnames)`: Check if user is in any of the given groups
- `UsersOfGroup(groupname)`: List users in a group
- `ParseEntries(entry)`: Parse DN or attribute string into a map
---
### Error Handling
All errors are wrapped with custom codes for diagnostics, such as:
- `ErrorParamEmpty`
- `ErrorLDAPContext`
- `ErrorLDAPServerConfig`
- `ErrorLDAPServerConnection`
- `ErrorLDAPBind`
- `ErrorLDAPSearch`
- `ErrorLDAPUserNotFound`
- `ErrorLDAPGroupNotFound`
- ...and more
Use `err.Error()` for user-friendly messages and check error codes for diagnostics.
---
### Example Usage
```go
import (
"context"
"github.com/nabbar/golib/ldap"
)
cfg := ldap.Config{
Uri: "ldap.example.com",
PortLdap: 389,
Portldaps: 636,
Basedn: "dc=example,dc=com",
FilterUser: "(uid=%s)",
FilterGroup: "(&(objectClass=groupOfNames)(cn=%s))",
}
if err := cfg.Validate(); err != nil {
// handle config error
}
helper, err := ldap.NewLDAP(context.Background(), &cfg, ldap.GetDefaultAttributes())
if err != nil {
// handle error
}
helper.SetCredentials("cn=admin,dc=example,dc=com", "password")
if err := helper.Connect(); err != nil {
// handle connection/bind error
}
userInfo, err := helper.UserInfo("jdoe")
if err != nil {
// handle user lookup error
}
// ... use userInfo map
helper.Close()
```
---
### Notes
- The package is thread-safe for most operations.
- Designed for Go 1.18+.
- Logging is optional but recommended for debugging.
- The API and code structure are legacy and may not follow modern Go conventions.

1267
logger/README.md
View File

File diff suppressed because it is too large Load Diff

120
mail/README.md Normal file
View File

@@ -0,0 +1,120 @@
# `mail` Package Documentation
The `mail` package provides a flexible and extensible API for composing, configuring, and sending emails in Go applications. It supports advanced features such as multiple recipients, attachments, inline files, custom headers, content types, encoding, and priority management.
---
## Features
- Compose emails with plain text and HTML bodies
- Add attachments and inline files
- Set custom headers, subject, charset, encoding, and priority
- Manage sender, recipients (To, Cc, Bcc), reply-to, and return path
- Validate email configuration
- Send emails via SMTP with progress tracking and error handling
- Thread-safe and suitable for production use
---
## Main Types
### `Mail` Interface
Defines the main email object with methods to configure and retrieve all aspects of an email:
- Set and get charset, subject, encoding, and priority
- Set and get date/time
- Add and retrieve custom headers
- Manage email bodies (plain text, HTML)
- Add and retrieve attachments and inline files
- Manage sender and recipients
- Create a `Sender` for SMTP delivery
### `Config` Struct
A configuration struct for easy mapping from config files or environment variables:
- Charset, subject, encoding, priority
- Headers (map of key-value pairs)
- From, sender, reply-to, return path
- Lists of recipients (To, Cc, Bcc)
- Attachments and inline files (with name, mime, and path)
- Validation method to ensure all required fields are set
### `Sender` Interface
Handles the actual sending of the composed email via an SMTP client:
- `Send(ctx, smtp)`: Sends the email
- `SendClose(ctx, smtp)`: Sends and closes resources
- `Close()`: Cleans up resources
---
## Content Types and Encoding
- Supports plain text and HTML bodies
- Encoding options: none, binary, base64, quoted-printable
- Priority options: normal, low, high
---
## Example Usage
```go
import (
"github.com/nabbar/golib/mail"
"github.com/nabbar/golib/smtp"
"context"
)
m := mail.New()
m.SetSubject("Test Email")
m.SetCharset("UTF-8")
m.SetPriority(mail.PriorityHigh)
m.Email().SetFrom("sender@example.com")
m.Email().AddRecipients(mail.RecipientTo, "recipient@example.com")
// Add body, attachments, etc.
sender, err := m.Sender()
if err != nil {
// handle error
}
defer sender.Close()
smtpClient := /* initialize SMTP client */
err = sender.Send(context.Background(), smtpClient)
if err != nil {
// handle error
}
```
---
## Configuration Example
```go
cfg := mail.Config{
Charset: "UTF-8",
Subject: "Hello",
Encoding: "Base 64",
Priority: "High",
From: "me@example.com",
To: []string{"you@example.com"},
Attach: []mail.ConfigFile{{Name: "file.txt", Mime: "text/plain", Path: "/tmp/file.txt"}},
}
if err := cfg.Validate(); err != nil {
// handle validation error
}
mailer, err := cfg.NewMailer()
```
---
## Notes
- All operations are safe for concurrent use.
- Designed for Go 1.18+.
- Integrates with external SMTP clients and file progress tracking.
- Provides detailed error codes for robust error handling.

95
mailPooler/README.md Normal file
View File

@@ -0,0 +1,95 @@
# mailPooler Package Documentation
The `mailPooler` package provides a rate-limited SMTP client pooler for sending emails in Go applications. It wraps an SMTP client with configurable limits on the number of emails sent within a given time window, ensuring compliance with provider restrictions and preventing overload.
---
## Features
- Rate-limiting for SMTP email sending (max emails per duration)
- Thread-safe and context-aware
- Custom callback on pool reset
- Cloning and resetting of poolers
- Full SMTP client interface support (send, check, update config, close)
- Monitoring integration
---
## Main Types
### Pooler Interface
Defines the main pooler object, combining rate-limiting and SMTP client operations:
- `Reset() error` — Resets the pooler and underlying SMTP client
- `NewPooler() Pooler` — Clones the pooler with the same configuration
- All methods from the SMTP client interface (send, check, update config, close)
### Config Struct
Configuration for the pooler:
- `Max int` — Maximum number of emails allowed per window
- `Wait time.Duration` — Time window for the rate limit
- `SetFuncCaller(fct FuncCaller)` — Sets a callback function called on pool reset
### Counter
Internal rate-limiting logic:
- `Pool(ctx context.Context) error` — Checks and updates the rate limit before sending
- `Reset() error` — Resets the counter and triggers the callback
- `Clone() Counter` — Clones the counter
---
## Error Handling
- Custom error codes for empty parameters, generic pooler errors, and context cancellation
- Errors are returned with descriptive messages
---
## Example Usage
```go
import (
"github.com/nabbar/golib/mailPooler"
"github.com/nabbar/golib/smtp"
"context"
"time"
)
cfg := &mailPooler.Config{
Max: 10, // max 10 emails
Wait: 1 * time.Minute, // per minute
}
cfg.SetFuncCaller(func() error {
// Custom logic on reset (optional)
return nil
})
smtpClient, _ := smtp.New(/* ... */)
pooler := mailPooler.New(cfg, smtpClient)
err := pooler.Send(context.Background(), "from@example.com", []string{"to@example.com"}, /* data */)
if err != nil {
// handle error
}
```
---
## Monitoring
- The pooler supports monitoring integration, delegating to the underlying SMTP client.
---
## Notes
- Designed for Go 1.18+.
- All operations are thread-safe.
- Suitable for production and high-concurrency environments.
- The pooler can be cloned and reset as needed.

124
mailer/README.md Normal file
View File

@@ -0,0 +1,124 @@
# mailer Package Documentation
The `mailer` package provides a flexible API for composing, theming, and rendering transactional emails in Go applications. It leverages the Hermes library for HTML and plain text output, supports dynamic data injection, and allows full customization of product and message content.
---
## Features
- Compose emails with customizable themes and text direction
- Set product information: name, link, logo, copyright, trouble text
- Define email body with intros, outros, tables, actions, and markdown
- Inject dynamic data into all fields
- Generate HTML and plain text content
- Validate configuration with detailed error reporting
- Thread-safe and suitable for production use
---
## Main Types
### Mailer Interface
Defines the main email object with methods to configure and retrieve all aspects of an email:
- `SetTheme(t Themes)` / `GetTheme()`
- `SetTextDirection(d TextDirection)` / `GetTextDirection()`
- `SetBody(b *hermes.Body)` / `GetBody()`
- `SetCSSInline(disable bool)`
- `SetName(name string)` / `GetName()`
- `SetCopyright(copy string)` / `GetCopyright()`
- `SetLink(link string)` / `GetLink()`
- `SetLogo(logoUrl string)` / `GetLogo()`
- `SetTroubleText(text string)` / `GetTroubleText()`
- `ParseData(data map[string]string)` — injects dynamic values into all fields
- `GenerateHTML()` — returns the email as HTML
- `GeneratePlainText()` — returns the email as plain text
- `Clone()` — deep copy of the mailer
### Config Struct
A configuration struct for easy mapping from config files or environment variables:
- Theme, direction, name, link, logo, copyright
- Trouble text, disable CSS inlining
- Body (hermes.Body)
- Validation method to ensure all required fields are set
- `NewMailer()` — creates a Mailer from the config
### Themes and TextDirection
- `Themes`: `ThemeDefault`, `ThemeFlat`
- `TextDirection`: `LeftToRight`, `RightToLeft`
- Parsing helpers: `ParseTheme(string)`, `ParseTextDirection(string)`
---
## Example Usage
```go
import (
"github.com/nabbar/golib/mailer"
"github.com/matcornic/hermes/v2"
)
m := mailer.New()
m.SetTheme(mailer.ThemeFlat)
m.SetTextDirection(mailer.LeftToRight)
m.SetName("MyApp")
m.SetLink("https://myapp.example.com")
m.SetLogo("https://myapp.example.com/logo.png")
m.SetCopyright("© 2024 MyApp")
m.SetTroubleText("If you’re having trouble, contact support.")
body := &hermes.Body{
Name: "John Doe",
Intros: []string{"Welcome to MyApp!"},
Outros: []string{"Thank you for joining."},
Actions: []hermes.Action{{Button: hermes.Button{Text: "Get Started", Link: "https://myapp.example.com/start"}}},
}
m.SetBody(body)
// Inject dynamic data
m.ParseData(map[string]string{"MyApp": "YourApp"})
// Generate HTML and plain text
html, err := m.GenerateHTML()
text, err := m.GeneratePlainText()
```
---
## Configuration Example
```go
cfg := mailer.Config{
Theme: "Default",
Direction: "Left->Right",
Name: "MyApp",
Link: "https://myapp.example.com",
Logo: "https://myapp.example.com/logo.png",
Copyright: "© 2024 MyApp",
TroubleText: "If you’re having trouble, contact support.",
Body: hermes.Body{ /* ... */ },
}
if err := cfg.Validate(); err != nil {
// handle validation error
}
mailer := cfg.NewMailer()
```
---
## Error Handling
- Errors are returned as custom error types with codes for invalid config, HTML/text generation, or empty parameters.
---
## Notes
- Designed for Go 1.18+.
- All operations are thread-safe.
- Integrates with the [hermes](https://github.com/matcornic/hermes) library for email rendering.
- Suitable for high-concurrency and production environments.

482
monitor/README.md Normal file
View File

@@ -0,0 +1,482 @@
# Documentation - `github.com/nabbar/golib/monitor`
This documentation provides an overview and usage guide for the `github.com/nabbar/golib/monitor` package and its subpackages. The package is designed to help developers implement, manage, and monitor health checks for various components in their applications, with advanced features such as metrics collection, status management, and pooling of monitors.
## Subpackages
- [`monitor/pool`](#monitorpool-subpackage-documentation): Manages a collection of health monitors as a group. See the [monitor/pool documentation](#monitorpool-subpackage-documentation) for details.
- [`monitor`](#monitor-package-documentation): Core logic for defining, running, and monitoring health checks. See the [monitor documentation](#monitor-package-documentation) for details.
- [`monitor/info`](#monitorinfo-subpackage-documentation): Provides types and utilities for component metadata. See the [monitor/info documentation](#monitorinfo-subpackage-documentation) for details.
- [`monitor/status`](#monitorstatus-subpackage-documentation): Defines status types and utilities for health status management. See the [monitor/status documentation](#monitorstatus-subpackage-documentation) for details.
---
## monitor/pool Subpackage Documentation
The `monitor/pool` subpackage provides a system to manage and operate a collection of health monitors as a group. It enables dynamic addition, removal, lifecycle control, metrics aggregation, and operational shell commands for all monitors in the pool. All operations are thread-safe and suitable for concurrent environments.
---
### Features
- **Dynamic Monitor Management**: Add, get, set, delete, list, and walk through monitors in the pool.
- **Lifecycle Control**: Start, stop, and restart all or selected monitors.
- **Metrics Aggregation**: Collect and export metrics (latency, uptime, downtime, status, SLI, etc.) for all monitors.
- **Shell Command Integration**: Expose operational commands for listing, controlling, and querying monitors.
- **Prometheus & Logger Integration**: Register Prometheus and logger functions for observability.
- **Thread-Safe**: All operations are safe for concurrent use.
---
### Main Concepts
#### Pool Creation
Create a new pool by providing a context function. The pool can be further configured with Prometheus and logger integrations.
```go
import (
"github.com/nabbar/golib/monitor/pool"
"github.com/nabbar/golib/context"
)
p := pool.New(context.NewFuncContext())
```
#### Monitor Management
- **Add**: Add a new monitor to the pool.
- **Get**: Retrieve a monitor by name.
- **Set**: Update or replace a monitor in the pool.
- **Delete**: Remove a monitor by name.
- **List**: List all monitor names.
- **Walk**: Iterate over all monitors, optionally filtering by name.
```go
p.MonitorAdd(monitor)
mon := p.MonitorGet("name")
p.MonitorSet(monitor)
p.MonitorDel("name")
names := p.MonitorList()
p.MonitorWalk(func(name string, mon Monitor) bool { /* ... */ })
```
#### Lifecycle Management
- **Start**: Start all monitors in the pool.
- **Stop**: Stop all monitors.
- **Restart**: Restart all monitors.
- **IsRunning**: Check if any monitor is running.
- **Uptime**: Get the maximum uptime among all monitors.
```go
err := p.Start(ctx)
err := p.Stop(ctx)
err := p.Restart(ctx)
running := p.IsRunning()
uptime := p.Uptime()
```
#### Metrics Collection
- **InitMetrics**: Register Prometheus and logger functions, and initialize metrics.
- **TriggerCollectMetrics**: Periodically trigger metrics collection for all monitors.
- **Metrics Export**: Metrics include latency, uptime, downtime, rise/fall times, status, and SLI rates.
```go
p.InitMetrics(prometheusFunc, loggerFunc)
go p.TriggerCollectMetrics(ctx, time.Minute)
```
#### Shell Command Integration
The pool exposes shell commands for operational control:
- `list`: List all monitors.
- `info`: Show detailed info for monitors.
- `start`, `stop`, `restart`: Control monitor lifecycle.
- `status`: Show status and messages for monitors.
Retrieve available shell commands:
```go
cmds := p.GetShellCommand(ctx)
```
---
### Encoding and Export
- **MarshalText**: Export the pool as a human-readable text.
- **MarshalJSON**: Export the pool as a JSON object with monitor statuses.
```go
txt, err := p.MarshalText()
jsn, err := p.MarshalJSON()
```
---
### Notes
- All operations are thread-safe and suitable for concurrent use.
- Designed for Go 1.18+.
- Integrates with Prometheus and logging systems for observability.
- Can be used as a standalone pool or as part of a larger monitoring system.
---
## monitor Package Documentation
The `monitor` package provides the core logic for defining, running, and monitoring health checks for individual components in Go applications. It offers a flexible, thread-safe abstraction for health monitoring, status management, metrics collection, and integration with logging and Prometheus.
---
### Features
- **Monitor abstraction**: Encapsulates health check logic, status, metrics, and configuration.
- **Custom health checks**: Register any function as a health check.
- **Status management**: Tracks status (`OK`, `Warn`, `KO`), rise/fall transitions, and error messages.
- **Metrics collection**: Latency, uptime, downtime, rise/fall times, and status.
- **Flexible configuration**: Control check intervals, timeouts, and thresholds.
- **Logger integration**: Pluggable logging for each monitor.
- **Cloning**: Clone monitors with new contexts.
- **Prometheus integration**: Register and collect custom metrics.
- **Thread-safe**: All operations are safe for concurrent use.
---
### Main Concepts
#### Monitor Creation
Create a monitor by providing a context and an `Info` object describing the monitored component.
```go
import (
"github.com/nabbar/golib/monitor"
"github.com/nabbar/golib/monitor/info"
)
inf, _ := info.New("MyComponent")
mon, err := monitor.New(nil, inf)
if err != nil {
// handle error
}
```
#### Health Check Registration
Assign a health check function to the monitor. This function will be called periodically according to the configured intervals.
```go
mon.SetHealthCheck(func(ctx context.Context) error {
// custom health check logic
return nil // or return an error if unhealthy
})
```
#### Configuration
Configure the monitor with check intervals, timeouts, and thresholds for status transitions (rise/fall counts).
```go
cfg := monitor.Config{
Name: "MyMonitor",
CheckTimeout: 5 * time.Second,
IntervalCheck: 10 * time.Second,
IntervalFall: 10 * time.Second,
IntervalRise: 10 * time.Second,
FallCountKO: 2,
FallCountWarn: 1,
RiseCountKO: 1,
RiseCountWarn: 2,
Logger: /* logger options */,
}
mon.SetConfig(nil, cfg)
```
#### Status and Metrics
- **Status**: The monitor tracks its current status and transitions (rise/fall).
- **Metrics**: Latency, uptime, downtime, rise/fall times, and status are tracked and can be exported.
```go
status := mon.Status() // Current status (OK, Warn, KO)
latency := mon.Latency() // Last check latency
uptime := mon.Uptime() // Total uptime
downtime := mon.Downtime() // Total downtime
```
#### Lifecycle
- **Start**: Begin periodic health checks.
- **Stop**: Stop health checks.
- **Restart**: Restart the monitor.
- **IsRunning**: Check if the monitor is active.
```go
err := mon.Start(ctx)
defer mon.Stop(ctx)
```
#### Encoding and Export
Monitors can be encoded as text or JSON for reporting and integration.
```go
txt, _ := mon.MarshalText()
jsn, _ := mon.MarshalJSON()
```
---
### Metrics Integration
- Register custom metric names and collection functions for Prometheus integration.
- Collect latency, uptime, downtime, rise/fall times, and status.
```go
mon.RegisterMetricsName("my_metric")
mon.RegisterCollectMetrics(func(ctx context.Context, names ...string) {
// custom Prometheus collection logic
})
```
---
### Error Handling
- Custom error codes for empty parameters, missing health checks, invalid config, logger errors, and timeouts.
- Errors are returned as descriptive error types.
---
### Notes
- All operations are thread-safe and suitable for concurrent use.
- Designed for Go 1.18+.
- Integrates with logging and Prometheus for observability.
- Can be used standalone or as part of a monitor pool.
---
### Example
```go
import (
"github.com/nabbar/golib/monitor"
"github.com/nabbar/golib/monitor/info"
"context"
"time"
)
inf, _ := info.New("API")
mon, _ := monitor.New(nil, inf)
mon.SetHealthCheck(func(ctx context.Context) error {
// check API health
return nil
})
cfg := monitor.Config{
Name: "API",
CheckTimeout: 5 * time.Second,
IntervalCheck: 30 * time.Second,
// ...
}
mon.SetConfig(nil, cfg)
mon.Start(context.Background())
defer mon.Stop(context.Background())
```
---
## monitor/info Subpackage Documentation
The `monitor/info` subpackage provides types and utilities to describe and manage metadata for monitored components. It enables dynamic registration and retrieval of component names and additional information, supporting both static and runtime-generated data.
---
### Features
- Register custom functions to provide the component name and additional info dynamically.
- Store and retrieve metadata as key-value pairs.
- Thread-safe operations for concurrent environments.
- Encode info as string, bytes, text, or JSON for reporting and integration.
---
### Main Types
#### Info Interface
Defines the contract for managing component metadata:
- `RegisterName(FuncName)`: Register a function to provide the component name.
- `RegisterInfo(FuncInfo)`: Register a function to provide additional info as a map.
- `Name() string`: Retrieve the current name (from registered function or stored value).
- `Info() map[string]interface{}`: Retrieve the current info map (from registered function or stored values).
#### FuncName and FuncInfo
- `FuncName`: `func() (string, error)` — Function type to provide a name.
- `FuncInfo`: `func() (map[string]interface{}, error)` — Function type to provide info.
#### Encode Interface
- `String() string`: Returns a human-readable string representation.
- `Bytes() []byte`: Returns a byte slice representation.
---
### Usage
#### Creating an Info Object
```go
import "github.com/nabbar/golib/monitor/info"
inf, err := info.New("DefaultName")
if err != nil {
// handle error
}
```
#### Registering Dynamic Name and Info
```go
inf.RegisterName(func() (string, error) {
return "DynamicName", nil
})
inf.RegisterInfo(func() (map[string]interface{}, error) {
return map[string]interface{}{
"version": "1.0.0",
"env": "production",
}, nil
})
```
#### Retrieving Name and Info
```go
name := inf.Name()
meta := inf.Info()
```
#### Encoding
- `inf.MarshalText()` and `inf.MarshalJSON()` provide text and JSON representations for integration and reporting.
---
### Notes
- If no dynamic function is registered, the default name and info are used.
- The subpackage is thread-safe and suitable for concurrent use.
- Designed for extensibility and integration with the main monitor system.
---
## monitor/status Subpackage Documentation
The `monitor/status` subpackage defines the status types and utilities for managing and encoding the health status of monitored components. It provides a simple, extensible way to represent, parse, and serialize status values for health checks and monitoring systems.
---
### Features
- Defines standard status values: `KO`, `Warn`, `OK`
- Conversion between status and string, integer, or float representations
- Parsing from string or integer to status
- JSON marshaling and unmarshaling support
- Thread-safe and lightweight
---
### Main Types
#### Status Type
Represents the health status as an unsigned 8-bit integer with three possible values:
- `KO` (default, value 0): Component is not operational
- `Warn` (value 1): Component is in a warning state
- `OK` (value 2): Component is healthy
##### Methods
- `String() string`: Returns the string representation (`"OK"`, `"Warn"`, or `"KO"`)
- `Int() int64`: Returns the integer value of the status
- `Float() float64`: Returns the float value of the status
- `MarshalJSON() ([]byte, error)`: Serializes the status as a JSON string
- `UnmarshalJSON([]byte) error`: Parses the status from a JSON string or integer
---
### Constructors
- `NewFromString(sts string) Status`: Parses a status from a string (`"OK"`, `"Warn"`, or any other string for `KO`)
- `NewFromInt(sts int64) Status`: Parses a status from an integer (returns `OK`, `Warn`, or `KO`)
---
### Usage Example
```go
import "github.com/nabbar/golib/monitor/status"
var s status.Status
s = status.NewFromString("OK") // s == status.OK
s = status.NewFromInt(1) // s == status.Warn
str := s.String() // "Warn"
i := s.Int() // 1
f := s.Float() // 1.0
data, _ := s.MarshalJSON() // "\"Warn\""
_ = s.UnmarshalJSON([]byte("\"OK\"")) // s == status.OK
```
---
### Notes
- If parsing fails or the value is unknown, the status defaults to `KO`.
- The status type is designed for easy integration with monitoring, alerting, and reporting systems.
- Supports both string and numeric representations for flexibility in configuration and serialization.
---
## Usage Example
```go
import (
"github.com/nabbar/golib/monitor"
"github.com/nabbar/golib/monitor/pool"
// ... other imports
)
// Create a monitor
mon, err := monitor.New(ctx, info)
mon.SetHealthCheck(myHealthCheckFunc)
mon.SetConfig(ctx, myConfig)
// Create a pool and add monitors
p := pool.New(ctx)
p.MonitorAdd(mon)
p.Start(ctx)
// Collect metrics periodically
go p.TriggerCollectMetrics(ctx, time.Minute)
```
---
## Summary
- Use `monitor/pool` to manage multiple monitors as a group.
- Use `monitor` to define and run individual health checks.
- Integrate with logging and Prometheus for observability.
- Extend with `info` and `status` subpackages for richer metadata and status handling.