Files
golib/socket/server/unix/doc.go
nabbar 96ed6f9a1f [Package IOUtils/Delim]
- FIX: potential CWE-400 with bufio.ReadBytes & bufio.ReadSlices, with
  no limited read buffer
- ADD: test to check overflow buffer with discard or error
- REFACTOR: all buffering package, parsing process
- UPDATE: doc, examples, test following changes
- OPTIMIZE: rework code to optimize process
- REWORK: benchmark to check benefice of optimization
- FIX: wording error

Package IOUtils/Multi:
- REWORK: re-design all package to allow sequential/parallel mode
- UPDATE: package with adaptive mode to allow switch automaticly between
  sequential and parallel mode following measurment of sample
- OPTIMIZE: code to maximize bandwith and reduce time of write
- UPDATE: documentation, test and comments
- REWORK: testing organization and benchmark aggregation

Package HttpServer:
- FIX: bug with dial addr rewrite for healtcheck & testing PortUse

Package Logger/HookFile:
- FIX: bug with race condition on aggregator counter file

Other:
- Bump dependencies
- FIX: format / import file
2025-12-21 16:56:13 +01:00

446 lines
18 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//go:build linux || darwin
/*
* MIT License
*
* Copyright (c) 2025 Nicolas JUHEL
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
*/
// Package unix provides a robust, production-ready Unix domain socket server implementation
// for Go applications. It's designed to handle local inter-process communication (IPC)
// with a focus on reliability, performance, and ease of use.
//
// # Overview
//
// This package implements a high-performance Unix domain socket server that supports:
// - SOCK_STREAM (connection-oriented) Unix sockets for reliable IPC
// - File permissions and group ownership control for access management
// - Graceful shutdown with connection draining and timeout management
// - Context-aware operations with cancellation propagation
// - Configurable idle timeouts for inactive connections
// - Thread-safe concurrent connection handling (goroutine-per-connection)
// - Connection counting and tracking
// - Customizable connection configuration via UpdateConn callback
// - Comprehensive event callbacks for monitoring and logging
//
// # Architecture
//
// ## Component Diagram
//
// The server follows a layered architecture with clear separation of concerns:
//
// ┌─────────────────────────────────────────────────────┐
// │ Unix Socket Server │
// ├─────────────────────────────────────────────────────┤
// │ │
// │ ┌──────────────┐ ┌───────────────────┐ │
// │ │ Listener │ │ Context Manager │ │
// │ │ (Listen) │ │ (ctx tracking) │ │
// │ └──────┬───────┘ └─────────┬─────────┘ │
// │ │ │ │
// │ ▼ ▼ │
// │ ┌──────────────────────────────────────────┐ │
// │ │ Connection Acceptor │ │
// │ │ (Accept loop + file setup) │ │
// │ └──────────────┬───────────────────────────┘ │
// │ │ │
// │ ▼ │
// │ Per-Connection Goroutine │
// │ ┌─────────────────────┐ │
// │ │ Connection Context │ │
// │ │ - sCtx (I/O wrap) │ │
// │ │ - Idle timeout │ │
// │ │ - State tracking │ │
// │ └──────────┬──────────┘ │
// │ │ │
// │ ▼ │
// │ ┌─────────────────────┐ │
// │ │ User Handler Func │ │
// │ │ (custom logic) │ │
// │ └─────────────────────┘ │
// │ │
// │ Socket File Management: │
// │ - Creation with permissions │
// │ - Group ownership assignment │
// │ - Automatic cleanup on shutdown │
// │ │
// │ Optional Callbacks: │
// │ - UpdateConn: Connection configuration │
// │ - FuncError: Error reporting │
// │ - FuncInfo: Connection state changes │
// │ - FuncInfoSrv: Server lifecycle events │
// │ │
// └─────────────────────────────────────────────────────┘
//
// ## Data Flow
//
// 1. Server.Listen() starts the accept loop
// 2. For each new connection:
// a. net.Listener.Accept() receives the connection
// b. Connection counter incremented atomically
// c. UpdateConn callback invoked (if registered)
// d. Connection wrapped in sCtx (context + I/O)
// e. Handler goroutine spawned
// f. Idle timeout monitoring started (if configured)
// 3. Handler processes the connection
// 4. On close:
// a. Connection closed
// b. Context cancelled
// c. Counter decremented
// d. Goroutine terminates
//
// ## Lifecycle States
//
// The server maintains two atomic state flags:
//
// - IsRunning: Server is accepting new connections
//
// - false → true: Listen() called successfully
//
// - true → false: Shutdown/Close initiated
//
// - IsGone: Server is draining existing connections
//
// - false → true: Shutdown() called
//
// - Used to signal accept loop to stop
//
// ## Thread Safety Model
//
// Synchronization primitives used:
// - atomic.Bool: run, gon (server state)
// - atomic.Int64: nc (connection counter)
// - libatm.Value: fe, fi, fs, ad (atomic storage)
// - No mutexes: All state changes are lock-free
//
// Concurrency guarantees:
// - All exported methods are safe for concurrent use
// - Connection handlers run in isolated goroutines
// - No shared mutable state between connections
// - Atomic counters prevent race conditions
//
// # Features
//
// ## Unix Socket Benefits
// - Zero network overhead: Communication within the same host
// - File system permissions for access control
// - No TCP/IP stack overhead: Lower latency than TCP loopback
// - Higher throughput than TCP for local communication
// - Process credentials passing on Linux (SCM_CREDENTIALS)
// - File descriptor passing capability (SCM_RIGHTS)
//
// ## Security
// - File system permissions for access control (chmod)
// - Group ownership for fine-grained access (chown)
// - No network exposure: Not accessible over the network
// - Automatic socket file cleanup on shutdown
// - Configurable file permissions (0600, 0660, 0666, etc.)
//
// ## Reliability
// - Graceful shutdown with configurable timeouts
// - Connection draining during shutdown (wait for active connections)
// - Automatic resource reclamation (goroutines, memory, file descriptors)
// - Idle connection timeout with automatic cleanup
// - Context-aware operations with deadline and cancellation support
// - Error recovery and propagation
//
// ## Monitoring & Observability
// - Connection state change callbacks (new, read, write, close)
// - Error reporting through callback functions
// - Server lifecycle notifications
// - Real-time connection counting (OpenConnections)
// - Server state queries (IsRunning, IsGone)
//
// ## Performance
// - Goroutine-per-connection model (suitable for 100s-1000s of connections)
// - Lock-free atomic operations for state management
// - Zero-copy I/O where possible
// - Minimal memory overhead per connection (~10KB)
// - Efficient connection tracking without locks
// - Lower latency than TCP loopback (typically 2-5x faster)
//
// # Usage Example
//
// Basic echo server:
//
// import (
// "context"
// "io"
// "github.com/nabbar/golib/socket"
// "github.com/nabbar/golib/socket/config"
// "github.com/nabbar/golib/socket/server/unix"
// "github.com/nabbar/golib/file/perm"
// )
//
// func main() {
// handler := func(c socket.Context) {
// defer c.Close()
// io.Copy(c, c) // Echo back received data
// }
//
// cfg := config.Server{
// Network: protocol.NetworkUnix,
// Address: "/tmp/myapp.sock",
// PermFile: perm.New(0660),
// GroupPerm: -1, // Use default group
// }
//
// srv, err := unix.New(nil, handler, cfg)
// if err != nil {
// panic(err)
// }
//
// // Start the server
// if err := srv.Listen(context.Background()); err != nil {
// panic(err)
// }
// }
//
// # Concurrency Model
//
// ## Goroutine-Per-Connection
//
// The server uses a goroutine-per-connection model, where each accepted
// connection spawns a dedicated goroutine to handle it. This model is
// well-suited for:
//
// - Low to medium concurrent connections (100s to low 1000s)
// - Long-lived connections (persistent IPC, service communication)
// - Applications requiring per-connection state and context
// - Connections with varying processing times
// - Connections requiring blocking I/O operations
//
// ## Scalability Characteristics
//
// Typical performance profile:
//
// ┌─────────────────┬──────────────┬────────────────┬──────────────┐
// │ Connections │ Goroutines │ Memory Usage │ Throughput │
// ├─────────────────┼──────────────┼────────────────┼──────────────┤
// │ 10 │ ~12 │ ~100 KB │ Excellent │
// │ 100 │ ~102 │ ~1 MB │ Excellent │
// │ 1,000 │ ~1,002 │ ~10 MB │ Good │
// │ 10,000 │ ~10,002 │ ~100 MB │ Fair* │
// │ 100,000+ │ ~100,002+ │ ~1 GB+ │ Not advised │
// └─────────────────┴──────────────┴────────────────┴──────────────┘
//
// * At 10K+ connections, consider profiling and potentially switching to
// an event-driven model or worker pool architecture.
//
// ## Memory Overhead
//
// Per-connection memory allocation:
//
// Base overhead: ~8 KB (goroutine stack)
// Connection context: ~1 KB (sCtx structure)
// Buffers (handler): Variable (depends on implementation)
// ─────────────────────────────────
// Total minimum: ~10 KB per connection
//
// Example calculation for 1000 connections:
//
// 1000 connections × 10 KB = ~10 MB base
// + application buffers (e.g., 4KB read buffer × 1000 = 4 MB)
// = ~14 MB total for connections
//
// # Performance Considerations
//
// ## Throughput
//
// Unix sockets typically outperform TCP loopback for local IPC:
//
// - Unix socket: ~2-5x faster than TCP loopback
// - Lower latency: ~50% less than TCP loopback
// - Higher bandwidth: No TCP/IP stack overhead
//
// The server's throughput is primarily limited by:
//
// 1. Handler function complexity (CPU-bound operations)
// 2. Disk I/O if using file-based operations
// 3. System limits: File descriptors, kernel buffers
//
// Typical throughput (echo handler on localhost):
//
// - Unix socket: ~1M requests/sec (small payloads)
// - TCP loopback: ~500K requests/sec (same conditions)
//
// ## Latency
//
// Expected latency profile:
//
// ┌──────────────────────┬─────────────────┐
// │ Operation │ Typical Time │
// ├──────────────────────┼─────────────────┤
// │ Connection accept │ <500 µs │
// │ Handler spawn │ <100 µs │
// │ Context creation │ <10 µs │
// │ Read/Write syscall │ <50 µs │
// │ Graceful shutdown │ 100 ms - 1 s │
// └──────────────────────┴─────────────────┘
//
// ## Resource Limits
//
// System-level limits to consider:
//
// 1. File Descriptors:
// - Each connection uses 1 file descriptor
// - Check: ulimit -n (default often 1024 on Linux)
// - Increase: ulimit -n 65536 or via /etc/security/limits.conf
//
// 2. Socket Buffer Memory:
// - Per-connection send/receive buffers (typically smaller than TCP)
// - Tune: sysctl net.unix.max_dgram_qlen (for datagram sockets)
//
// 3. Filesystem:
// - Socket file must be in a writable directory
// - Path length limited (typically 108 characters on Linux)
// - Cleanup required if process crashes
//
// # Limitations
//
// ## Known Limitations
//
// 1. No built-in rate limiting or connection throttling
// 2. No support for connection pooling or multiplexing
// 3. Goroutine-per-connection model limits scalability >10K connections
// 4. No built-in protocol framing (implement in handler)
// 5. No built-in metrics export (Prometheus, etc.)
// 6. Platform support: Linux and macOS only (not Windows)
//
// ## Not Suitable For
//
// - Remote connections (use TCP instead)
// - Ultra-high concurrency scenarios (>50K simultaneous connections)
// - Low-latency HFT applications (<10µs response time)
// - Systems requiring protocol multiplexing (use gRPC)
// - Short-lived connections at very high rates (>100K conn/sec)
// - Windows platforms (Unix sockets not supported)
//
// ## Comparison with Alternatives
//
// ┌──────────────────┬────────────────┬──────────────────┬──────────────┐
// │ Feature │ Unix Socket │ TCP Loopback │ Named Pipe │
// ├──────────────────┼────────────────┼──────────────────┼──────────────┤
// │ Overhead │ Minimal │ TCP/IP stack │ Minimal │
// │ Throughput │ Very High │ High │ High │
// │ Latency │ Very Low │ Low │ Low │
// │ Permissions │ Filesystem │ Firewall │ Filesystem │
// │ Network Access │ No │ Yes (loopback) │ No │
// │ Platform │ Unix/Linux │ All platforms │ Windows │
// │ Best For │ Local IPC │ Network compat │ Windows IPC │
// └──────────────────┴────────────────┴──────────────────┴──────────────┘
//
// # Best Practices
//
// ## Resource Management
//
// 1. Always use defer for cleanup:
//
// defer srv.Close() // Server
// defer ctx.Close() // Connection (in handler)
//
// 2. Implement graceful shutdown:
//
// sigChan := make(chan os.Signal, 1)
// signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
//
// <-sigChan
// log.Println("Shutting down...")
//
// shutdownCtx, cancel := context.WithTimeout(
// context.Background(), 30*time.Second)
// defer cancel()
//
// if err := srv.Shutdown(shutdownCtx); err != nil {
// log.Printf("Shutdown error: %v", err)
// }
//
// 3. Monitor connection count:
//
// go func() {
// ticker := time.NewTicker(10 * time.Second)
// defer ticker.Stop()
//
// for range ticker.C {
// count := srv.OpenConnections()
// if count > warnThreshold {
// log.Printf("WARNING: High connection count: %d", count)
// }
// }
// }()
//
// ## Security
//
// 1. Set restrictive file permissions:
//
// cfg.PermFile = perm.New(0600) // Owner only
// cfg.PermFile = perm.New(0660) // Owner + group
//
// 2. Use group ownership for access control:
//
// cfg.GroupPerm = getGidForService()
//
// 3. Configure idle timeouts to prevent resource exhaustion:
//
// cfg.ConIdleTimeout = 5 * time.Minute
//
// 4. Validate input in handlers (prevent injection, DoS, etc.)
//
// 5. Clean up socket files on unexpected termination
//
// ## Testing
//
// 1. Test with concurrent connections:
//
// for i := 0; i < numClients; i++ {
// go func() {
// conn, _ := net.Dial("unix", socketPath)
// defer conn.Close()
// // Test logic...
// }()
// }
//
// 2. Test graceful shutdown under load
//
// 3. Test with slow/misbehaving clients
//
// 4. Run with race detector: go test -race
//
// # Platform Support
//
// This package is only available on:
// - Linux (all architectures)
// - macOS/Darwin (all architectures)
//
// For Windows or other platforms, the package provides stub implementations
// that return nil. See ignore.go for unsupported platform handling.
//
// # Related Packages
//
// - github.com/nabbar/golib/socket: Base interfaces and types
// - github.com/nabbar/golib/socket/config: Server configuration
// - github.com/nabbar/golib/file/perm: File permission management
// - github.com/nabbar/golib/network/protocol: Protocol constants
//
// See the example_test.go file for runnable examples covering common use cases.
package unix