golib/retro

Retro Package

Version-aware struct serialization library for Go with semantic versioning support, enabling backward compatibility across multiple data formats.

---

Table of Contents

• Overview
• Key Features
• Installation
• Architecture
• Quick Start
• Performance
• Use Cases
• Best Practices
• Testing
• Contributing
• Future Enhancements
• AI Transparency Notice
• License
• Resources

---

Overview

This library provides version-aware serialization for Go structs, allowing fields to be conditionally included or excluded based on semantic version constraints. It enables smooth data format evolution while maintaining backward compatibility across API versions, configuration files, and data migrations.

Design Philosophy

1. Version-Driven: Fields visibility controlled by semantic versioning constraints
2. Format Agnostic: Supports JSON, YAML, and TOML with consistent behavior
3. Type-Safe: Generic implementation ensures compile-time type safety
4. Zero Dependencies: Only uses standard library and well-established serialization packages
5. Backward Compatible: Seamlessly handle multiple struct versions in the same codebase

---

Key Features

• Semantic Versioning: Full support for semver constraints (>=, <=, >, <, ranges)
• Multiple Formats: JSON, YAML, and TOML serialization with identical semantics
• Generic Implementation: Type-safe wrapper using Go generics (Go 1.18+)
• Version Ranges: Support for dual boundaries (e.g., ">=v1.0.0,<v2.0.0")
• Omitempty Support: Respects omitempty tags for cleaner output
• Standard Mode: Optional bypass for standard serialization without filtering
• Custom Unmarshalers: Full support for custom marshal/unmarshal implementations

---

Installation

go get github.com/nabbar/golib/retro

Requirements:

• Go 1.18 or higher (for generics support)
• encoding/json (standard library)
• gopkg.in/yaml.v3
• github.com/pelletier/go-toml

---

Architecture

Package Structure

The package is organized into focused components for version-aware serialization:

retro/
├── format.go        # Format types and validation
├── model.go         # Generic Model wrapper and core logic
├── encoding.go      # Marshal/Unmarshal implementations
├── version.go       # Version comparison and constraint evaluation
└── utils.go         # Helper functions (empty value detection)

Component Overview

┌─────────────────────────────────────────────────────────┐
│                    Retro Package                         │
│  Model[T], Version Constraints, Format Support          │
└──────────────┬──────────────┬──────────────┬────────────┘
               │              │              │
      ┌────────▼─────┐  ┌────▼─────┐  ┌────▼────────┐
      │   Encoding   │  │  Version │  │   Format    │
      │              │  │          │  │             │
      │ JSON/YAML/   │  │ Semver   │  │ Validation  │
      │ TOML Support │  │ Compare  │  │ Constants   │
      └──────────────┘  └──────────┘  └─────────────┘

Component	Purpose	Complexity	Test Coverage
Model	Generic wrapper with filtering logic	O(n) fields	74.4%
Encoding	Format-specific marshal/unmarshal	O(1)	85.4%
Version	Semver parsing and comparison	O(1)	100%
Utils	Empty value detection	O(1)	100%

Version Constraint Flow

Struct Field
     │
     ▼
┌─────────────────┐
│ Extract Version │  ← From struct.Version field
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Parse Retro Tag │  ← ">=v1.0.0,<v2.0.0"
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Validate Format │  ← Check operators, boundaries
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Compare Versions│  ← Semver comparison
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Include/Exclude │  ← Field visibility decision
└─────────────────┘

---

Performance

Memory Efficiency

The library maintains minimal memory overhead:

• Reflection Usage: Only during marshal/unmarshal operations
• No Caching: Stateless operations, no memory accumulation
• Generic Wrapper: Zero-cost abstraction at runtime
• Example: Serialize 1000 structs with 50 fields using ~5MB RAM

Version Comparison Performance

Operation	Throughput	Complexity	Notes
Version Parse	~10M ops/s	O(1)	Regex-based validation
Version Compare	~50M ops/s	O(1)	Numeric comparison
Constraint Check	~5M ops/s	O(n)	n = number of constraints
Field Filter	~1M fields/s	O(n)	n = number of fields
JSON Marshal	~500K ops/s	O(n)	Standard json.Marshal
YAML Marshal	~200K ops/s	O(n)	yaml.v3 overhead
TOML Marshal	~300K ops/s	O(n)	TOML encoding

Measured on AMD64, Go 1.21, 8 cores

Serialization Overhead

Overhead vs standard serialization:
├─ Version filtering    → ~10-20% (reflection cost)
├─ Constraint evaluation → ~5% (version comparison)
├─ Format conversion    → ~0% (same underlying encoder)
└─ Total overhead       → ~15-25% typical case

---

Use Cases

This library is designed for scenarios requiring version-aware data serialization:

API Versioning

• Serve multiple API versions from a single struct definition
• Gradually deprecate fields across versions
• Maintain backward compatibility with old clients
• Example: REST API with v1, v2, v3 endpoints

Configuration Management

• Evolve configuration formats without breaking existing deployments
• Support multiple application versions with different config schemas
• Migrate configurations incrementally
• Example: Kubernetes-style versioned configs

Data Migration

• Transform data between schema versions
• Export data compatible with older application versions
• Import data from multiple schema versions
• Example: Database schema evolution

Feature Flags

• Enable/disable fields based on version-as-feature-flag
• Progressive rollout of new data fields
• A/B testing with different data structures
• Example: Gradual feature deployment

Protocol Evolution

• Maintain wire protocol compatibility across versions
• Support multiple protocol versions simultaneously
• Graceful protocol upgrades
• Example: gRPC/Protobuf-style versioning

---

Quick Start

Basic Version Filtering

Control field visibility based on version:

package main

import (
    "encoding/json"
    "fmt"
    "github.com/nabbar/golib/retro"
)

type User struct {
    Version  string `json:"version"`
    Name     string `json:"name"`
    Email    string `json:"email" retro:">=v1.0.0"`
    Phone    string `json:"phone" retro:">=v2.0.0"`
    LegacyID string `json:"legacy_id" retro:"<v2.0.0"`
}

func main() {
    // v1.5.0: includes Name, Email, LegacyID (no Phone)
    user := User{
        Version:  "v1.5.0",
        Name:     "Alice",
        Email:    "alice@example.com",
        Phone:    "555-1234",
        LegacyID: "old-123",
    }
    
    model := retro.Model[User]{Struct: user}
    data, _ := json.Marshal(model)
    fmt.Println(string(data))
    // Output: {"version":"v1.5.0","name":"Alice","email":"alice@example.com","legacy_id":"old-123"}
}

Version Ranges

Define fields valid for specific version ranges:

type Config struct {
    Version    string `json:"version"`
    Host       string `json:"host"`
    
    // Only in v1.x.x
    OldTimeout int `json:"old_timeout" retro:">=v1.0.0,<v2.0.0"`
    
    // Only in v2.x.x and above
    NewTimeout int `json:"new_timeout" retro:">=v2.0.0"`
    
    // Specific versions only
    BetaFeature string `json:"beta_feature" retro:"v1.5.0,v1.6.0"`
}

func main() {
    // v1.8.0: includes Host, OldTimeout (no NewTimeout, no BetaFeature)
    cfg := Config{
        Version:     "v1.8.0",
        Host:        "localhost",
        OldTimeout:  30,
        NewTimeout:  60,
        BetaFeature: "enabled",
    }
    
    model := retro.Model[Config]{Struct: cfg}
    data, _ := json.Marshal(model)
    fmt.Println(string(data))
    // Output: {"version":"v1.8.0","host":"localhost","old_timeout":30}
}

Multiple Formats

Use the same struct with different serialization formats:

package main

import (
    "encoding/json"
    "github.com/nabbar/golib/retro"
    "github.com/pelletier/go-toml"
    "gopkg.in/yaml.v3"
)

type Data struct {
    Version string `json:"version" yaml:"version" toml:"version"`
    Field1  string `json:"field1" yaml:"field1" toml:"field1" retro:">=v1.0.0"`
    Field2  string `json:"field2" yaml:"field2" toml:"field2" retro:">=v2.0.0"`
}

func main() {
    data := Data{Version: "v1.5.0", Field1: "value1", Field2: "value2"}
    model := retro.Model[Data]{Struct: data}
    
    // JSON
    jsonData, _ := json.Marshal(model)
    
    // YAML
    yamlData, _ := yaml.Marshal(model)
    
    // TOML
    tomlData, _ := toml.Marshal(model)
    
    // All formats respect version constraints identically
}

Standard Mode

Bypass version filtering when needed:

func main() {
    user := User{
        Version: "v1.0.0",
        Name:    "Bob",
        Email:   "bob@example.com",
        Phone:   "555-5678",
    }
    
    // Standard mode: serialize all fields regardless of version
    model := retro.Model[User]{
        Struct:   user,
        Standard: true,  // Disable version filtering
    }
    
    data, _ := json.Marshal(model)
    // Output includes all fields, version constraints ignored
}

---

Best Practices

1. Version Field

✅ DO: Always include a Version field

type MyStruct struct {
    Version string `json:"version"`  // Required for version filtering
    Field1  string `json:"field1" retro:">=v1.0.0"`
}

❌ DON'T: Omit the Version field

type MyStruct struct {
    // Missing Version field - filtering won't work
    Field1 string `json:"field1" retro:">=v1.0.0"`
}

2. Version Constraints

✅ DO: Use clear, specific constraints

// Good: Clear version boundaries
Field1 string `retro:">=v1.0.0"`           // From v1.0.0 onwards
Field2 string `retro:">=v1.0.0,<v2.0.0"`  // v1.x.x only
Field3 string `retro:"<v2.0.0"`            // Before v2.0.0

❌ DON'T: Use ambiguous or conflicting constraints

// Bad: Conflicting constraints
Field1 string `retro:">=v2.0.0,>=v1.0.0"`  // Duplicate operators
Field2 string `retro:">=v2.0.0,<v1.0.0"`   // Impossible range

3. Format Tags

✅ DO: Include format tags for all supported formats

type Config struct {
    Version string `json:"version" yaml:"version" toml:"version"`
    Host    string `json:"host" yaml:"host" toml:"host" retro:">=v1.0.0"`
}

❌ DON'T: Mix format tags inconsistently

type Config struct {
    Version string `json:"version"`  // Missing yaml/toml tags
    Host    string `json:"host" yaml:"host" retro:">=v1.0.0"`
}

4. Omitempty

✅ DO: Use omitempty for optional fields

type User struct {
    Version string  `json:"version"`
    Name    string  `json:"name"`
    Email   *string `json:"email,omitempty" retro:">=v1.0.0"`
}

❌ DON'T: Forget omitempty for pointer/optional fields

type User struct {
    Version string  `json:"version"`
    Email   *string `json:"email" retro:">=v1.0.0"`  // Will serialize as null
}

5. Migration Strategy

✅ DO: Plan version transitions carefully

// Phase 1: Add new field for v2.0.0
type Config struct {
    Version    string `json:"version"`
    OldTimeout int    `json:"old_timeout" retro:"<v2.0.0"`
    NewTimeout int    `json:"new_timeout" retro:">=v2.0.0"`
}

// Phase 2: After migration, remove old field
type Config struct {
    Version    string `json:"version"`
    NewTimeout int    `json:"new_timeout" retro:">=v2.0.0"`
}

❌ DON'T: Remove fields without transition period

// Bad: Immediate breaking change
type Config struct {
    Version    string `json:"version"`
    NewTimeout int    `json:"new_timeout"`  // Old clients break
}

---

Testing

Test Coverage

Package                Coverage    Specs
retro                  84.2%       156 tests
────────────────────────────────────────────
Components:
- Encoding             85.4%       48 tests
- Version              100.0%      50 tests
- Model                74.4%       35 tests
- Utils                100.0%      20 tests
- Format               100.0%      3 tests

Running Tests

# Run all tests
go test ./...

# Run with coverage
go test -coverprofile=coverage.out
go tool cover -html=coverage.out

# Run with race detector
CGO_ENABLED=1 go test -race

# Verbose output
go test -v

# Specific test
go test -run TestModel_MarshalJSON

Test Categories

Version Comparison (50 tests)

• Semantic version parsing
• Operator evaluation (>=, <=, >, <, ==)
• Range constraints
• Boundary detection
• Invalid version handling

Serialization (48 tests)

• JSON marshal/unmarshal
• YAML marshal/unmarshal
• TOML marshal/unmarshal
• Standard mode
• Omitempty behavior

Model Operations (35 tests)

• Field filtering
• Version extraction
• Custom unmarshalers
• Error handling
• Edge cases

Utilities (20 tests)

• Empty value detection
• Format validation
• Helper functions

For detailed testing documentation, see TESTING.md.

---

Contributing

Contributions are welcome! Please follow these guidelines:

Code Contributions

• Do not use AI to generate package implementation code
• AI may assist with tests, documentation, and bug fixing
• All contributions must pass go test -race
• Maintain or improve test coverage (≥80%)
• Follow existing code style and patterns

Documentation

• Update README.md for new features
• Add examples for common use cases
• Keep TESTING.md synchronized with test changes
• Document all exported functions with GoDoc

Testing

• Write tests for all new features
• Test edge cases and error conditions
• Verify version constraint logic thoroughly
• Include examples in test cases

Pull Requests

• Describe the problem and solution
• Reference related issues
• Include test results and coverage
• Update documentation as needed

---

Future Enhancements

Potential improvements for future versions:

Version Constraints

• Pre-release version support (v1.0.0-alpha)
• Build metadata handling (v1.0.0+build.123)
• Wildcard versions (v1..)
• Version aliases (latest, stable)

Serialization

• XML format support
• Protocol Buffers integration
• MessagePack support
• Custom format plugins

Performance

• Constraint caching for repeated operations
• Lazy field evaluation
• Parallel field processing
• Zero-allocation mode

Features

• Field transformation functions
• Conditional default values
• Version migration helpers
• Schema validation

Tooling

• CLI tool for version analysis
• Struct version diff tool
• Migration path generator
• Documentation generator

---

AI Transparency Notice

In accordance with Article 50.4 of the EU AI Act, AI assistance has been used for testing, documentation, and bug fixing under human supervision.

---

License

MIT License - See LICENSE file for details.

---

Resources

Official Documentation

• GoDoc
• Semantic Versioning
• Go Generics

Serialization Libraries

• encoding/json - JSON support
• gopkg.in/yaml.v3 - YAML support
• github.com/pelletier/go-toml - TOML support

Related Concepts

• API Versioning Best Practices
• Backward Compatibility
• Schema Evolution

Testing

• Ginkgo - BDD testing framework
• Gomega - Matcher library